Index of every page on this site: llms.txt
All skills
Run it yourself

Run Django jobs in the background with Celery. Covers task queues, workers, retries, schedules, and Redis or RabbitMQ.

  • 1 file
  • 5.1 KB
  • Updated 2 months ago
  • GitHub
Use this Skill: https://skilld.dev/gh/agenticluke/django-celery-jobs-plus/skill

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

SKILL.md

≈31 tokens for metadata: the name and description. ≈1.3k when used: this file.

Description uses 1.7% of example budget

Before choosing a Skill, your Agent reads its name and description. All available Skills share that space.

  • A shorter description leaves more room for other Skills. This entry exceeds our 1% size suggestion.

In our Claude Code example, all Skill names and descriptions share 8,000 characters. This Skill uses ≈134 characters, or 1.7%.

The 1% threshold is a size suggestion. Longer descriptions can still fit.

Your model, settings, and other Skills decide how much text your Agent can read.

Example settings and source

The example uses a 200k-token context and default Claude Code settings. The count includes the name, description, separators, and when_to_use when present. Codex also counts local file paths.

Skit's source and limits: Codex 0.160.1, Claude Code 2.1.292.

Django and Celery Tasks

Use Celery to run slow Django work outside web requests.

Use This Skill When

  • Send email in the background.
  • Run slow jobs.
  • call an API without making the user wait.
  • Run jobs at set times.
  • Split a large job into small jobs.

Do not use Celery for work that must finish before the page loads.

Setup

1. Install Celery

For Redis:

pip install celery redis

For RabbitMQ:

pip install celery

2. Set the Broker

Add this to Django settings:

CELERY_BROKER_URL = "redis://localhost:6379/0"
CELERY_RESULT_BACKEND = "redis://localhost:6379/1"
CELERY_TASK_TIME_LIMIT = 300
CELERY_TASK_SOFT_TIME_LIMIT = 270

Keep the broker URL in an env file when it has a password.

3. Create the Celery App

Create myapp/celery.py:

import os

from celery import Celery

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myapp.settings")

app = Celery("myapp")
app.config_from_object("django.conf:settings", namespace="CELERY")
app.autodiscover_tasks()

Add this to myapp/__init__.py:

from .celery import app as celery_app

__all__ = ("celery_app",)

Change myapp to the name of your Django project.

4. Write a Task

Create emails/tasks.py:

from celery import shared_task
from django.core.mail import send_mail


@shared_task(
    autoretry_for=(Exception,),
    retry_backoff=True,
    retry_jitter=True,
    retry_kwargs={"max_retries": 5},
)
def send_welcome_email(user_id):
    from django.contrib.auth import get_user_model

    user = get_user_model().objects.get(pk=user_id)

    send_mail(
        subject="Welcome",
        message="Thanks for joining.",
        from_email=None,
        recipient_list=[user.email],
    )

Pass IDs, text, numbers, lists, or maps to tasks. Do not pass Django model objects.

5. Start a Worker

celery -A myapp worker --loglevel=INFO

Run at least one worker in each live app setup.

Queue a Task

from emails.tasks import send_welcome_email

send_welcome_email.delay(user.id)

If the task depends on saved data, queue it after the database change is done:

from django.db import transaction

transaction.on_commit(
    lambda: send_welcome_email.delay(user.id)
)

This keeps the worker from reading data before it is saved.

Run Tasks on a Schedule

Add this to myapp/celery.py:

from celery.schedules import crontab

app.conf.beat_schedule = {
    "send-report-daily": {
        "task": "reports.tasks.send_report",
        "schedule": crontab(hour=9, minute=0),
    },
}

Start the scheduler:

celery -A myapp beat --loglevel=INFO

Run only one scheduler. Two schedulers may send the same task twice.

Set the time zone in Django settings:

TIME_ZONE = "America/Chicago"
USE_TZ = True
CELERY_TIMEZONE = TIME_ZONE

Concrete Example

Queue an image resize after an upload is saved:

# photos/tasks.py
from celery import shared_task


@shared_task(bind=True, max_retries=3)
def resize_photo(self, photo_id):
    from .models import Photo

    try:
        photo = Photo.objects.get(pk=photo_id)
        photo.make_thumbnail()
    except Photo.DoesNotExist:
        return
    except OSError as error:
        raise self.retry(exc=error, countdown=30)
# photos/views.py
from django.db import transaction
from django.shortcuts import redirect

from .forms import PhotoForm
from .tasks import resize_photo


def upload_photo(request):
    form = PhotoForm(request.POST, request.FILES)

    if form.is_valid():
        photo = form.save()
        transaction.on_commit(
            lambda: resize_photo.delay(photo.id)
        )
        return redirect("photo-detail", pk=photo.id)

The page can return at once. The worker resizes the photo later.

Error Rules

  • Retry short network or broker errors.
  • Do not retry bad input.
  • Set a retry limit.
  • Set a time limit for slow tasks.
  • Log the task name and record ID.
  • Keep failed tasks so they can be checked or run again.
  • Use a dead-letter queue when the broker supports it.
  • Do not log passwords, tokens, or private user data.

Safe Task Rules

A task may run more than once. Write it so a second run is safe.

Before sending money, email, or a web request, check if the work was already done. Use a unique key or a saved status when needed.

Keep tasks small. Pass file paths or record IDs instead of large files.

Do not wait for one task inside another task with .get(). Queue the next task instead.

Checks Before Release

  • The broker is running.
  • The worker can load Django settings.
  • The task is found by Celery.
  • Retries have a limit.
  • Slow tasks have a time limit.
  • Tasks are safe to run twice.
  • Tasks are queued after database commit.
  • Only one scheduler is running.
  • Worker logs are saved.
  • Failed jobs can be checked and run again.
  • Workers stop in a safe way during release.

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No third-party reports yet.

Signed by skilld at 54ba74b. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub last month.

Steadyupdated 2 months ago
origin
ECC

README badge

README badge for agenticluke/django-celery-jobs-plus