---
name: django-celery
description: Run Django jobs in the background with Celery. Covers task queues, workers, retries, schedules, and Redis or RabbitMQ.
origin: ECC
---

# Django and Celery Tasks

Use Celery to run slow Django work outside web requests.

## Use This Skill When

- Send email in the background.
- Run slow jobs.
- call an API without making the user wait.
- Run jobs at set times.
- Split a large job into small jobs.

Do not use Celery for work that must finish before the page loads.

## Setup

### 1. Install Celery

For Redis:

```bash
pip install celery redis
```

For RabbitMQ:

```bash
pip install celery
```

### 2. Set the Broker

Add this to Django settings:

```python
CELERY_BROKER_URL = "redis://localhost:6379/0"
CELERY_RESULT_BACKEND = "redis://localhost:6379/1"
CELERY_TASK_TIME_LIMIT = 300
CELERY_TASK_SOFT_TIME_LIMIT = 270
```

Keep the broker URL in an env file when it has a password.

### 3. Create the Celery App

Create `myapp/celery.py`:

```python
import os

from celery import Celery

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myapp.settings")

app = Celery("myapp")
app.config_from_object("django.conf:settings", namespace="CELERY")
app.autodiscover_tasks()
```

Add this to `myapp/__init__.py`:

```python
from .celery import app as celery_app

__all__ = ("celery_app",)
```

Change `myapp` to the name of your Django project.

### 4. Write a Task

Create `emails/tasks.py`:

```python
from celery import shared_task
from django.core.mail import send_mail


@shared_task(
    autoretry_for=(Exception,),
    retry_backoff=True,
    retry_jitter=True,
    retry_kwargs={"max_retries": 5},
)
def send_welcome_email(user_id):
    from django.contrib.auth import get_user_model

    user = get_user_model().objects.get(pk=user_id)

    send_mail(
        subject="Welcome",
        message="Thanks for joining.",
        from_email=None,
        recipient_list=[user.email],
    )
```

Pass IDs, text, numbers, lists, or maps to tasks. Do not pass Django model objects.

### 5. Start a Worker

```bash
celery -A myapp worker --loglevel=INFO
```

Run at least one worker in each live app setup.

## Queue a Task

```python
from emails.tasks import send_welcome_email

send_welcome_email.delay(user.id)
```

If the task depends on saved data, queue it after the database change is done:

```python
from django.db import transaction

transaction.on_commit(
    lambda: send_welcome_email.delay(user.id)
)
```

This keeps the worker from reading data before it is saved.

## Run Tasks on a Schedule

Add this to `myapp/celery.py`:

```python
from celery.schedules import crontab

app.conf.beat_schedule = {
    "send-report-daily": {
        "task": "reports.tasks.send_report",
        "schedule": crontab(hour=9, minute=0),
    },
}
```

Start the scheduler:

```bash
celery -A myapp beat --loglevel=INFO
```

Run only one scheduler. Two schedulers may send the same task twice.

Set the time zone in Django settings:

```python
TIME_ZONE = "America/Chicago"
USE_TZ = True
CELERY_TIMEZONE = TIME_ZONE
```

## Concrete Example

Queue an image resize after an upload is saved:

```python
# photos/tasks.py
from celery import shared_task


@shared_task(bind=True, max_retries=3)
def resize_photo(self, photo_id):
    from .models import Photo

    try:
        photo = Photo.objects.get(pk=photo_id)
        photo.make_thumbnail()
    except Photo.DoesNotExist:
        return
    except OSError as error:
        raise self.retry(exc=error, countdown=30)
```

```python
# photos/views.py
from django.db import transaction
from django.shortcuts import redirect

from .forms import PhotoForm
from .tasks import resize_photo


def upload_photo(request):
    form = PhotoForm(request.POST, request.FILES)

    if form.is_valid():
        photo = form.save()
        transaction.on_commit(
            lambda: resize_photo.delay(photo.id)
        )
        return redirect("photo-detail", pk=photo.id)
```

The page can return at once. The worker resizes the photo later.

## Error Rules

- Retry short network or broker errors.
- Do not retry bad input.
- Set a retry limit.
- Set a time limit for slow tasks.
- Log the task name and record ID.
- Keep failed tasks so they can be checked or run again.
- Use a dead-letter queue when the broker supports it.
- Do not log passwords, tokens, or private user data.

## Safe Task Rules

A task may run more than once. Write it so a second run is safe.

Before sending money, email, or a web request, check if the work was already done. Use a unique key or a saved status when needed.

Keep tasks small. Pass file paths or record IDs instead of large files.

Do not wait for one task inside another task with `.get()`. Queue the next task instead.

## Checks Before Release

- [ ] The broker is running.
- [ ] The worker can load Django settings.
- [ ] The task is found by Celery.
- [ ] Retries have a limit.
- [ ] Slow tasks have a time limit.
- [ ] Tasks are safe to run twice.
- [ ] Tasks are queued after database commit.
- [ ] Only one scheduler is running.
- [ ] Worker logs are saved.
- [ ] Failed jobs can be checked and run again.
- [ ] Workers stop in a safe way during release.