BackgroundTasks in FastAPI is the inviting trap. It’s right there in the framework; it looks like a job queue. It isn’t. This post is when it’s enough and when you need a real queue.

What BackgroundTasks actually does

from fastapi import BackgroundTasks

@app.post("/signup")
async def signup(payload: SignUp, bg: BackgroundTasks):
    user = await create_user(payload)
    bg.add_task(send_welcome_email, user.id)
    return user

After the response is sent, FastAPI runs the queued task in the same process. If the process dies between response and task running: task lost.

When BackgroundTasks is fine

  • The task completes in < 100ms.
  • Loss is acceptable (analytics ping, opportunistic cache warm).
  • No retry needed.
  • No coordination across processes.

A genuinely fire-and-forget call.

When you need a real queue

  • The task takes seconds (image processing, ML inference, third-party API).
  • Loss is not acceptable (welcome email, billing event).
  • You need retries.
  • The task should run on a worker pool, not your API process.
  • You want scheduled / cron-like jobs.

For these: arq, Dramatiq, Taskiq, Celery .

Why this matters

A common bug pattern:

@app.post("/register")
async def register(payload, bg: BackgroundTasks):
    user = await create_user(payload)
    bg.add_task(send_welcome_email, user.id)
    return user

User signs up. Process restarts (deploy, OOM, autoscaler). Email never sent. User confused. Support ticket.

With a real queue:

@app.post("/register")
async def register(payload):
    user = await create_user(payload)
    await arq_pool.enqueue_job("send_welcome_email", user.id)
    return user

Job persists. Worker picks it up. Retries on failure. Email sent.

Migration path

  1. Pick a queue (arq for async-native ).
  2. Define the task in worker module.
  3. Replace bg.add_task(fn, ...) with await pool.enqueue_job("fn_name", ...).
  4. Run a worker process alongside the API.

Hours of work. Permanent reliability win.

Hybrid pattern

For a service with mixed needs:

@app.post("/widget")
async def create_widget(payload, bg: BackgroundTasks):
    widget = await save_widget(payload)
    
    # OK to lose: cache warm
    bg.add_task(warm_cache, widget.id)
    
    # Must run: email + webhook
    await arq_pool.enqueue_job("send_widget_email", widget.id)
    await arq_pool.enqueue_job("fire_webhook", widget.id)
    
    return widget

Use the right tool per task. Fire-and-forget for cheap; queue for important.

What to put in workers

Things that benefit from being out of the request path:

  • Email sending.
  • Webhook delivery.
  • Image / video processing.
  • LLM calls > 1 second.
  • DB cleanup / aggregation.
  • Periodic sync jobs.
  • Anything that retries.

For LLM-specific patterns some calls stream to the user; others run async.

Common mistakes

1. BackgroundTasks for “this should always run”

It doesn’t. Migrate.

2. Async work that blocks

bg.add_task running a CPU-heavy task → API process stalls. CPU work goes to a separate worker pool.

3. No idempotency

Webhooks retry. Without idempotency , retries duplicate.

4. No DLQ

Tasks fail forever. No human notices. Every queue should have a dead-letter and review process.

5. No observability

You don’t know if tasks are running. Track per-task: queued, running, completed, failed. See LLM Observability — same patterns apply.

Read this next

If you want a FastAPI + arq + Postgres outbox starter, it’s at rajpoot.dev .

Building something AI-, backend-, or data-heavy and want a second pair of eyes? I do consulting and freelance work — see my projects and ways to reach me at rajpoot.dev .