/backend-engineer

Idempotent Worker Pattern

Process queue messages safely with retry tolerance. Same message processed twice produces same result.

Pattern

// Pseudocode — send notification email from queue

function processNotification(message):
  jobId = message.id

  // 1. Check if already processed
  existing = db.find("processed_jobs", { jobId })
  if existing:
    log.info("Already processed", { jobId })
    return  // ack message

  // 2. Execute work
  try:
    user = db.find("users", { id: message.userId })
    email.send({
      to: user.email,
      template: message.template,
      data: message.data
    })

    // 3. Mark as processed (atomic with unique constraint)
    db.insert("processed_jobs", {
      jobId,
      processedAt: now(),
      result: "sent"
    })

    log.info("Notification sent", { jobId, userId: message.userId })

  catch error:
    if error is DuplicateKeyError:
      // Another worker processed it — safe to ignore
      return
    // Transient error — let queue retry
    throw error

Rules

• Use unique constraint on job/message ID as safety net.
• Handle DuplicateKeyError gracefully — means another worker won.
• Only retry on transient errors. Permanent failures go to DLQ.
• Log: job ID, attempt number, result, duration.
• Set max retries (5–10) and backoff delay.