What is BullMQ? Redis-Backed Queues for Node.js Applications
BullMQ is a high-performance background job and message queue library for NodeJS and TypeScript, built directly on top of Redis. It coordinates asynchronous task processing, distributed workers, retries with backoff, and scheduled jobs across microservices.
Why It Matters in Production
In Node.js, running CPU-intensive operations (image processing, PDF compilation, AI model inference, bulk emails) directly inside an Express or Next.js API route blocks the single-threaded event loop. BullMQ offloads these operations to background worker instances, maintaining sub-50ms API response times.
How It Works Under the Hood
Producers add JSON job payloads to named queues. BullMQ uses Redis data structures (Lists, Hashes, Sorted Sets) and Lua scripts to transition jobs through states: waiting -> active -> completed (or failed). Dedicated worker processes pop jobs, renew locks via heartbeats, and report progress.
Code Architecture Example
// BullMQ Worker Instance
import { Worker } from 'bullmq';
const worker = new Worker('pdf-transcoder', async (job) => {
console.log(`Transcoding document for ${job.data.documentId}`);
await transcodePdf(job.data.filePath);
return { status: 'success' };
}, { connection: { host: 'localhost', port: 6379 } });Common Architectural Pitfalls
- Sharing the same Redis cache instance for BullMQ and cache-aside storage: cache eviction policies (such as allkeys-lru) can silently evict active job keys.
- Failing to handle unhandled exceptions in worker handlers, leading to unacknowledged or stalled jobs.
BullMQ with Node.js and Redis: Production Background Jobs
Explore the complete, step-by-step tutorial published on Aditya's engineering blog.