Why background jobs outlive the request that started them
A request should answer fast. Anything that doesn't need to block the response belongs in a queue.
Every request handler I write starts the same way: figure out what actually has to finish before the response goes out, and push everything else somewhere else.
Sending a welcome email, resizing an upload, recalculating a seller's rating — none of that needs the client to wait. It needs to happen reliably, which is a different problem than happening immediately.
That's where a queue like BullMQ earns its keep. The HTTP request enqueues a job and returns in milliseconds. A worker process, running separately from the web server, picks the job up, retries it if it fails, and reports back through whatever channel makes sense — a webhook, a status row, a socket event.
The failure mode I've seen most often is treating the queue as an afterthought: no retry policy, no dead-letter handling, no idea what happens if the worker crashes mid-job. Once a system has background jobs, the jobs are part of the architecture, not a implementation detail you bolt on later.
The rule I keep coming back to: if losing it silently would hurt, it needs a queue with retries and monitoring — not a fire-and-forget promise inside a request handler.
Comments (1)
- DreySep 18, 3:52 PM
ganda po