What is a Message Queue

A message queue is a data structure that stores messages sent by a producer until a consumer retrieves and processes them. It acts as a buffer between components, enabling asynchronous communication.

How it works

A producer enqueues a message. One consumer dequeues it and processes it. Unlike pub/sub, each message is delivered to exactly one consumer. If multiple consumers (workers) read from the same queue, they compete for messages — this is the competing consumers pattern.

Messages are typically processed in FIFO order. After processing, the consumer sends an acknowledgment and the broker removes the message. If the consumer fails, the message returns to the queue for redelivery.

Where it is used

RabbitMQ, Amazon SQS, Redis lists (with LPUSH/BRPOP), and Kafka consumer groups all provide queue semantics. Common use cases include async task processing (image transcoding, email sending), rate smoothing (absorbing traffic spikes), and work distribution across multiple workers.

Why it matters

Queues decouple producers from consumers in time — the producer does not wait for the consumer. They absorb bursts, provide resilience (messages survive consumer crashes), and scale horizontally by adding more workers.

For the full explanation including acknowledgments, dead letter queues, and backpressure, see How Message Queues Work.