Notes from production
MessagingArchitectureEvent-Driven

Message brokers: one mental model

Kafka, NSQ, NATS, RabbitMQ, AWS — five message brokers, and every time you move from one to the next it feels like starting over. The names change, the docs are thick, and the same idea shows up under a different word in each system.

But at the layer that actually matters, all five run the same idea. Once you see the model, the five brokers stop feeling like five different things.

The one model

A producer publishes a message to a topic. Every consumer group that subscribes gets its own copy — that's the fan-out layer. Inside a group, several workers share that copy round-robin, each message handled once.

Those two layers are the whole thing: fan-out to groups (who gets a copy) versus the split across workers (who processes it). Kafka calls a group a consumer group, NSQ calls it a channel, RabbitMQ uses a queue, AWS uses an SQS queue — different names, same idea.

Where they actually differ

The differences live in the details, not the model. Kafka caps the workers per group at its partition count — the rest sit idle; NSQ, NATS, and RabbitMQ don't. Some push messages to consumers, some let consumers pull. Some can replay history, some can't. RabbitMQ adds an exchange to choose which queues match.

See it move

Theory lands fastest when you can move it. In the interactive guide, add consumers to a Kafka group past its 4 partitions — the fifth goes idle instantly. Switch to NSQ at the same count — everyone keeps working, the load splits evenly. One control, two opposite behaviors — that's the whole partition lesson in a single click.

Open the interactive guide →

Sources

  1. Apache Kafka — Concepts and Terminology
  2. RabbitMQ — AMQP 0-9-1 Model Explained
  3. NATS — JetStream
  4. NSQ — Design