When you send an email, you don't expect an instant reply. Your email client has to periodically check a server for new mail. This model, called polling, works fine for email but would feel incredibly slow for a chat app. We expect messages to appear the moment they're sent.

To achieve this, real-time systems need a persistent, two-way connection between your device and the server. Think of it like an open phone line instead of sending a series of letters. This is where WebSockets come in. A WebSocket connection starts as a standard web request but then upgrades to a dedicated, low-latency channel that stays open, allowing both the client and server to send data to each other at any time.

With millions of users online, servers can't manage direct connections between every single person. It would be chaos. Instead, when you send a message, it goes to a central sorting facility on the backend. This is the role of a message broker.

A message broker is like a highly efficient post office. It receives incoming messages from senders (called producers) and routes them to the correct recipients (called consumers). This decouples the services. The sender doesn't need to know if the recipient is online or even what device they're using; it just hands the message to the broker, trusting it will be delivered.

Brokers use two main models for routing messages: queues and topics.

• Queue (Point-to-Point): Imagine a single-file line. A message is placed in the queue and delivered to exactly one consumer. This ensures each message is processed only once, which is perfect for tasks like sending a password reset notification.
• Topic (Publish/Subscribe): This is like a community bulletin board. A producer posts a message to a topic, and every consumer subscribed to that topic gets a copy. This is ideal for group chats, where one message needs to go to multiple people.

In a conversation, message order is critical. "Want to get lunch?" followed by "Just kidding" has a very different meaning than the reverse. But in a distributed system, messages can take different paths and arrive out of order. How do systems guarantee you read things in the sequence they were sent?

This is achieved through a few strategies. First, messages are often assigned sequence numbers. The recipient's app can then reassemble the messages in the correct order before displaying them. Second, many message brokers use a concept called partitioning. A conversation between two people can be assigned to a specific partition, which acts like a dedicated, ordered log. All messages within that partition are guaranteed to be processed in the order they were received.

What happens when a messaging service becomes massively popular? The architecture must be built for scalability and fault tolerance from the start. Scalability means the system can handle a growing number of users without slowing down. Fault tolerance means it can withstand parts of the system failing without a complete outage.

Instead of running on one giant server, these systems are distributed across thousands of smaller, commodity machines. If one machine fails, others are ready to take over its workload. Message brokers are also designed this way. Data is replicated, or copied, across multiple machines. If the primary machine holding a message log goes down, a replica can step in immediately, ensuring no data is lost and service is not interrupted.

This combination of persistent connections, decoupled message routing, and a distributed, fault-tolerant design is what allows modern chat applications to feel so seamless and reliable, no matter how many people are using them.

This architecture ensures that your messages are delivered quickly, in order, and reliably, forming the invisible backbone of modern communication.