Distributing traffic evenly is a good start, but high-traffic systems need more intelligent strategies. Simple round-robin routing treats all servers as equals and has no awareness of the requests themselves. This breaks down when servers have different capacities, when user sessions need to be maintained, or when geography matters.

To make smarter routing decisions, a load balancer needs more context. The first place it can get that context is from the network layer where the request is processed.

Load balancers operate at different layers of the OSI model, primarily Layer 4 (Transport) and Layer 7 (Application). The layer determines how much information the balancer has about incoming traffic, which dictates the complexity of its routing logic.

Layer 4 (L4) Load Balancing An L4 load balancer works at the transport layer. It sees network information like IP addresses and TCP/UDP ports, but not the content of the messages. It forwards packets to servers without inspecting them. This makes L4 balancing extremely fast and network-efficient.

Layer 7 (L7) Load Balancing An L7 load balancer works at the application layer. It can inspect the content of the request itself, such as HTTP headers, URLs, cookies, and message bodies. This allows for much more sophisticated routing. For example, it can route requests to different backend services based on the URL path (/api vs. /images) or read a user's session cookie to send them to the same server they used previously.

Choosing between them is a trade-off. L4 offers raw speed, while L7 provides intelligent, content-aware routing. Many modern systems use a combination, with a fast L4 balancer handling initial traffic distribution to a fleet of more sophisticated L7 balancers.

Beyond the routing layer, the algorithm itself can be improved to account for server health and capacity.

Weighted Round Robin This algorithm is a simple enhancement to basic round-robin. Each server in the pool is assigned a weight, typically based on its processing power or memory. A server with a weight of 3 will receive three times as many requests as a server with a weight of 1. This is ideal for environments with heterogeneous hardware.

Least Connections This method is dynamic. The load balancer tracks the number of active connections to each backend server and sends the next request to the server with the fewest connections. It's particularly effective for applications where sessions have varying lengths, as it prevents a few long-running requests from tying up one server while others sit idle.

For global applications like Twitter or Netflix, routing users to the nearest datacenter is critical for reducing latency. This is often handled by GSLB (Global Server Load Balancing).

Unlike traditional load balancers that distribute traffic to servers, GSLB distributes traffic to other load balancers or data centers. It most commonly works via DNS. When a user's browser requests the IP for twitter.com, the DNS service can provide the IP address of the datacenter that is geographically closest to the user. This ensures a faster experience and provides a simple way to route around a datacenter outage.

One of the biggest challenges in a distributed system is managing data consistency, especially in caches. If you have ten servers and a user's data is cached on Server 1, you want subsequent requests from that user to hit Server 1. But what happens if you add or remove servers? With a simple hashing algorithm like server_index = hash(user_id) % N, where N is the number of servers, adding or removing a server changes the result for almost every user. This would invalidate most of your cache, causing a massive surge in database queries.

This is the problem consistent hashing solves.