When a website or application becomes popular, a single server quickly becomes a bottleneck. The solution is to add more servers, a process known as horizontal scaling. But this creates a new problem: how do you distribute incoming requests among all these servers? You need a traffic cop.

That's the job of a load balancer. It sits in front of your server farm and directs incoming traffic to the various servers that can fulfill the request. Its goal is to prevent any single server from becoming overwhelmed, ensuring a fast, responsive experience for all users. Just like a dispatcher at a taxi company, it needs a strategy for sending the next ride to the right driver.

But not all traffic direction is the same. The intelligence of a load balancer depends on what information it's allowed to see. This distinction is best understood by looking at where the load balancer operates in the network stack.

Load balancers make decisions at different network layers, primarily Layer 4 (the transport layer) and Layer 7 (the application layer). The layer determines how much context the load balancer has to make its routing decision.

A Layer 4 load balancer operates at the transport level. It sees network information like source and destination IP addresses and ports. It routes traffic based on this limited information, forwarding packets to a server without inspecting their content. Think of it as a mail sorter that only reads the address on the envelope. It's incredibly fast and efficient because it's not opening the mail.

A Layer 7 load balancer operates at the application level. It can inspect the content of the request itself, such as HTTP headers, cookies, and URL paths. This allows for far more intelligent routing. For example, it could route requests for /api/video to a fleet of powerful video-processing servers, while sending requests for /api/user_profile to a different set of servers optimized for database queries. It's slower than a Layer 4 balancer, but much more flexible.

The choice between them is a classic engineering trade-off: speed versus intelligence. Many modern systems use a combination, with a fast Layer 4 load balancer handling initial traffic distribution to groups of more specialized Layer 7 balancers.

Once a load balancer decides to forward a request, it needs an algorithm to pick the specific server. There are several common strategies.

Weighted Round Robin: This is an improvement on the simple round-robin method (1 -> 2 -> 3 -> 1...). If you have servers with different capacities, you can assign them a "weight." A server with a weight of 3 will receive three times as many requests as a server with a weight of 1 before the cycle repeats. It’s a simple way to account for uneven hardware.

Least Connections: This dynamic approach sends the next request to the server that currently has the fewest active connections. It's great for situations where some requests take much longer to process than others, as it naturally prevents a single server from getting bogged down with slow tasks.

IP Hash: The load balancer creates a hash from the client's IP address to determine which server to send the request to. The key benefit is stickiness. As long as the pool of servers doesn't change, a user will consistently be sent to the same server. This is useful for applications that store session state locally on the web server. The downside is that if a server is added or removed, the hash mapping changes, and most users will be re-routed to new servers. This can be disruptive.

Distributing traffic is only half the battle. A load balancer is also the first line of defense for high availability (HA). It needs to know which servers are healthy and which are not, so it doesn't send users to a dead end.

This is done through health checks. The load balancer periodically pings each server in its pool to check if it's operational. A simple health check might just see if the server responds on a specific port. A more sophisticated one might request a specific URL (like /health) and expect a 200 OK status code in response.

If a server fails a health check, the load balancer temporarily removes it from the pool of available servers and stops sending traffic its way. Once the server starts passing health checks again, it's automatically added back into rotation.

A related concept is the circuit breaker pattern. If a backend service is failing, repeatedly sending requests to it can make the situation worse and exhaust resources on the calling service. A circuit breaker in the load balancer (or application) can detect a high failure rate, "trip," and immediately fail subsequent requests without even trying to contact the struggling service. After a timeout, it will allow a limited number of test requests through. If they succeed, the circuit closes and normal operation resumes.

The load balancers we've discussed so far, like Nginx, are typically used to manage traffic within a single data center. But what if your users are spread all over the world? Sending a user in Japan to a server in Virginia is not ideal for performance. This is where Global Server Load Balancing (GSLB) comes in.

GSLB works at the DNS level. When a user's browser looks up your domain (e.g., www.example.com), the GSLB-enabled DNS server doesn't just return a single IP address. It returns the IP address of a data center that is geographically closest to the user, or one that is currently healthiest or has the lowest load. This directs the user's traffic to the nearest and best-performing entry point of your entire global infrastructure before the request even leaves their region. From there, local load balancers take over to manage traffic within that data center.

By combining global and local load balancing, organizations can build systems that are not only scalable but also resilient and performant for users anywhere on the planet.

These strategies form the foundation of how modern, large-scale applications handle immense traffic loads reliably. By intelligently routing requests at both a global and local level, and by constantly monitoring the health of the underlying servers, load balancers are the unsung heroes of the internet's infrastructure.