Load Balancers Demystified: Layer 4 vs Layer 7, Algorithms and Health Checks
What a load balancer decides, how Layer 4 and Layer 7 differ, which algorithm fits which traffic, and the health-check mistakes that cause outages.
On this page
Adding a second server is easy. Deciding which of the two a request goes to, and noticing when one of them is broken, is the actual problem. That job belongs to the load balancer, and most outages involving one come from the details below.
What it does
| Job | How |
|---|---|
| Distribute traffic | Spread connections or requests across a pool |
| Keep the service available | Stop sending traffic to unhealthy backends |
| Improve performance | Avoid overloading one server while others idle |
| Enable scaling | Add or remove backends without clients noticing |
Clients talk to one stable address, and the pool behind it changes freely. That stable front is what makes rolling deploys and autoscaling possible.
Layer 4 versus Layer 7
| Need | Choose |
|---|---|
| Raw throughput, non-HTTP protocol, TLS passthrough | Layer 4 |
| Path or host routing, canary releases, header-based rules | Layer 7 |
| WAF, authentication at the edge, caching | Layer 7 |
| Both | Layer 4 in front for scale, Layer 7 behind for logic |
Many real systems stack them: a Layer 4 tier spreads connections over a fleet of Layer 7 proxies, which then route requests to services.
Hardware, software, cloud
- Hardware appliances are physical devices with high throughput and a price to match. Common in older data centres.
- Software balancers such as NGINX or HAProxy run on ordinary servers or VMs. Flexible and cheap, and you operate them.
- Cloud-managed balancers (AWS Elastic Load Balancing, Google Cloud Load Balancing, Azure Load Balancer) are distributed across zones, scale automatically and are billed by use. The default choice when you are already in a cloud.
Choosing an algorithm
upstream app {
least_conn; # or: round robin (default), ip_hash, hash $key consistent
server 10.0.0.11 weight=3; # bigger machine gets more
server 10.0.0.12;
server 10.0.0.13 backup; # only used if the others fail
}
| Algorithm | Good for | Watch out for |
|---|---|---|
| Round robin | Uniform, short requests | Ignores how busy each backend is |
| Weighted round robin | Mixed server sizes | Weights go stale when hardware changes |
| Least connections | Requests of uneven duration, long-lived connections | Counts connections, not CPU cost |
| IP hash or consistent hash | Cache locality, sticky sessions | Hot keys concentrate load on one node |
| Random with two choices | Large pools, simple and surprisingly even | Needs load information per backend |
Stickiness is a smell. If a user must hit the same server every time, state is living on that server, and a failure loses it. Move session state into a shared store and let any backend serve any request.
Health checks: where most incidents hide
- Check readiness, not just liveness
“The process is running” is not “this server can serve requests”. A good check exercises what the request needs: a cheap dependency call, a loaded cache, an open connection pool.
- Keep the check cheap and bounded
A check that runs a heavy query will itself overload a struggling database. Give it a short timeout and a cached result.
- Remove slowly, restore carefully
Require several consecutive failures before ejecting a backend, and several successes before adding it back, so one blip does not flap the whole pool.
- Never let every backend fail the check at once
If a shared dependency breaks, all checks fail and the balancer has nowhere to send traffic. Many systems fail open (route to everything) once too many backends are marked down, rather than serve nothing.
When one server is overloaded and slows down, a naive balancer keeps feeding it, which makes things worse. Use timeouts, concurrency limits per backend, and load shedding that rejects excess work quickly, so the service degrades instead of collapsing.
Global load balancing
One balancer serves one location. To survive losing a region, or to send users to the nearest one, add global server load balancing, typically through DNS with health-aware answers or anycast routing. It adds a layer of redundancy and cuts latency for distant users. Expect DNS caching to delay failover by the record’s TTL, so keep it short for critical services.
Make the balancer itself survive
Do
Run the balancer redundantly or use a managed one, test failover by actually killing a backend and a balancer, enable access logs with the chosen backend and latency per request, and alert on rising 5xx and unhealthy-host counts.
Don't
Put one self-managed NGINX box in front of a highly available fleet. You have moved the single point of failure, not removed it.
Takeaway
Pick Layer 4 for speed and protocol freedom, Layer 7 for routing intelligence, and often both. Choose the simplest algorithm that keeps load even, make health checks measure real readiness, and make the balancer itself redundant. Most “load balancer problems” turn out to be health-check problems.
Frequently asked questions
What is the difference between Layer 4 and Layer 7 load balancing?
A Layer 4 balancer routes by IP address and TCP or UDP port without reading the payload, so it is fast and protocol-agnostic. A Layer 7 balancer understands HTTP, so it can route by path, host or header, terminate TLS, rewrite requests and apply per-request policies, at higher cost per connection.
Which load balancing algorithm should I use?
Round robin suits uniform, short requests. Least connections suits requests with uneven duration. Consistent hashing or IP hash suits cases that need affinity, such as caches. Weighted variants help when servers differ in capacity. Start simple and change only after measuring uneven load.
What is a health check and why does it matter?
A periodic probe that tells the balancer whether a backend can take traffic. Without it, a dead server keeps receiving requests. A bad one is just as harmful: a check that only tests the process is up will happily route traffic to a server whose database connection is broken.
Is a load balancer itself a single point of failure?
It can be. Run balancers in redundant pairs or use a managed cloud balancer, which is distributed across zones for you. Cross-region redundancy needs DNS-level or global load balancing on top, so a whole region failing does not take the service down.



/* Comments */
Comments are offline right now — we reconnect automatically, nothing is lost.