Load balancing becomes necessary when a single server can’t handle your traffic volume, when you need redundancy to survive server failures, or when you want to deploy updates without downtime. Understanding load balancing architectures and tradeoffs helps you implement solutions that match your actual scaling requirements.
The fundamental concept is simple: distribute incoming requests across multiple backend servers. The complexity emerges from decisions about which algorithms to use for request distribution, how to handle stateful connections, and what happens when backends fail.
Layer 4 vs Layer 7 Load Balancing
Load balancers operate at different layers of the network stack, with distinct capabilities and performance characteristics.
Layer 4 Load Balancing
Layer 4 load balancers work at the transport layer, making routing decisions based on IP addresses and TCP/UDP ports. They don’t inspect packet contents beyond establishing which backend should handle a connection. This approach offers maximum throughput because packet inspection is minimal. Layer 4 load balancing works for any TCP or UDP traffic, not just HTTP.
The limitation is lack of application awareness. You can’t route requests based on URL paths, HTTP headers, or cookie values. SSL termination must happen on backend servers, preventing content-based routing decisions.
Layer 7 Load Balancing
Layer 7 load balancers operate at the application layer, fully parsing HTTP requests before making routing decisions. This enables sophisticated routing based on URL paths, hostnames, headers, or cookies. You can route `/api` requests to API servers and `/static` requests to asset servers using a single load balancer.
SSL termination at the load balancer decrypts traffic once rather than at each backend, reducing CPU load on application servers. Session affinity based on cookies is reliable because the load balancer reads application-level data. The cost is reduced throughput compared to Layer 4 due to HTTP parsing overhead.
Load Balancing Algorithms
The algorithm determining which backend receives each request affects load distribution and user experience.
Round Robin
Round robin cycles through available backends in order, sending each new request to the next server. It’s simple and distributes load evenly when requests have similar resource costs. The problem is that not all requests are equal. If request durations vary significantly, round robin might create uneven load distribution.
Least Connections
Least connections directs new requests to the backend with the fewest active connections. This works better than round robin when request durations vary significantly. A server handling long-running requests accumulates connections and receives fewer new requests until some complete. The downside is tracking connection counts adds overhead.
Weighted Algorithms
Weighted variants of round robin or least connections allow specifying relative capacities for backends. If one server has double the CPU and memory of others, giving it weight 2 while others have weight 1 sends it twice as many requests. This handles heterogeneous server configurations gracefully but requires manual configuration.
IP Hash
IP hash algorithms compute a hash of the client IP address and use it to select a backend. The same client always reaches the same backend, providing implicit session affinity. This works well for caching scenarios. The failure mode is problematic: when a backend fails, clients whose IPs mapped to it get reassigned, losing cached state.
Health Checks and Failure Detection
Load balancers must detect failed backends and stop sending them traffic. The health check configuration significantly affects how your application handles failures.
Active Health Checks
Active health checks periodically send requests to each backend to verify it’s responding correctly. TCP checks simply establish a connection. HTTP checks request a specific URL and expect a particular status code, confirming the application is running.
Check intervals balance quick failure detection against overhead. Checking every second detects failures quickly but generates significant monitoring traffic. Checking every 30 seconds reduces overhead but means failed backends might receive up to 30 seconds of traffic. Unhealthy thresholds determine how many consecutive failures trigger marking a backend unhealthy. Typical values are 2-3 failures.
Passive Health Checks
Passive health checks monitor actual request results to backends. When a backend returns errors or times out on real requests, the load balancer notes this. After reaching a threshold of failures, it stops sending new requests temporarily. This avoids generating synthetic monitoring traffic and responds to actual failure modes users encounter. Combining active checks for baseline health with passive checks for real failures often works best.
Session Affinity and Stickiness
Many applications maintain state in memory on backend servers. Without session affinity, subsequent requests from the same user might reach different backends that don’t have their session data.
Cookie-Based Affinity
Cookie-based session affinity has the load balancer set a cookie identifying which backend handled the first request. Subsequent requests include this cookie, directing them to the same backend. This works reliably for web applications. The backend must handle the session cookie expiring or clients clearing cookies. Consider what happens when the sticky backend fails.
IP-Based Affinity
IP-based affinity routes requests from the same source IP to the same backend. This works without application changes and handles non-HTTP protocols. The problems are clients behind NAT appear as the same IP, forcing them to share a backend, and mobile clients changing networks get reassigned.
Session Storage Alternatives
The better long-term solution is removing the need for sticky sessions entirely. Store session data in Redis or a database that all backends can access. This allows true stateless backends where any server can handle any request. Load balancing becomes simpler and backend failures don’t cause session loss. The cost is additional infrastructure and slight latency from fetching session data.
SSL/TLS Termination
Deciding where to terminate SSL connections affects security, performance, and operational complexity.
Terminating at Load Balancer
Terminating SSL at the load balancer means it decrypts traffic, inspects it for routing decisions, then forwards it to backends over unencrypted connections. This works when backends are on a trusted private network. Performance improves because backends don’t spend CPU on encryption. The load balancer can make routing decisions based on decrypted content.
End-to-End Encryption
End-to-end encryption maintains encrypted connections from client through load balancer to backend. This provides defense in depth if your private network is compromised. The cost is increased complexity and reduced performance. For most applications, terminating at the load balancer provides sufficient security.
Deployment Patterns
How you architect load balancing affects reliability, performance, and complexity.
Single Load Balancer
The simplest approach uses one load balancer in front of multiple backends. This eliminates the single backend failure point but makes the load balancer a potential failure point. For many applications, this is acceptable. Cloud provider load balancers are inherently redundant within a region, making a “single load balancer” actually multiple load balancers managed by the provider.
Multiple Load Balancers with DNS
For additional redundancy, deploy multiple load balancers with DNS returning multiple IP addresses. Clients try each IP until one succeeds. DNS caching means clients might cache a failed load balancer’s IP for the DNS TTL duration. Health-checked DNS services remove failed load balancers automatically.
Geographic Distribution
Global applications benefit from geographic load balancing where DNS routes clients to the nearest regional deployment. Each region has its own load balancers and backends. This reduces latency and provides disaster recovery. The complexity increases significantly with deployment automation, data replication, and partition handling across regions.
When to Implement Load Balancing
Start with a single server until you have evidence it can’t handle your load. Vertically scale by upgrading to a more powerful server first. This is simpler and often cheaper than horizontal scaling with load balancing.
Implement load balancing when one server can’t handle peak traffic, when you need zero-downtime deployments through rolling updates, when you want redundancy to survive hardware failures, or when traffic patterns vary enough that elastic scaling makes economic sense. Start with the simplest solution. Use your cloud provider’s managed load balancer. Layer 7 load balancing handles most web application needs. Avoid session affinity if possible by using shared session storage.
Operational Considerations
Monitor load balancer metrics including request rates, backend response times, error rates, and unhealthy backend counts. Set alerts when error rates exceed thresholds or when all backends become unhealthy.
Test your failure scenarios. What happens when a backend fails during a request? What happens when all backends fail? Testing these scenarios before production issues occur prevents surprises during real incidents. Plan capacity for the load balancer itself. Cloud load balancers scale automatically but have rate limits. Self-hosted load balancers need capacity planning like any other server.