Load Balancing

Load balancing distributes incoming requests across multiple service instances to improve availability and capacity. It is a key technique for horizontal scaling because traffic can be spread across independently running application instances.

A load balancer sits between clients and multiple backend instances. When a request arrives, it selects one healthy instance and forwards the request to it. This lets the application handle more traffic by adding instances rather than relying on one increasingly large server.

Common routing strategies include round robin, which cycles through instances, and least connections, which prefers an instance currently handling fewer connections. More advanced systems can also use weights, client affinity, or application-specific routing rules.

Health checks are important because a load balancer should avoid sending traffic to an instance that is unavailable. In a typical Java backend deployment, several instances of the same application may run on different machines or containers behind the load balancer.

Load balancing improves availability and capacity, but it does not automatically make an application stateless or solve every scaling problem. Shared session state, database bottlenecks, and uneven workloads may require additional architectural changes.

Key points

• Distributes requests across multiple backend instances.
• Enables horizontal scaling by adding service instances.
• Uses routing algorithms such as round robin or least connections.
• Health checks prevent traffic from being sent to unhealthy instances.
• Stateless services are easier to scale behind a load balancer.

Interview questions

Why do load-balanced applications often need to be stateless?

If a request can reach any instance, an instance should not be the only place where important user state exists. Stateless application instances can process requests independently, making it easier to add or remove instances and route traffic freely. Shared state can instead be stored in systems such as a database or distributed cache when appropriate.

What should a load balancer do when a backend instance becomes unhealthy?

The load balancer should detect the failure through health checks and stop routing new requests to that instance. Healthy instances continue receiving traffic. Once the failed instance passes the configured recovery checks, it can potentially be returned to the available pool.

What is load balancing, and why is it useful?

Load balancing distributes incoming requests across multiple backend instances. This prevents one instance from receiving all the traffic and allows additional instances to be added as demand increases. It can also improve availability because traffic can be redirected when an instance becomes unhealthy.

What is round-robin load balancing?

Round robin assigns requests to backend instances in a repeating sequence. For three instances, requests might go to A, then B, then C, and then return to A. It is simple and works reasonably well when instances have similar capacity and requests have similar cost.