Load Balancing

A load balancer (LB) sits in front of your servers and distributes incoming requests across them. It is what makes horizontal scaling actually work — and it removes single points of failure by routing around dead servers via health checks.

Key ideas

Concept	Detail
Algorithms	Round-robin, least-connections, IP-hash (sticky), weighted
Layer	L4 (transport, fast) vs L7 (HTTP-aware, can route by path)
Health checks	LB pings servers; stops routing to unhealthy ones
Sticky sessions	Same client → same server (needed if state is local)
Redundancy	The LB itself is made HA (active-passive pair)

Exam Tricks & Tips

• 🎯 Round-robin is the default answer; mention least-connections when request costs vary.
• 🎯 Health checks are the reliability win — the LB routes around failed servers automatically.
• 🎯 Prefer stateless servers over sticky sessions — sticky sessions undermine even load distribution.
• 🎯 L7 lets you route by URL path (e.g., /api vs /static) and do SSL termination.
• 🎯 The LB needs redundancy too — mention an active-passive LB pair so it isn't a single point of failure.
• ❌ Common mistake: forgetting the load balancer itself can fail — always make it highly available.

Expected pattern: Appears the moment you add a second server; interviewers ask which algorithm and how health checks work.

Quick recap: LB spreads requests (round-robin/least-conn), uses health checks to skip dead servers, works best with stateless servers, and must itself be HA.

Cover the answer, recall, then check. 10 cards on load balancing.

Q1. What does a load balancer do?
A1. Distributes incoming requests across multiple servers behind it.

Q2. Why is a load balancer essential for horizontal scaling?
A2. It spreads traffic and routes around failed servers, enabling many servers to share load.

Q3. Default load-balancing algorithm to cite?
A3. Round-robin; use least-connections when request costs vary.

Q4. What do health checks do?
A4. The LB pings servers and stops routing to unhealthy ones.

Q5. L4 vs L7 load balancing?
A5. L4 works at the transport layer (fast); L7 is HTTP-aware and can route by URL path.

Q6. What are sticky sessions?
A6. Routing the same client to the same server — needed when session state is stored locally.

Q7. Why prefer stateless servers over sticky sessions?
A7. Sticky sessions undermine even load distribution; stateless servers balance freely.

Q8. Why must the LB itself be redundant?
A8. Otherwise it becomes a single point of failure — use an active-passive HA pair.

Q9. What can an L7 load balancer also do?
A9. SSL termination and path-based routing (e.g., /api vs /static).

Q10. Common load-balancing mistake?
A10. Forgetting the load balancer itself can fail.

The moment you scale horizontally, you need something to spread requests across your servers — that's the load balancer. It's a fixture of nearly every system-design answer, so you should be able to explain what it does, name a couple of routing algorithms, and mention health checks without hesitating. Skipping the load balancer in a design that has multiple servers is an immediate red flag.

What this covers / why it matters: what a load balancer does, common algorithms, health checks, and how it enables reliable horizontal scaling. Small topic, near-guaranteed to come up.

The approach

Beginner — what and why

A load balancer sits in front of your servers and distributes incoming requests across them. It gives you three things at once: spread load (no single server is overwhelmed), high availability (route around a server that dies), and a single entry point (clients hit one address; servers can be added/removed behind it invisibly).

Intermediate — the routing algorithms

Algorithm	How it routes
Round robin	Each request to the next server in turn — simple, even spread
Least connections	To the server with the fewest active connections — good for uneven request durations
Weighted	More traffic to more powerful servers
IP hash	Same client IP always to the same server — a way to keep session affinity

Pair this with health checks: the balancer periodically pings each server and stops routing to any that fail, then resumes when they recover. That's what turns "a server crashed" into a non-event for users.

Advanced — L4 vs L7 and sticky sessions

• Layer 4 (transport): routes by IP/port, fast, protocol-agnostic.
• Layer 7 (application): routes by HTTP content — path, headers, cookies — enabling smart routing (e.g. /api to one pool, /images to another) at slightly higher cost.
• Sticky sessions (session affinity) pin a user to one server; useful if the app is stateful, but the better design is a stateless app with shared session storage, so you don't need stickiness and can balance freely. Mentioning this connects load balancing back to statelessness.

Worked example

"You have three app servers. How do requests reach them reliably?"

"I'd put a load balancer in front as the single public entry point. It distributes requests across the three servers — round robin is a fine default, or least-connections if request durations vary a lot. Critically, it runs health checks: it pings each server every few seconds, and if one stops responding it's pulled out of rotation automatically, so users just get served by the healthy two — no downtime. When the server recovers and passes health checks again, it's added back. For this to work cleanly the servers should be stateless, so any of them can handle any request; if I were forced to keep state on the servers I could use sticky sessions via IP hash, but I'd prefer shared session storage instead. For high availability I'd also run the load balancer itself redundantly, so it isn't a new single point of failure."

Covers distribution, algorithm choice, health checks, statelessness, and even the balancer's own redundancy.

What interviewers look for

• You add a load balancer whenever there are multiple servers.
• You name an algorithm and can justify it.
• Health checks for automatic failover.
• Awareness that the balancer itself needs redundancy, and that statelessness beats sticky sessions.

Do's and don'ts

• Do include a load balancer in any multi-server design.
• Do mention health checks and automatic failover.
• Don't forget the balancer can be its own single point of failure.
• Don't rely on sticky sessions when shared session storage is cleaner.
• Mnemonic: DASH — Distribute, Availability, Single entry, Health checks.

Worked Example

Problem/Question: "You have five identical app servers. Explain how a load balancer distributes traffic and which algorithm you'd pick."
Solution/Model answer:
Role: a load balancer (LB) sits between clients and servers, spreading requests to avoid overloading any node, and improving availability by routing away from unhealthy servers (via health checks).
Common algorithms:

• Round robin: requests cycle through servers in order — simple, good when servers and requests are uniform.
• Least connections: send to the server with the fewest active connections — better when request durations vary.
• Weighted: bigger servers get proportionally more traffic.
• IP hash / sticky sessions: route a given client consistently to one server (needed if sessions are local).
  My choice: "With five identical servers and mostly uniform requests, round robin is fine. If some requests are long-lived (e.g., file uploads), I'd switch to least-connections. I'd add health checks so failed nodes are removed, and run the LB in an active-passive pair to avoid it being a single point of failure."
  Answer/Takeaway: A load balancer distributes requests (round robin, least-connections, weighted, or IP-hash) and routes around unhealthy nodes via health checks. Choose round-robin for uniform load, least-connections for variable request durations, and make the LB itself redundant.