Index of every page on this site: llms.txt
All skills
proyecto26 avatar

/load-balancing

@228d57d
Run it yourself

This skill should be used when the user adds a "load balancer", asks about "L4 vs L7" (transport vs application layer), picks a balancing algorithm ("round robin", "least connections", "weighted", "IP hash"), configures "health checks", needs "sticky sessions" / "session affinity", adds a "reverse proxy", does "SSL/TLS termination", spreads "traffic distribution" across an "autoscaling group", or wants a stateless web tier behind a single entry point. Use it whenever a design has more than one server behind one address, or a single box is the entry-point bottleneck or SPOF, even if the user doesn't say "load balancer".

Use this Skill: https://skilld.dev/gh/proyecto26/system-design-skills/load-balancing

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

referencesdeep-dive.md

≈1.5k tokens on demand. Your agent reads this file only when SKILL.md points to it.

Load balancing deep-dive

Mechanics that don't belong in the lean SKILL.md. Read when configuring a balancer in detail.

L4 forwarding: NAT vs. DSR

An L4 balancer never reads the payload; it forwards packets by IP/port.

  • NAT (network address translation): the balancer rewrites the destination (and on the way back, the source) so both directions traverse it. Simple, but the balancer is in the return path too — return traffic (often the larger half) flows back through it, capping throughput.
  • DSR (direct server return): the backend replies straight to the client, bypassing the balancer on egress. The balancer only handles inbound, so it scales much further for response-heavy workloads (video, downloads). Costs more setup (loopback VIP on backends, L2 adjacency or tunneling) and loses return-path visibility.

L4 keeps the client's TCP connection essentially end-to-end, so it adds minimal latency and works for any protocol (gRPC streams, databases, custom TCP/UDP).

L7 termination: how it actually routes

An L7 balancer is a full proxy: it terminates the client's TCP/TLS connection, parses the HTTP request, makes a routing decision, then opens (or reuses, via a connection pool) a separate connection to the chosen backend. Two connections, two congestion windows. This is what enables:

  • Content routing: by Host, path prefix, header, method, or cookie — e.g. /api/* to the API pool, /static/* to a cache tier, video hosts to media servers, billing to hardened nodes.
  • Request manipulation: header injection (X-Forwarded-For, request IDs), rewrites, redirects, compression.
  • Backend connection pooling / multiplexing: many short client connections reuse a small pool of warm backend connections (huge win for HTTP/2, gRPC).

Cost: per-request CPU (parsing, TLS), added latency, and the balancer now holds connection state — making it more stateful and its memory a sizing concern.

Distribution algorithms in detail

  • Round robin: rotate through the pool. Weighted assigns more turns to bigger nodes. Blind to actual load — a node stuck on slow requests still gets its share.
  • Least connections: route to the fewest active connections; least response time also factors latency. Best when request cost is uneven (mixed or long-lived connections). Requires the balancer to track live state; a just-added node looks idle and can get flooded — pair with slow-start.
  • Power of two choices: sample two backends at random, pick the less loaded. Near-optimal balance with O(1) state and no global coordination — the default in many modern proxies.
  • Hash-based (IP hash / header hash): deterministic client→backend mapping for affinity without a session store. Plain modulo hashing remaps almost everything when the pool changes; consistent hashing moves only ~K/N keys on a pool change, preserving most affinity and cache locality. The full consistent-hashing mechanics (rings, virtual nodes) are owned by consistency-coordination — use it here purely as the "affinity that survives scaling" tool.

Health checks (the part people get wrong)

  • Active checks: the balancer probes each backend (GET /healthz, a TCP connect, or gRPC health RPC) on an interval. Passive checks observe real traffic and eject a node after N consecutive errors.
  • Shallow vs. deep: a shallow check (process accepts a connection / returns 200 from a trivial handler) tests "is the box up." A deep check (verifies DB, cache, downstream) tests "can it serve" — but if the shared dependency hiccups, every backend fails the deep check simultaneously and the whole pool is evicted, turning a slow dependency into a total outage. Prefer shallow liveness for pool membership; handle dependency failure with degradation in the app (→ resilience-failure).
  • Thresholds & jitter: require multiple consecutive fails before eviction and multiple passes before re-admission (anti-flap). Jitter the interval so checks don't synchronize into a probe storm.

Slow-start / connection ramping (anti-stampede)

When a node (re)joins, ramp its share from near-zero up over a warm-up window instead of handing it a full load instantly. This lets caches fill, JITs warm, and connection pools build before peak traffic arrives — directly preventing the recover→overload→drop→recover oscillation. Combine with least-connections-aware weighting so a "0 connections" fresh node isn't treated as the most attractive target.

Connection draining (graceful scale-in / deploy)

On deregistering a node (scale-in, deploy, scale to a smaller instance), stop sending it new connections but let in-flight requests finish for a drain timeout before terminating. Without draining, every deploy or scale event drops live requests. Set the drain window to cover your slowest reasonable request.

TLS termination vs. passthrough

  • Termination at the balancer: decrypt once at the edge; backends speak plain HTTP. Offloads CPU, centralizes certs, and is required for L7 content routing (you can't route on a path you can't read). Re-encrypt to backends (mTLS in the mesh) if the internal hop must stay encrypted.
  • Passthrough (TCP/L4): the balancer forwards encrypted bytes; the backend terminates TLS. Needed for end-to-end encryption or client-cert auth that must reach the app. Loses L7 routing and inspection.

Common mistakes

  • One balancer, no standby — the balancer is now the SPOF it was meant to remove.
  • Deep health checks that evict the whole fleet when a shared dependency blips.
  • No slow-start → recovered nodes get firehosed and flap.
  • No connection draining → every deploy drops in-flight requests.
  • Sticky sessions used as a crutch instead of externalizing state, then scale-in drops live sessions.
  • Unbounded retries at the balancer amplifying load onto survivors (cap + backoff; → resilience-failure).
  • Treating cross-region steering as an L4/L7 problem instead of DNS/anycast/CDN (→ content-delivery).

Source: SKILL.md on GitHub

No rule matched.

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No alerts24d3 checks · Risk SAFE
  • Gen Agent Trust Hub24d

    The skill is a set of architectural guidelines and reference materials for load balancing strategies. It contains no executable code, remote downloads, or dangerous commands.

  • Socket24d

    No alerts

  • Snyk24d

    Risk: LOW · No issues

Signed by skilld at 228d57d. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub 3 weeks ago.

Activeupdated 4 weeks ago

README badge

README badge for proyecto26/system-design-skills/load-balancing