Load balancing cheatsheet.

Upstream

upstream app {
    server 10.0.0.1:8000;
    server 10.0.0.2:8000;
    server 10.0.0.3:8000;
}

server {
    location / { proxy_pass http://app; }
}

Algorithms

upstream app {
    # default: round-robin
    
    least_conn;                          # fewest active
    
    ip_hash;                             # sticky by client IP
    
    hash $request_uri consistent;        # consistent by URI
    
    random two least_conn;               # power-of-two-choices
}

Weights

upstream app {
    server a.example.com weight=3;
    server b.example.com weight=1;
}

3:1 distribution.

Backup server

upstream app {
    server primary;
    server secondary backup;
}

Backup used only when primaries fail.

down

upstream app {
    server a;
    server b down;          # temporarily disabled
}

Passive health checks

upstream app {
    server 10.0.0.1 max_fails=3 fail_timeout=30s;
    server 10.0.0.2 max_fails=3 fail_timeout=30s;
}

3 consecutive failures → mark unavailable for 30s.

What counts as failure

location / {
    proxy_pass http://app;
    proxy_next_upstream error timeout http_502 http_503 http_504;
}

Active health checks (Nginx Plus only)

upstream app {
    zone app 64k;
    server 10.0.0.1;
}

location / {
    proxy_pass http://app;
    health_check interval=10 fails=3 passes=2 uri=/health;
}

Open source: passive only.

Keepalive to backend

upstream app {
    server 10.0.0.1:8000;
    keepalive 32;
    keepalive_requests 1000;
    keepalive_timeout 60s;
}

location / {
    proxy_pass http://app;
    proxy_http_version 1.1;
    proxy_set_header Connection "";
}

Critical for performance — avoids per-request TCP/TLS overhead.

Slow start (Nginx Plus)

server a slow_start=30s;

Gradually ramp traffic after server recovers.

Sticky sessions

ip_hash: by IP, but breaks behind NAT.

hash $cookie_jsessionid: sticky by cookie.

Better: Nginx Plus sticky directive (route, cookie, learn).

Failover (active-passive)

upstream app {
    server primary.example.com:8000 max_fails=2 fail_timeout=10s;
    server failover.example.com:8000 backup;
}

Retries

proxy_next_upstream error timeout invalid_header http_502 http_503 http_504;
proxy_next_upstream_tries 3;
proxy_next_upstream_timeout 30s;

Try next upstream up to 3 times within 30s.

Health endpoint pattern

Backend exposes /health returning 200 if OK. Nginx checks via max_fails on the proxied path (passive).

Read multiple backends per request

upstream search {
    server search-1;
    server search-2;
    server search-3;
}

Can’t natively merge responses; do that in app.

Cross-region

upstream app_us {
    server us-1;
    server us-2;
}

upstream app_eu {
    server eu-1;
    server eu-2;
}

map $geoip2_data_country_code $upstream {
    default app_us;
    US      app_us;
    DE      app_eu;
    FR      app_eu;
}

location / {
    proxy_pass http://$upstream;
}

Requires geoip2 module.

TCP / UDP load balancing (stream block)

stream {
    upstream pg {
        server 10.0.0.1:5432;
        server 10.0.0.2:5432;
    }
    
    server {
        listen 5432;
        proxy_pass pg;
    }
}

For non-HTTP services.

L7 routing examples

location /v1/ {
    proxy_pass http://app_v1;
}

location /v2/ {
    proxy_pass http://app_v2;
}

# Path-based to different upstreams
location ~ ^/api/(users|orders)/ {
    proxy_pass http://app_users;
}

location /api/payments/ {
    proxy_pass http://app_payments;
}

Canary by header

map $http_x_canary $upstream {
    default app;
    "1"     app_canary;
}

upstream app { server stable; }
upstream app_canary { server canary; }

location / {
    proxy_pass http://$upstream;
}

Common mistakes

  • Forgetting keepalive → connection storm.
  • ip_hash behind shared NAT → uneven distribution.
  • No health check semantics → broken backends still get traffic.
  • Same max_fails for many flaky upstreams → flapping.
  • Sticky session as substitute for shared session storage.

Read this next

If you want my upstream + LB templates, they’re at rajpoot.dev .

Building something AI-, backend-, or data-heavy and want a second pair of eyes? I do consulting and freelance work — see my projects and ways to reach me at rajpoot.dev .