Nginx Cheatsheet 11 — Performance Tuning

Nginx performance cheatsheet.

Workers

worker_processes auto;            # match CPU cores
worker_rlimit_nofile 65536;       # max FDs per worker

events {
    worker_connections 10240;     # max conns per worker
    use epoll;                    # Linux best
    multi_accept on;              # accept multiple per event
}

Max conns = worker_processes * worker_connections.

sysctl tuning

# /etc/sysctl.d/nginx.conf
net.core.somaxconn = 65535
net.core.netdev_max_backlog = 65535
net.ipv4.tcp_max_syn_backlog = 65535
net.ipv4.ip_local_port_range = 1024 65535
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_keepalive_time = 30
net.ipv4.tcp_keepalive_intvl = 10
net.ipv4.tcp_keepalive_probes = 6
fs.file-max = 2097152

sysctl -p

ulimit

# /etc/security/limits.d/nginx.conf
nginx soft nofile 65536
nginx hard nofile 65536

Or in systemd unit:

[Service]
LimitNOFILE=65536

Keepalive

keepalive_timeout 65;
keepalive_requests 1000;

To backends:

upstream app {
    server 10.0.0.1;
    keepalive 64;
    keepalive_requests 1000;
    keepalive_timeout 60s;
}

sendfile

sendfile on;
tcp_nopush on;
tcp_nodelay on;
sendfile_max_chunk 1m;

aio threads

aio threads;
aio_write on;
directio 4m;

For large file I/O.

Buffer sizes

client_body_buffer_size 16k;
client_header_buffer_size 1k;
client_max_body_size 10m;
large_client_header_buffers 4 8k;

Don’t go too high; each connection uses these.

proxy_buffers

proxy_buffering on;
proxy_buffer_size 16k;
proxy_buffers 8 16k;
proxy_busy_buffers_size 32k;
proxy_temp_file_write_size 64k;

For SSE / WebSocket: proxy_buffering off.

open_file_cache

open_file_cache max=200000 inactive=60s;
open_file_cache_valid 30s;
open_file_cache_min_uses 2;
open_file_cache_errors on;

Cached fd avoids open() syscall per request.

Reduce logging

access_log /var/log/nginx/access.log main buffer=64k flush=5s;

Or off for healthchecks:

location /health {
    access_log off;
    return 200;
}

TLS perf

ssl_session_cache shared:SSL:50m;
ssl_session_timeout 1d;
ssl_session_tickets off;

ssl_protocols TLSv1.3 TLSv1.2;

TLS 1.3 is faster (1-RTT handshake).

HTTP/2 / HTTP/3

listen 443 ssl http2;

listen 443 quic reuseport;
add_header Alt-Svc 'h3=":443"; ma=86400';

Cache for hot paths

See caching cheatsheet. Microcache (1s) absorbs traffic spikes.

CPU affinity (rarely needed)

worker_cpu_affinity auto;

Pins workers to specific CPUs.

Reuse port (multiple listening sockets)

listen 80 reuseport;

Each worker has its own accept queue. Better balance.

Profiling

# perf top -p <nginx-worker-pid>
# strace -f -p <pid>
# nginx -V                        # see compile-time options

Module: ngx_http_status_module (NGINX+) or nginx-vts for per-zone stats.

Load test

# wrk
wrk -t 12 -c 400 -d 30s http://example.com/

# bombardier
bombardier -c 1000 -d 30s http://example.com/

# vegeta
echo "GET http://example.com/" | vegeta attack -duration=30s -rate=1000 | vegeta report

Symptoms vs fix

Symptom	Fix
`worker_connections are not enough`	Increase `worker_connections` + sysctl `somaxconn`
`accept() failed (24: Too many open files)`	Raise `worker_rlimit_nofile` + ulimit
High CPU on TLS	Enable session cache; use TLS 1.3
Slow connections in keep-alive	Tune `keepalive_timeout`, `tcp_keepalive_*`
OOM	Lower `worker_connections`, buffer sizes

Common mistakes

worker_processes higher than cores → context switching.
worker_connections huge without OS limits raised.
Forgetting tcp_nodelay / tcp_nopush.
gzip level 9 (CPU expensive, marginal gain over 6).
Logging every request to slow disk in high-volume.

Workers#

sysctl tuning#

ulimit#

Keepalive#

sendfile#

aio threads#

Buffer sizes#

proxy_buffers#

open_file_cache#

Reduce logging#

TLS perf#

HTTP/2 / HTTP/3#

Cache for hot paths#

CPU affinity (rarely needed)#

Reuse port (multiple listening sockets)#

Profiling#

Load test#

Symptoms vs fix#

Common mistakes#

Read this next#