Nginx Cheatsheet 19 — Debugging Nginx

Nginx debugging cheatsheet.

error log levels

error_log /var/log/nginx/error.log warn;
# debug info notice warn error crit alert emerg

debug requires nginx built with --with-debug. Most distro builds include it; check:

nginx -V 2>&1 | grep -- --with-debug

Per-location debug

error_log /var/log/nginx/debug.log debug;

server {
    location /api {
        # everything here logged at debug
    }
}

debug_connection

events {
    debug_connection 192.168.1.1;
    debug_connection 10.0.0.0/8;
}

Debug for specific client IPs.

Common errors

connect() failed (111: Connection refused) while connecting to upstream

Backend not listening or wrong address.

upstream timed out (110: Connection timed out)

Slow backend; increase proxy_read_timeout or fix backend.

client intended to send too large body

client_max_body_size too small.

upstream sent too big header

proxy_buffer_size too small for backend response headers.

worker_connections are not enough

Increase worker_connections in events block, raise ulimit.

accept() failed (24: Too many open files)

worker_rlimit_nofile too low or OS file limits.

SSL_do_handshake() failed

TLS issue: protocol mismatch, cert problems, SNI not handled.

no live upstreams while connecting to upstream

All backends marked failed (max_fails reached).

unknown directive

Module not loaded, wrong nginx version, or typo.

[emerg] still could not bind()

Port already in use; check lsof -i :80.

Test config

nginx -t                       # test
nginx -T                       # test + print full config (after includes)
nginx -t -c /path/to/config    # specific file

Tail logs

tail -f /var/log/nginx/error.log
tail -f /var/log/nginx/access.log

# with grep
tail -f /var/log/nginx/error.log | grep -v "client closed"

Verbose access log with timing

log_format debug '$remote_addr - $remote_user [$time_local] '
                 '"$request" $status $body_bytes_sent '
                 'rt=$request_time uct=$upstream_connect_time '
                 'uht=$upstream_header_time urt=$upstream_response_time '
                 '"$http_referer" "$http_user_agent"';

Investigate slow requests.

Check listening

ss -tlnp | grep nginx
lsof -i :80 -i :443

netstat -an | grep -E ":80|:443" | grep LISTEN

strace

strace -f -p $(pidof nginx | tr ' ' ',')

System call trace. Heavy; for short bursts only.

tcpdump

tcpdump -i any -A -s 0 'port 80 and host 1.2.3.4'

Capture traffic for specific client.

Test from another container

docker run --rm --network host curlimages/curl curl -v http://localhost/

Debug specific connections

curl -v https://example.com/
curl -v --resolve example.com:443:1.2.3.4 https://example.com/   # override DNS
curl -v -H "Host: example.com" http://1.2.3.4/                   # force host

Connection state

curl --trace-ascii out.txt -o /dev/null https://example.com/

Full request/response.

Health check why?

curl -v http://nginx/_health        # see status
curl -v http://backend:8000/health  # check backend directly

If nginx returns 502: backend unreachable or wrong port.

SSL debugging

openssl s_client -connect example.com:443 -servername example.com -showcerts
openssl x509 -in cert.pem -text -noout

Tracing per request

log_format trace '$request_id $request_uri';

server {
    location / {
        add_header X-Trace-ID $request_id always;
        proxy_set_header X-Trace-ID $request_id;
    }
}

Correlate with backend logs.

Reload didn’t work?

nginx -t
nginx -s reload
# OR
systemctl reload nginx

If broken: old config still active. Fix + reload again.

Master vs worker

ps -ef | grep nginx
# nginx: master process
# nginx: worker process

Master spawns workers. nginx -s reload makes master spawn new workers and stop old.

Common patterns and their fixes

Common mistakes

• Looking at access.log when error.log has the answer.
• Debugging old config (forgot reload).
• Missing -t before reload, broken config sticks.
• Editing one of many includes, not realizing precedence.
• Comparing $remote_addr without real_ip_header set.

Read this next

• Cheatsheet 20 — Production Nginx Setup

If you want my debugging checklist, it’s at rajpoot.dev .

Building something AI-, backend-, or data-heavy and want a second pair of eyes? I do consulting and freelance work — see my projects and ways to reach me at rajpoot.dev .