Learn how HTTP persistent connections eliminate TCP handshake overhead, what HTTP/2 multiplexing adds on top, and why connection keep-alive settings are a frequent source of production 502 errors.
21 min read2026-03-28easyhttpkeep-alivenetworkingperformancehttp2
Your nginx reverse proxy has keepalive_timeout 75s (the default). Your Node.js app server has keepAliveTimeout set to 5000 ms (the old default in Node.js 16 and earlier). nginx holds upstream connections open for up to 75 seconds. Node.js closes them after 5 seconds of idle time.
After 5 seconds of inactivity, Node.js sends a FIN on the socket. nginx does not see this immediately. When the next request arrives, nginx picks that connection from its upstream pool and sends the request. Node.js responds with a TCP RST: that socket is closed. nginx gets the RST mid-write and returns a 502 Bad Gateway to the client.
This is not a fluke. It happens on every connection that idles for more than 5 seconds. At low traffic, connections are always busy, so the window never opens. At moderate traffic with brief pauses between requests (typical for user-facing APIs), you see a steady background rate of 502s that seems random but always clusters around the 5-second idle mark.
HTTP keep-alive (persistent connections) reuses one TCP connection for multiple request-response cycles. Without it, every HTTP request pays the TCP handshake cost: at least 1.5 round trips before a single byte of application data is transferred.
Analogy: A phone call versus a series of walkie-talkie exchanges. Each walkie-talkie message (HTTP/1.0) requires keying the transmitter, speaking, releasing, and waiting for acknowledgment before the next message. A phone call (keep-alive) holds the channel open for the full conversation. You pay the connection cost once.
HTTP/1.1 enables keep-alive by default. You must explicitly opt out with Connection: close. HTTP/2 adds connection multiplexing on top: multiple requests travel on the same connection simultaneously, not sequentially.
In HTTP/1.0, every request requires a new TCP connection. The three-way handshake alone costs 1.5 RTT before the request is even sent. For 100 requests over 50 ms RTT, that is 7.5 seconds of pure handshake overhead.
HTTP/1.1 keeps the connection open between requests. The handshake is paid once. The tradeoff is head-of-line (HOL) blocking: request B cannot be sent on the same connection until response A is fully received.
HTTP/2 introduces binary framing with stream IDs. Multiple requests are in flight on the same connection simultaneously. Request B does not wait for response A.
Comments
For a 50 ms RTT, HTTP/1.0 wastes 75 ms on handshake per request. HTTP/1.1 amortizes that 75 ms across all requests on the connection. HTTP/2 also eliminates the sequential wait: responses A and B can overlap in transit.
The root cause: the upstream server closes a connection that the proxy still holds in its pool as "idle but usable." The proxy sends the next request on a socket that is already closed.
The fix is to ensure Node.js keeps connections alive longer than nginx's upstream idle period, so Node.js never closes a connection nginx might still use:
const server = http.createServer(app);// Must be greater than the proxy's upstream keepalive timeout.// If nginx keepalive_timeout is 60s, set this to 65s.server.keepAliveTimeout = 65_000;// Must be greater than keepAliveTimeout.// Node.js enforces a separate deadline for receiving HTTP headers.// If headersTimeout fires before keepAliveTimeout, you get a different 502.server.headersTimeout = 66_000;
Lower nginx's keepalive_timeout to 60s (or whatever is below your Node.js value), and set server.keepAliveTimeout = 65000. Node.js will now hold connections open past nginx's cutoff point. nginx closes idle connections first, gracefully, without sending a new request. Node.js handles the incoming FIN cleanly.
Always set headersTimeout alongside keepAliveTimeout
Setting only keepAliveTimeout without headersTimeout is a common partial fix. If headersTimeout defaults to 60 000 ms and keepAliveTimeout is 65 000 ms, the headers timeout fires first on slow requests, producing its own 408 or 502 error. Always set headersTimeout = keepAliveTimeout + 1000 to ensure they fire in the right order.
HTTP/2 eliminates application-layer HOL blocking. Streams 1, 3, and 5 are all in flight simultaneously. A slow response on stream 1 does not block stream 3 from being sent or received.
But HTTP/2 still runs over a single TCP connection. TCP guarantees in-order byte delivery. If one TCP segment is lost, the OS retransmit timer fires and all HTTP/2 streams on that connection stall, even streams whose data has already arrived in the receive buffer.
This is TCP head-of-line blocking. HTTP/2 trades application-layer HOL for TCP-layer HOL.
HTTP/3 (QUIC) eliminates this by replacing TCP with UDP-based QUIC streams. Each QUIC stream has independent loss detection and retransmission. A lost packet on stream 1 does not affect stream 3.
HOL blocking by protocol:
Protocol | App-layer HOL | TCP-layer HOL
----------+---------------+--------------
HTTP/1.0 | N/A | Yes
HTTP/1.1 | Yes | Yes
HTTP/2 | No | Yes
HTTP/3 | No | No (QUIC)
In practice: TCP HOL blocking only causes measurable harm under high packet loss (above 1-2%), which is common on mobile networks and uncommon on datacenter links. HTTP/2 is a clear improvement over HTTP/1.1 for most workloads. HTTP/3 is the right choice for user-facing endpoints on unreliable or high-latency networks.
nginx maintains a pool of idle persistent upstream connections per worker process. The keepalive N directive in an upstream block sets the maximum number of idle connections per worker:
upstream backend { server app:3000; keepalive 32; # up to 32 idle connections per worker process keepalive_timeout 60s; # close idle connections after 60s}server { location / { proxy_pass http://backend; proxy_http_version 1.1; # required for keep-alive upstream proxy_set_header Connection ""; # strip downstream Connection header }}
Note the two required additions: proxy_http_version 1.1 (nginx defaults to HTTP/1.0 for upstream, which cannot use keep-alive), and clearing the Connection header (otherwise a Connection: close from a browser propagates upstream and defeats the pool).
The diagram shows how a single nginx worker reuses pooled connections instead of opening a new TCP connection for each request:
D2 render error.
spawnSync d2 ENOENT
Browsers vs server-side HTTP clients handle keep-alive differently
Browsers maintain per-origin connection pools with hard caps: Chrome opens at most 6 parallel HTTP/1.1 connections per hostname, but collapses to 1 connection per origin under HTTP/2. Browsers close idle connections automatically on tab close or after a few minutes. Server-side HTTP clients (Axios, Got, Node.js fetch) do not enable connection reuse by default. To get keep-alive in outgoing Node.js requests, pass new http.Agent({ keepAlive: true }) explicitly or use a library that does it for you. Missing this in a microservice that makes many outbound calls is a common and silent performance gap.
Too low a keepalive value means nginx opens a new connection when all idle connections are in use, paying the handshake cost per request. Too high a value wastes file descriptors: with 16 worker processes at keepalive 256, that is 4,096 idle sockets held open even when traffic is low.
A practical starting point is keepalive 32. Check $upstream_connect_time in nginx access logs. If more than 5% of requests show a non-zero connect time, connections are not being reused and the pool should be larger.
keepalive N per upstream block (disabled by default)
Requires proxy_http_version 1.1 and proxy_set_header Connection "" to work correctly; without these, keep-alive is silently not used
Node.js http
server.keepAliveTimeout (default was 5 000 ms in Node 16; changed to 0 in Node 18 LTS)
The 5 s default was the root cause of widespread nginx+Node.js 502 errors; always set this explicitly rather than relying on version defaults
AWS ALB
60-second idle timeout (configurable 1-4 000 s)
ALB closes idle connections after the configured timeout regardless of application settings; set server.keepAliveTimeout a few seconds below the ALB value so the server closes gracefully before ALB forces a reset
Chrome
6 parallel HTTP/1.1 connections per host, automatic HTTP/2
HTTP/1.1 browser parallelism cap of 6 drove domain sharding (splitting assets across hostnames); HTTP/2 collapses all requests to one connection per origin, making domain sharding counterproductive
Keep-alive holds a file descriptor open for the duration of the idle timeout. High idle timeouts on servers with large numbers of clients waste kernel memory even when no data is flowing.
The nginx/Node.js 502 race condition is the most common keep-alive misconfiguration. When debugging 502s from a proxy, check whether the upstream timeout is shorter than the proxy's idle timeout before looking anywhere else.
HTTP/2 does not fix TCP head-of-line blocking. On high-loss networks, a single lost segment stalls all streams on the connection simultaneously. HTTP/3 (QUIC) is the correct solution for that problem.
Disabling keep-alive is not always wrong. Short-lived batch processes or CLI tools that make a handful of requests and exit do not benefit from connection reuse. For those patterns, the file descriptor cost of keeping a connection open exceeds the handshake cost saved.
AWS ALB and similar managed load balancers have their own idle timeout that is independent of application settings. If the ALB closes a connection before the application does, the application sees an unexpected RST from the load balancer side.
HTTPS amplifies the cost of per-request connections. TCP handshake is 1.5 RTT; TLS 1.3 adds one more RTT for a total of 2.5 RTT. Over TLS 1.2, that is 3.5 RTT. Keep-alive is more valuable in HTTPS environments precisely because the connection setup cost is higher.
When 502 errors cluster around nginx and Node.js with no other obvious cause, immediately ask about keepAliveTimeout. The classic scenario: nginx at 75 s, Node.js at 5 s. Fix with server.keepAliveTimeout = 65_000; server.headersTimeout = 66_000; and reduce nginx's upstream timeout to 60 s.
When asked what HTTP/1.1 keep-alive buys, state that it eliminates the TCP handshake per request (saving 1.5 RTT), enables sequential request reuse, and that the tradeoff is head-of-line blocking: request B waits for response A to complete.
When asked about HTTP/2 benefits over HTTP/1.1, state: multiplexing eliminates application-layer HOL blocking. Multiple requests are in flight on one connection simultaneously. TCP-layer HOL blocking still applies.
When asked about HTTP/3 or QUIC, state: HTTP/3 runs over UDP-based QUIC, eliminating TCP HOL blocking. Each QUIC stream recovers lost packets independently. Most impactful on mobile and high-loss networks.
When enabling nginx upstream keep-alive, always mention the two required companion settings: proxy_http_version 1.1 and proxy_set_header Connection "". Without both, keep-alive silently falls back to per-request connections.
When load testing reveals unexpected 502s, check $upstream_connect_time in nginx logs before anything else. Non-zero values mean connections are not being reused. Zero values with 502s point to the race condition scenario.
For AWS ALB deployments, set server.keepAliveTimeout to the ALB idle timeout minus 5 seconds, not above it. Let the server close idle connections before the ALB hard-resets them.
When someone suggests disabling keep-alive to eliminate the 502 race, acknowledge it solves the problem but at the cost of a TCP (and likely TLS) handshake on every request. Acceptable for very low-traffic services; unacceptable at scale.
HTTP keep-alive reuses one TCP connection for multiple request-response cycles, eliminating the TCP handshake cost (1.5 RTT minimum) per request.
The nginx+Node.js 502 race condition occurs when Node.js closes idle connections faster than nginx's upstream keepalive timeout. Fix by setting server.keepAliveTimeout greater than the proxy's upstream idle period, and always set headersTimeout one second above it.
HTTP/2 multiplexing eliminates application-layer head-of-line blocking: multiple requests travel concurrently on one connection. TCP-layer HOL blocking still applies.
HTTP/3 (QUIC) uses UDP-based streams with independent loss recovery per stream, eliminating TCP-layer HOL blocking. Most impactful on high-loss networks such as mobile.
nginx upstream keep-alive requires three settings together: keepalive N, proxy_http_version 1.1, and proxy_set_header Connection "". Missing any one of them silently disables reuse.
For reverse-proxy deployments, timeouts should increase going outward: app server smallest, then proxy, then load balancer, then browser. Each layer should close idle connections before the layer outside it tries to reuse them.
Networking — TCP connection mechanics (SYN/SYN-ACK/ACK, FIN, RST) underpin why keep-alive saves latency and why a closed socket produces a RST rather than a clean error.
Load balancing — Load balancers maintain their own keep-alive pools toward both clients and upstreams. Misconfigured idle timeouts at the load balancer layer are the most common source of production 502 and 504 errors.
Connection pooling — Database connection pooling solves the same reuse problem at the application layer. The timeout mismatch issue (pool holding connections longer than the database's idle killer allows) mirrors the nginx/Node.js 502 scenario directly.