Server

Connection drain

Connection drain is taking an origin out of the rotation so it receives no new requests while the ones in progress finish. It is how a deploy or a scale-in avoids cutting visitors off mid-response.

How it is measured

You see it as a state on the load balancer: the target is marked draining, new connections go elsewhere, and the active connection count falls toward zero. The window is bounded by a drain timeout, often 30 to 300 seconds.

Measure the connection count on the draining node every few seconds, and compare the 5xx rate during the window to the baseline. A clean drain shows connections reaching zero before the timeout with no new errors.

Worked example

A Node API behind a load balancer is replaced during a deploy. The old instance has 38 open requests, mostly short, plus 3 file downloads running for 40 seconds. The balancer sends no new traffic to it, and the count drops to 3 within 5 seconds, then to zero at 41 seconds.

With a 30 second drain timeout those three downloads would have been cut at 30 seconds, and three customers would have seen a half file. Raising the timeout to 60 fixed it.

How it differs

Connection drain is the load balancer stopping new traffic to a target. Graceful shutdown is the app itself finishing accepted work and closing cleanly. Drain excludes any behaviour inside the process. Graceful shutdown excludes the routing decision. You usually need both, in that order.

Common errors

Setting the drain timeout shorter than your longest request. Killing the process the instant the deploy starts. Forgetting that WebSocket and streaming connections never drain on their own. Counting only HTTP and ignoring long-lived keep-alive sockets. Skipping drain on scale-in events.

In practice

Look up the longest legitimate request on your site, such as an export, an upload or a stream, and set the drain timeout above it. Run one deploy while watching the 5xx count on the load balancer and see whether any spike lines up with the swap.

See also

Graceful shutdown, Load balancer

Sources

Count this on a real site.

Watch my website