How TCP works

The Problem Statement

Interviewer: "You are building a service that streams sensor data from thousands of IoT devices. Your team is debating whether to use TCP or UDP. Before we get to the tradeoff, walk me through what TCP actually does under the hood. How does it guarantee delivery, ordering, and flow control?"

This question tests three things: whether you understand the machinery that makes reliable communication possible (sequence numbers, acknowledgments, retransmission timers), whether you understand how TCP adapts to network conditions (sliding window, congestion control), and whether you can connect that knowledge to system design decisions (connection pooling, head-of-line blocking, keep-alive).

Most candidates can say "TCP is reliable and ordered." Strong candidates explain how it achieves those guarantees. The best candidates connect TCP behavior to real production problems: why connection pooling matters, why Nagle's algorithm causes latency spikes, and why HTTP/2 still suffers from head-of-line blocking at the TCP layer.

What you will walk away with after reading this:

The full mechanics of the three-way handshake and why each packet exists.
How sequence numbers and acknowledgments provide reliable, ordered delivery.
How the sliding window and receiver flow control prevent overwhelm.
How congestion control algorithms (slow start, AIMD, fast retransmit) adapt to the network.
Why TCP behaviors like Nagle's algorithm, delayed ACKs, and TIME_WAIT matter in system design.
The production patterns (connection pooling, TCP_NODELAY, keep-alive) that mitigate TCP's overhead.

TCP is the transport layer under every HTTP request, every database query, every gRPC call in a typical distributed system. Understanding how it works gives you the intuition to reason about latency, throughput, and failure modes. Every time someone says "the network is slow," TCP behavior is usually the first place to look.

Clarifying the Scenario

You: "Great question. Let me make sure I scope this correctly."

You: "When you say 'what TCP does under the hood,' do you want me to cover connection setup, data transfer, and teardown as three phases? Or focus on one?"

Interviewer: "Cover all three, but go deeper on the data transfer phase, especially flow and congestion control."

You: "Got it. Should I also discuss how TCP behavior affects system design decisions, like connection pooling and keep-alive?"

Interviewer: "Yes, tie it back to real-world architecture at the end."

You: "Perfect. I will structure this in four parts: first the three-way handshake, then sequence numbers and reliable delivery, then flow and congestion control, and finally why all of this matters when you are designing distributed systems."

My Approach

TCP is a connection-oriented protocol that provides four guarantees over an unreliable network: reliable delivery, in-order delivery, flow control (do not overwhelm the receiver), and congestion control (do not overwhelm the network). Everything TCP does exists to deliver these four properties.

I break this into four parts:

Connection lifecycle: The three-way handshake, data transfer, and four-way teardown.
Reliable, ordered delivery: How sequence numbers, ACKs, and retransmission timers guarantee that every byte arrives in order.
Flow and congestion control: The sliding window, receiver window, and congestion window working together to maximize throughput without causing collapse.
System design implications: Connection pooling, Nagle's algorithm, TCP_NODELAY, keep-alive, and head-of-line blocking.

Here is a quick reference for the core mechanisms:

Guarantee	Mechanism	How it works
Reliable delivery	Sequence numbers + ACKs	Sender retransmits unacknowledged segments after timeout
Ordered delivery	Sequence numbers	Receiver reassembles out-of-order segments using sequence offsets
Flow control	Receiver window (rwnd)	Receiver advertises how much buffer space it has, sender respects it
Congestion control	Congestion window (cwnd)	Sender probes network capacity gradually, backs off on loss

The core insight is that TCP is a feedback loop. The sender transmits data, the receiver responds with acknowledgments and window updates, and the sender adjusts its behavior based on that feedback. Every optimization and every problem in TCP comes down to how quickly and accurately that feedback loop operates.

The Architecture

Here is the full lifecycle of a TCP connection, from handshake through data transfer to teardown.

Walk through what happens:

Phase 1: The three-way handshake. The client sends a SYN with a random initial sequence number (ISN). The server responds with SYN-ACK, acknowledging the client's ISN and proposing its own. The client ACKs the server's ISN. Both sides are now synchronized on sequence numbers. This takes one full round trip (1 RTT) before any data can flow.

Why random ISNs? If sequence numbers started at zero, an attacker who knows the port numbers could predict the next sequence number and inject forged packets into the connection. Random ISNs make this prediction computationally infeasible.

Phase 2: Data transfer. The client sends segments with increasing sequence numbers. The server acknowledges received data by sending back the next expected sequence number. The win field tells the sender how much buffer space the receiver has (flow control). I will go deep on this phase in the next sections.

Phase 3: Teardown. Either side can initiate a FIN. The other side ACKs the FIN, then sends its own FIN when ready. The initiator enters TIME_WAIT for 2 times the Maximum Segment Lifetime (typically 60 seconds on Linux) to handle any delayed packets from the closed connection.

The three-way handshake exists because both sides need to agree on initial sequence numbers. Using random ISNs prevents an attacker from injecting packets into an existing connection (TCP sequence prediction attacks). If the ISNs were predictable, an off-path attacker could forge packets that the receiver would accept.

For your interview: say "TCP uses a three-way handshake to synchronize sequence numbers between both endpoints, then uses those sequence numbers for reliable, ordered delivery" and move to the data transfer phase.

The Three-Way Handshake and Connection State Machine

The handshake is simple in concept but the state machine behind it explains many production behaviors. Understanding the states explains why you see SYN_RECV flooding during DDoS attacks, why TIME_WAIT sockets accumulate on busy servers, and why connection timeouts behave the way they do.

A few states deserve special attention:

SYN_RECEIVED: The server has allocated a Transmission Control Block (TCB) for this half-open connection. During a SYN flood attack, thousands of SYN packets arrive and the server allocates TCB resources for each, never receiving the final ACK. This is why SYN cookies exist: the server encodes state in the SYN-ACK's ISN instead of allocating memory, only creating the TCB when the final ACK arrives.

TIME_WAIT: After the active closer sends its final ACK, it waits for 2 times the Maximum Segment Lifetime (2 x MSL, typically 60 seconds). This ensures that any delayed packets from the old connection do not get misinterpreted by a new connection reusing the same port. On a high-traffic server closing thousands of connections per second, TIME_WAIT sockets can exhaust the local port range.

CLOSE_WAIT: If you see thousands of sockets in CLOSE_WAIT, it means your application received a FIN from the remote side but has not called close() on the socket. This is almost always a bug (resource leak).

TIME_WAIT accumulation is the most common TCP problem on high-traffic servers. If your service handles thousands of short-lived connections per second (like a load balancer), you can exhaust the local port range (65,535 ports). Mitigation: enable tcp_tw_reuse (allows reusing TIME_WAIT sockets for new outgoing connections), use connection pooling to reduce connection churn, and consider SO_LINGER with a zero timeout for connections you know are safe to reset.

Congestion Control Algorithms

This is where TCP gets clever. The network between sender and receiver has finite capacity, and TCP has no way to directly ask "how much bandwidth is available?" Instead, it uses packet loss and latency as indirect signals of congestion.

TCP maintains two windows that limit how much data can be in flight:

Receiver window (rwnd): Advertised by the receiver. "I have this much buffer space."
Congestion window (cwnd): Maintained by the sender. "I think the network can handle this much."

The effective window is min(rwnd, cwnd). The sender can have at most this many unacknowledged bytes in flight at any time.

Here is what each phase does:

Slow start: Start with a tiny window (typically 10 MSS on modern Linux, about 14KB). Double the window every RTT. This sounds slow but is actually exponential growth. From 14KB it takes about 4 RTTs to reach 224KB, which is around 1.5 Mbps on a 100ms link. The name "slow start" is misleading; it is called that because the original TCP sent the entire window at once.

Congestion avoidance: Once cwnd reaches the slow-start threshold (ssthresh), switch to linear growth. Increase cwnd by roughly 1 MSS per RTT. This probes for additional capacity carefully.

Fast retransmit: If the sender receives three duplicate ACKs (the receiver keeps acknowledging the same sequence number), it assumes the next packet was lost and retransmits it immediately without waiting for the retransmission timeout. This is much faster than waiting for the timer.

Fast recovery: After fast retransmit, halve the congestion window and enter congestion avoidance (do not restart from slow start). The logic: three duplicate ACKs means the network is still delivering packets (just one was lost), so conditions are not as bad as a full timeout would suggest.

Timeout: If the retransmission timer expires (RTO, typically starting at 1 second), TCP assumes severe congestion. Reset cwnd to 1 MSS and restart slow start. This is the most aggressive backoff.

I see candidates overcomplicate this. For your interview: "TCP uses slow start to ramp up quickly, congestion avoidance for steady-state probing, and fast retransmit plus fast recovery to handle isolated losses without resetting to zero." That single sentence covers the algorithm.

TCP in System Design: Head-of-Line Blocking and Nagle's Algorithm

This section covers the TCP behaviors that matter most when you are designing systems. These are the details that separate someone who knows TCP theory from someone who has debugged TCP problems in production.

Head-of-Line Blocking

TCP guarantees in-order delivery. If segment 5 of 10 is lost, the receiver buffers segments 6-10 but cannot deliver them to the application until segment 5 arrives (via retransmission). All subsequent data is blocked behind one lost packet.

This is fine for a single request/response flow. But HTTP/2 multiplexes many requests over a single TCP connection. If one multiplexed stream loses a packet, all streams on that connection are blocked, even though the lost packet only belongs to one stream.

This is the fundamental reason HTTP/3 moved from TCP to QUIC (which runs over UDP and implements its own per-stream reliability). With QUIC, a lost packet only blocks the stream it belongs to.

Nagle's Algorithm and Delayed ACKs

Nagle's algorithm, enabled by default on most TCP stacks, buffers small writes. If the sender has unacknowledged data in flight, Nagle holds back any new write smaller than MSS (about 1460 bytes) until either the outstanding data is ACKed or enough data accumulates to fill a full segment.

Delayed ACKs, also on by default, cause the receiver to wait up to 40ms before sending an ACK, hoping to piggyback the ACK on a response packet.

When both are enabled simultaneously, they interact badly: the sender writes a small message and Nagle holds it (waiting for the previous ACK). The receiver applies delayed ACK and waits 40ms before ACKing. The sender waits for that delayed ACK before releasing the buffered data. Result: an artificial 40ms latency on every small message.

The Nagle + delayed ACK interaction is the most common source of mysterious 40ms latency spikes in RPC systems. If you are seeing consistent ~40ms delays on small messages between services, the first thing to check is whether TCP_NODELAY is set on the sockets. Redis, gRPC, and most modern RPC frameworks set TCP_NODELAY by default for exactly this reason.

Here is the interaction in detail. Suppose a client sends a 100-byte RPC request in two writes (header + body):

First write (50 bytes): Nagle sends it immediately (nothing unacknowledged yet).
Second write (50 bytes): Nagle holds it because the first 50 bytes are not yet ACKed.
Server receives the first 50 bytes but delays its ACK by 40ms (delayed ACK timer).
After 40ms, the server sends the ACK, Nagle releases the second 50 bytes.
The server now has the full request, 40ms late.

The fix is a single socket option: setsockopt(fd, IPPROTO_TCP, TCP_NODELAY, &one, sizeof(one)). This disables Nagle entirely. Every write goes out immediately regardless of outstanding ACKs.

For real-time protocols (gaming, trading, interactive RPCs), always set TCP_NODELAY to disable Nagle's algorithm. The small-packet overhead is negligible compared to the latency penalty. For bulk transfers (file uploads, streaming large payloads), keep Nagle enabled. It reduces the number of small packets, which is more efficient for the network.

The rule of thumb: if your protocol sends many small messages and cares about latency, set TCP_NODELAY. If your protocol sends large payloads in bulk, leave Nagle alone. Every RPC framework (gRPC, Thrift, Redis protocol) disables Nagle by default.

The Tricky Parts

TCP keepalive is not the same as HTTP keep-alive. TCP keepalive is an OS-level probe that detects dead connections by sending empty ACKs at intervals (default: 2 hours on Linux). HTTP keep-alive controls whether the TCP connection stays open between HTTP requests. Confusing the two is a common mistake. For service meshes and load balancers, you need both: TCP keepalive to detect dead peers, and HTTP keep-alive to reuse connections.
The receive buffer size limits throughput on high-latency links. TCP throughput is bounded by window_size / RTT. If your receive buffer is 64KB and the RTT is 100ms, maximum throughput is 640KB/s (about 5 Mbps), regardless of the link's capacity. On transcontinental links (200ms RTT), you need large buffers (several MB) or TCP window scaling to saturate the pipe. Linux enables window scaling by default, but the actual buffer size is controlled by net.core.rmem_max.
Connection setup latency dominates short requests. For a request that takes 5ms to process, the TCP handshake (1 RTT) and TLS handshake (1 RTT) add 200ms on a 100ms link. The transport overhead is 40x the actual work. This is why connection pooling is not optional in distributed systems.
TCP does not preserve message boundaries. TCP is a byte-stream protocol. If you call send() twice with 100 bytes each, the receiver might read 50 bytes, then 150 bytes. Your application protocol must handle framing (length-prefix, delimiter, or fixed-size messages). This catches people who assume TCP "sends messages."
Retransmission timeout (RTO) calculation is adaptive but conservative. TCP computes RTO from smoothed RTT measurements using Jacobson's algorithm. The minimum RTO on Linux is 200ms. If a packet is genuinely lost (not just reordered), you wait at least 200ms before retransmitting. On a fast local network where RTT is 0.5ms, this 200ms floor is 400x the actual RTT. This is why fast retransmit (triggered by 3 duplicate ACKs, typically within a few RTTs) is so much faster than waiting for timeout.

What Most People Get Wrong

Mistake	What they say	Why it is wrong	What to say instead
Treating TCP as lossless	"TCP guarantees delivery so I do not need to handle failures"	TCP guarantees delivery within a connection. If the connection drops (timeout, reset), data in flight is lost. Applications still need retry logic.	"TCP retransmits lost segments within a connection, but connection failures still require application-level retry with idempotency."
Ignoring slow start	"Our service can handle 10 Gbps"	A new TCP connection starts at ~14KB/RTT. On a cold connection, it takes many RTTs to reach full speed.	"Warm connections via pooling reach full throughput. New connections ramp up over several RTTs via slow start."
Confusing flow and congestion control	"The receiver tells the sender to slow down"	That is only flow control (rwnd). Congestion control (cwnd) is the sender independently limiting itself based on packet loss. Both constrain the sending rate.	"Flow control prevents overwhelming the receiver. Congestion control prevents overwhelming the network. The effective limit is the minimum of both."
Assuming TCP is fast for small messages	"TCP adds minimal overhead"	SYN + SYN-ACK + ACK = 1 RTT before any data. For a 50-byte RPC, the handshake dominates. Add TLS and it is 2-3 RTT.	"For small messages, connection reuse is critical. The handshake overhead dwarfs the payload size."
Not knowing about TIME_WAIT	"We just close the connection"	The active closer enters TIME_WAIT for 60-120 seconds. On a high-churn server, this exhausts ports.	"The active closer stays in TIME_WAIT for 2xMSL. Use connection pooling and tcp_tw_reuse to manage this."

How I Would Communicate This in an Interview

Here is how I would actually say this:

"TCP provides reliable, ordered byte-stream delivery over an unreliable network. It achieves this through three key mechanisms.

First, the three-way handshake. The client sends SYN, the server responds with SYN-ACK, the client sends ACK. This synchronizes initial sequence numbers between both sides. It costs one round trip before any data flows.

Second, reliable delivery through sequence numbers and acknowledgments. Every byte gets a sequence number. The receiver ACKs the next expected byte. If the sender does not get an ACK within its retransmission timeout, it resends. The sliding window allows multiple segments in flight simultaneously, so we do not wait for each ACK before sending the next segment.

Third, congestion control. TCP starts with a small congestion window and doubles it every round trip during slow start. Once it reaches a threshold, it switches to linear growth. If packet loss is detected via three duplicate ACKs, it halves the window. On a full timeout, it resets to one segment.

For system design, the key takeaway is that TCP connections are expensive to establish and slow to ramp up. Production systems use connection pooling to amortize that cost, set TCP_NODELAY for latency-sensitive RPCs to avoid Nagle's algorithm delays, and monitor TIME_WAIT socket accumulation on high-traffic servers."

Interview Cheat Sheet

Trigger: "How does TCP guarantee delivery?" → "Sequence numbers, cumulative ACKs, and retransmission timers. The sender tracks which bytes are acknowledged and resends anything unconfirmed after a timeout or three duplicate ACKs."
Trigger: "What happens during the three-way handshake?" → "SYN to propose client ISN, SYN-ACK to propose server ISN and acknowledge client's, ACK to confirm. One RTT of latency before data flows."
Trigger: "What is flow control?" → "The receiver advertises its buffer capacity via the window field. The sender never sends more than the receiver can buffer. This prevents receiver overwhelm."
Trigger: "What is congestion control?" → "The sender maintains a congestion window (cwnd) that limits bytes in flight. Slow start doubles cwnd each RTT. Congestion avoidance adds linearly. Loss halves the window. This prevents network collapse."
Trigger: "Why does HTTP/2 have head-of-line blocking?" → "HTTP/2 multiplexes streams over one TCP connection. TCP delivers bytes in order. If one packet is lost, all streams stall until the retransmit arrives. QUIC fixes this with per-stream delivery."
Trigger: "What is Nagle's algorithm?" → "Nagle buffers small writes until the previous segment is ACKed. Combined with delayed ACKs (40ms wait), it introduces 40ms latency on small messages. Disable with TCP_NODELAY for RPCs."
Trigger: "Why do TIME_WAIT sockets accumulate?" → "The active closer stays in TIME_WAIT for 2xMSL (60-120 seconds) to handle delayed packets. High-churn servers exhaust ports. Mitigate with connection pooling and tcp_tw_reuse."
Trigger: "TCP vs UDP for real-time systems?" → "TCP's retransmission and ordering add latency that real-time systems cannot tolerate. UDP lets the application decide what to retransmit. But UDP requires implementing your own reliability if you need it, which is what QUIC does."
Trigger: "Why does connection pooling matter?" → "TCP handshake + TLS handshake + slow start = hundreds of ms before full throughput. Pooling amortizes this across many requests. Every production database driver and HTTP client pools connections."
Trigger: "What is TCP window scaling?" → "The original TCP window field is 16 bits, limiting advertised window to 64KB. Window scaling is a TCP option negotiated in the handshake that allows windows up to 1GB. Required for high-bandwidth, high-latency paths."

Test Your Understanding

Quick Recap

TCP provides reliable, ordered byte-stream delivery using sequence numbers, cumulative acknowledgments, and retransmission timers.
The three-way handshake (SYN, SYN-ACK, ACK) synchronizes initial sequence numbers and costs one round trip before data can flow.
Flow control uses the receiver's advertised window to prevent the sender from overwhelming the receiver's buffer.
Congestion control uses slow start, congestion avoidance, and fast retransmit/recovery to probe network capacity and back off on loss.
Head-of-line blocking in TCP means a single lost packet blocks all multiplexed streams, which is why HTTP/3 uses QUIC over UDP.
Nagle's algorithm and delayed ACKs interact to cause 40ms latency on small messages; set TCP_NODELAY for latency-sensitive RPCs.
TIME_WAIT sockets last 60-120 seconds and can exhaust port ranges on high-churn servers; mitigate with connection pooling and tcp_tw_reuse.
TCP connections are expensive to establish and slow to ramp up, making connection pooling essential for any production distributed system.

How TLS handshake works: TLS runs on top of TCP. Understanding TCP's handshake latency explains why TLS 1.3's 1-RTT design matters (TCP 1 RTT + TLS 1 RTT = 2 RTT minimum connection setup).
How WebSockets work: WebSockets upgrade an HTTP connection (which runs over TCP) to a persistent, full-duplex channel. TCP's keepalive and flow control directly affect WebSocket behavior.
How HTTP/3 and QUIC work: QUIC was built to solve TCP's head-of-line blocking by implementing transport reliability in user space over UDP with per-stream loss recovery.
Networking fundamentals: The broader networking model that TCP sits within, including DNS resolution, IP routing, and how data moves across the internet.

title: "How TCP works" description: "How TCP delivers reliable, ordered byte streams: the three-way handshake, sequence numbers, flow control, congestion control, and why TCP behavior matters when designing distributed systems." tags: ["tcp", "networking", "internals", "distributed-systems"] difficulty: "medium" category: "situational/architecture" order: 16 publishedAt: "2026-04-04" relatedArticles:

"how-tls-handshake-works"
"how-websockets-work"
"networking"

The Problem Statement

Interviewer: "You are building a service that streams sensor data from thousands of IoT devices. Your team is debating whether to use TCP or UDP. Before we get to the tradeoff, walk me through what TCP actually does under the hood. How does it guarantee delivery, ordering, and flow control?"

What you will walk away with after reading this:

The full mechanics of the three-way handshake and why each packet exists.
How sequence numbers and acknowledgments provide reliable, ordered delivery.
How the sliding window and receiver flow control prevent overwhelm.
How congestion control algorithms (slow start, AIMD, fast retransmit) adapt to the network.
Why TCP behaviors like Nagle's algorithm, delayed ACKs, and TIME_WAIT matter in system design.
The production patterns (connection pooling, TCP_NODELAY, keep-alive) that mitigate TCP's overhead.

Clarifying the Scenario

You: "Great question. Let me make sure I scope this correctly."

You: "When you say 'what TCP does under the hood,' do you want me to cover connection setup, data transfer, and teardown as three phases? Or focus on one?"

Interviewer: "Cover all three, but go deeper on the data transfer phase, especially flow and congestion control."

You: "Got it. Should I also discuss how TCP behavior affects system design decisions, like connection pooling and keep-alive?"

Interviewer: "Yes, tie it back to real-world architecture at the end."

My Approach

I break this into four parts:

Connection lifecycle: The three-way handshake, data transfer, and four-way teardown.
Reliable, ordered delivery: How sequence numbers, ACKs, and retransmission timers guarantee that every byte arrives in order.
Flow and congestion control: The sliding window, receiver window, and congestion window working together to maximize throughput without causing collapse.
System design implications: Connection pooling, Nagle's algorithm, TCP_NODELAY, keep-alive, and head-of-line blocking.

Here is a quick reference for the core mechanisms:

Guarantee	Mechanism	How it works
Reliable delivery	Sequence numbers + ACKs	Sender retransmits unacknowledged segments after timeout
Ordered delivery	Sequence numbers	Receiver reassembles out-of-order segments using sequence offsets
Flow control	Receiver window (rwnd)	Receiver advertises how much buffer space it has, sender respects it
Congestion control	Congestion window (cwnd)	Sender probes network capacity gradually, backs off on loss

The Architecture

Here is the full lifecycle of a TCP connection, from handshake through data transfer to teardown.

Walk through what happens:

The Three-Way Handshake and Connection State Machine

A few states deserve special attention:

Congestion Control Algorithms

TCP maintains two windows that limit how much data can be in flight:

Receiver window (rwnd): Advertised by the receiver. "I have this much buffer space."
Congestion window (cwnd): Maintained by the sender. "I think the network can handle this much."

The effective window is min(rwnd, cwnd). The sender can have at most this many unacknowledged bytes in flight at any time.

Here is what each phase does:

Congestion avoidance: Once cwnd reaches the slow-start threshold (ssthresh), switch to linear growth. Increase cwnd by roughly 1 MSS per RTT. This probes for additional capacity carefully.

Timeout: If the retransmission timer expires (RTO, typically starting at 1 second), TCP assumes severe congestion. Reset cwnd to 1 MSS and restart slow start. This is the most aggressive backoff.

TCP in System Design: Head-of-Line Blocking and Nagle's Algorithm

Head-of-Line Blocking

This is the fundamental reason HTTP/3 moved from TCP to QUIC (which runs over UDP and implements its own per-stream reliability). With QUIC, a lost packet only blocks the stream it belongs to.

Nagle's Algorithm and Delayed ACKs

Delayed ACKs, also on by default, cause the receiver to wait up to 40ms before sending an ACK, hoping to piggyback the ACK on a response packet.

Here is the interaction in detail. Suppose a client sends a 100-byte RPC request in two writes (header + body):

First write (50 bytes): Nagle sends it immediately (nothing unacknowledged yet).
Second write (50 bytes): Nagle holds it because the first 50 bytes are not yet ACKed.
Server receives the first 50 bytes but delays its ACK by 40ms (delayed ACK timer).
After 40ms, the server sends the ACK, Nagle releases the second 50 bytes.
The server now has the full request, 40ms late.

The fix is a single socket option: setsockopt(fd, IPPROTO_TCP, TCP_NODELAY, &one, sizeof(one)). This disables Nagle entirely. Every write goes out immediately regardless of outstanding ACKs.

For real-time protocols (gaming, trading, interactive RPCs), always set TCP_NODELAY to disable Nagle's algorithm. The small-packet overhead is negligible compared to the latency penalty.

For bulk transfers (file uploads, streaming large payloads), keep Nagle enabled. It reduces the number of small packets, which is more efficient for the network.

The Tricky Parts

TCP keepalive is not the same as HTTP keep-alive. TCP keepalive is an OS-level probe that detects dead connections by sending empty ACKs at intervals (default: 2 hours on Linux). HTTP keep-alive controls whether the TCP connection stays open between HTTP requests. Confusing the two is a common mistake. For service meshes and load balancers, you need both: TCP keepalive to detect dead peers, and HTTP keep-alive to reuse connections.
The receive buffer size limits throughput on high-latency links. TCP throughput is bounded by window_size / RTT. If your receive buffer is 64KB and the RTT is 100ms, maximum throughput is 640KB/s (about 5 Mbps), regardless of the link's capacity. On transcontinental links (200ms RTT), you need large buffers (several MB) or TCP window scaling to saturate the pipe. Linux enables window scaling by default, but the actual buffer size is controlled by net.core.rmem_max.
Connection setup latency dominates short requests. For a request that takes 5ms to process, the TCP handshake (1 RTT) and TLS handshake (1 RTT) add 200ms on a 100ms link. The transport overhead is 40x the actual work. This is why connection pooling is not optional in distributed systems.
TCP does not preserve message boundaries. TCP is a byte-stream protocol. If you call send() twice with 100 bytes each, the receiver might read 50 bytes, then 150 bytes. Your application protocol must handle framing (length-prefix, delimiter, or fixed-size messages). This catches people who assume TCP "sends messages."
Retransmission timeout (RTO) calculation is adaptive but conservative. TCP computes RTO from smoothed RTT measurements using Jacobson's algorithm. The minimum RTO on Linux is 200ms. If a packet is genuinely lost (not just reordered), you wait at least 200ms before retransmitting. On a fast local network where RTT is 0.5ms, this 200ms floor is 400x the actual RTT. This is why fast retransmit (triggered by 3 duplicate ACKs, typically within a few RTTs) is so much faster than waiting for timeout.

What Most People Get Wrong

Mistake	What they say	Why it is wrong	What to say instead
Treating TCP as lossless	"TCP guarantees delivery so I do not need to handle failures"	TCP guarantees delivery within a connection. If the connection drops (timeout, reset), data in flight is lost. Applications still need retry logic.	"TCP retransmits lost segments within a connection, but connection failures still require application-level retry with idempotency."
Ignoring slow start	"Our service can handle 10 Gbps"	A new TCP connection starts at ~14KB/RTT. On a cold connection, it takes many RTTs to reach full speed.	"Warm connections via pooling reach full throughput. New connections ramp up over several RTTs via slow start."
Confusing flow and congestion control	"The receiver tells the sender to slow down"	That is only flow control (rwnd). Congestion control (cwnd) is the sender independently limiting itself based on packet loss. Both constrain the sending rate.	"Flow control prevents overwhelming the receiver. Congestion control prevents overwhelming the network. The effective limit is the minimum of both."
Assuming TCP is fast for small messages	"TCP adds minimal overhead"	SYN + SYN-ACK + ACK = 1 RTT before any data. For a 50-byte RPC, the handshake dominates. Add TLS and it is 2-3 RTT.	"For small messages, connection reuse is critical. The handshake overhead dwarfs the payload size."
Not knowing about TIME_WAIT	"We just close the connection"	The active closer enters TIME_WAIT for 60-120 seconds. On a high-churn server, this exhausts ports.	"The active closer stays in TIME_WAIT for 2xMSL. Use connection pooling and tcp_tw_reuse to manage this."

How I Would Communicate This in an Interview

Here is how I would actually say this:

"TCP provides reliable, ordered byte-stream delivery over an unreliable network. It achieves this through three key mechanisms.

Interview Cheat Sheet

Trigger: "How does TCP guarantee delivery?" → "Sequence numbers, cumulative ACKs, and retransmission timers. The sender tracks which bytes are acknowledged and resends anything unconfirmed after a timeout or three duplicate ACKs."
Trigger: "What happens during the three-way handshake?" → "SYN to propose client ISN, SYN-ACK to propose server ISN and acknowledge client's, ACK to confirm. One RTT of latency before data flows."
Trigger: "What is flow control?" → "The receiver advertises its buffer capacity via the window field. The sender never sends more than the receiver can buffer. This prevents receiver overwhelm."
Trigger: "What is congestion control?" → "The sender maintains a congestion window (cwnd) that limits bytes in flight. Slow start doubles cwnd each RTT. Congestion avoidance adds linearly. Loss halves the window. This prevents network collapse."
Trigger: "Why does HTTP/2 have head-of-line blocking?" → "HTTP/2 multiplexes streams over one TCP connection. TCP delivers bytes in order. If one packet is lost, all streams stall until the retransmit arrives. QUIC fixes this with per-stream delivery."
Trigger: "What is Nagle's algorithm?" → "Nagle buffers small writes until the previous segment is ACKed. Combined with delayed ACKs (40ms wait), it introduces 40ms latency on small messages. Disable with TCP_NODELAY for RPCs."
Trigger: "Why do TIME_WAIT sockets accumulate?" → "The active closer stays in TIME_WAIT for 2xMSL (60-120 seconds) to handle delayed packets. High-churn servers exhaust ports. Mitigate with connection pooling and tcp_tw_reuse."
Trigger: "TCP vs UDP for real-time systems?" → "TCP's retransmission and ordering add latency that real-time systems cannot tolerate. UDP lets the application decide what to retransmit. But UDP requires implementing your own reliability if you need it, which is what QUIC does."
Trigger: "Why does connection pooling matter?" → "TCP handshake + TLS handshake + slow start = hundreds of ms before full throughput. Pooling amortizes this across many requests. Every production database driver and HTTP client pools connections."
Trigger: "What is TCP window scaling?" → "The original TCP window field is 16 bits, limiting advertised window to 64KB. Window scaling is a TCP option negotiated in the handshake that allows windows up to 1GB. Required for high-bandwidth, high-latency paths."

Test Your Understanding

Quick Recap

TCP provides reliable, ordered byte-stream delivery using sequence numbers, cumulative acknowledgments, and retransmission timers.
The three-way handshake (SYN, SYN-ACK, ACK) synchronizes initial sequence numbers and costs one round trip before data can flow.
Flow control uses the receiver's advertised window to prevent the sender from overwhelming the receiver's buffer.
Congestion control uses slow start, congestion avoidance, and fast retransmit/recovery to probe network capacity and back off on loss.
Head-of-line blocking in TCP means a single lost packet blocks all multiplexed streams, which is why HTTP/3 uses QUIC over UDP.
Nagle's algorithm and delayed ACKs interact to cause 40ms latency on small messages; set TCP_NODELAY for latency-sensitive RPCs.
TIME_WAIT sockets last 60-120 seconds and can exhaust port ranges on high-churn servers; mitigate with connection pooling and tcp_tw_reuse.
TCP connections are expensive to establish and slow to ramp up, making connection pooling essential for any production distributed system.

How TLS handshake works: TLS runs on top of TCP. Understanding TCP's handshake latency explains why TLS 1.3's 1-RTT design matters (TCP 1 RTT + TLS 1 RTT = 2 RTT minimum connection setup).
How WebSockets work: WebSockets upgrade an HTTP connection (which runs over TCP) to a persistent, full-duplex channel. TCP's keepalive and flow control directly affect WebSocket behavior.
How HTTP/3 and QUIC work: QUIC was built to solve TCP's head-of-line blocking by implementing transport reliability in user space over UDP with per-stream loss recovery.
Networking fundamentals: The broader networking model that TCP sits within, including DNS resolution, IP routing, and how data moves across the internet.

The Problem Statement

Interviewer: "You are building a service that streams sensor data from thousands of IoT devices. Your team is debating whether to use TCP or UDP. Before we get to the tradeoff, walk me through what TCP actually does under the hood. How does it guarantee delivery, ordering, and flow control?"

What you will walk away with after reading this:

The full mechanics of the three-way handshake and why each packet exists.
How sequence numbers and acknowledgments provide reliable, ordered delivery.
How the sliding window and receiver flow control prevent overwhelm.
How congestion control algorithms (slow start, AIMD, fast retransmit) adapt to the network.
Why TCP behaviors like Nagle's algorithm, delayed ACKs, and TIME_WAIT matter in system design.
The production patterns (connection pooling, TCP_NODELAY, keep-alive) that mitigate TCP's overhead.

Clarifying the Scenario

You: "Great question. Let me make sure I scope this correctly."

You: "When you say 'what TCP does under the hood,' do you want me to cover connection setup, data transfer, and teardown as three phases? Or focus on one?"

Interviewer: "Cover all three, but go deeper on the data transfer phase, especially flow and congestion control."

You: "Got it. Should I also discuss how TCP behavior affects system design decisions, like connection pooling and keep-alive?"

Interviewer: "Yes, tie it back to real-world architecture at the end."

My Approach

I break this into four parts:

Connection lifecycle: The three-way handshake, data transfer, and four-way teardown.
Reliable, ordered delivery: How sequence numbers, ACKs, and retransmission timers guarantee that every byte arrives in order.
Flow and congestion control: The sliding window, receiver window, and congestion window working together to maximize throughput without causing collapse.
System design implications: Connection pooling, Nagle's algorithm, TCP_NODELAY, keep-alive, and head-of-line blocking.

Here is a quick reference for the core mechanisms:

Guarantee	Mechanism	How it works
Reliable delivery	Sequence numbers + ACKs	Sender retransmits unacknowledged segments after timeout
Ordered delivery	Sequence numbers	Receiver reassembles out-of-order segments using sequence offsets
Flow control	Receiver window (rwnd)	Receiver advertises how much buffer space it has, sender respects it
Congestion control	Congestion window (cwnd)	Sender probes network capacity gradually, backs off on loss

The Architecture

Here is the full lifecycle of a TCP connection, from handshake through data transfer to teardown.

Walk through what happens:

The Three-Way Handshake and Connection State Machine

A few states deserve special attention:

Congestion Control Algorithms

TCP maintains two windows that limit how much data can be in flight:

Receiver window (rwnd): Advertised by the receiver. "I have this much buffer space."
Congestion window (cwnd): Maintained by the sender. "I think the network can handle this much."

The effective window is min(rwnd, cwnd). The sender can have at most this many unacknowledged bytes in flight at any time.

Here is what each phase does:

Congestion avoidance: Once cwnd reaches the slow-start threshold (ssthresh), switch to linear growth. Increase cwnd by roughly 1 MSS per RTT. This probes for additional capacity carefully.

Timeout: If the retransmission timer expires (RTO, typically starting at 1 second), TCP assumes severe congestion. Reset cwnd to 1 MSS and restart slow start. This is the most aggressive backoff.

TCP in System Design: Head-of-Line Blocking and Nagle's Algorithm

Head-of-Line Blocking

This is the fundamental reason HTTP/3 moved from TCP to QUIC (which runs over UDP and implements its own per-stream reliability). With QUIC, a lost packet only blocks the stream it belongs to.

Nagle's Algorithm and Delayed ACKs

Delayed ACKs, also on by default, cause the receiver to wait up to 40ms before sending an ACK, hoping to piggyback the ACK on a response packet.

Here is the interaction in detail. Suppose a client sends a 100-byte RPC request in two writes (header + body):

First write (50 bytes): Nagle sends it immediately (nothing unacknowledged yet).
Second write (50 bytes): Nagle holds it because the first 50 bytes are not yet ACKed.
Server receives the first 50 bytes but delays its ACK by 40ms (delayed ACK timer).
After 40ms, the server sends the ACK, Nagle releases the second 50 bytes.
The server now has the full request, 40ms late.

The fix is a single socket option: setsockopt(fd, IPPROTO_TCP, TCP_NODELAY, &one, sizeof(one)). This disables Nagle entirely. Every write goes out immediately regardless of outstanding ACKs.

The Tricky Parts

TCP keepalive is not the same as HTTP keep-alive. TCP keepalive is an OS-level probe that detects dead connections by sending empty ACKs at intervals (default: 2 hours on Linux). HTTP keep-alive controls whether the TCP connection stays open between HTTP requests. Confusing the two is a common mistake. For service meshes and load balancers, you need both: TCP keepalive to detect dead peers, and HTTP keep-alive to reuse connections.
The receive buffer size limits throughput on high-latency links. TCP throughput is bounded by window_size / RTT. If your receive buffer is 64KB and the RTT is 100ms, maximum throughput is 640KB/s (about 5 Mbps), regardless of the link's capacity. On transcontinental links (200ms RTT), you need large buffers (several MB) or TCP window scaling to saturate the pipe. Linux enables window scaling by default, but the actual buffer size is controlled by net.core.rmem_max.
Connection setup latency dominates short requests. For a request that takes 5ms to process, the TCP handshake (1 RTT) and TLS handshake (1 RTT) add 200ms on a 100ms link. The transport overhead is 40x the actual work. This is why connection pooling is not optional in distributed systems.
TCP does not preserve message boundaries. TCP is a byte-stream protocol. If you call send() twice with 100 bytes each, the receiver might read 50 bytes, then 150 bytes. Your application protocol must handle framing (length-prefix, delimiter, or fixed-size messages). This catches people who assume TCP "sends messages."
Retransmission timeout (RTO) calculation is adaptive but conservative. TCP computes RTO from smoothed RTT measurements using Jacobson's algorithm. The minimum RTO on Linux is 200ms. If a packet is genuinely lost (not just reordered), you wait at least 200ms before retransmitting. On a fast local network where RTT is 0.5ms, this 200ms floor is 400x the actual RTT. This is why fast retransmit (triggered by 3 duplicate ACKs, typically within a few RTTs) is so much faster than waiting for timeout.

What Most People Get Wrong

Mistake	What they say	Why it is wrong	What to say instead
Treating TCP as lossless	"TCP guarantees delivery so I do not need to handle failures"	TCP guarantees delivery within a connection. If the connection drops (timeout, reset), data in flight is lost. Applications still need retry logic.	"TCP retransmits lost segments within a connection, but connection failures still require application-level retry with idempotency."
Ignoring slow start	"Our service can handle 10 Gbps"	A new TCP connection starts at ~14KB/RTT. On a cold connection, it takes many RTTs to reach full speed.	"Warm connections via pooling reach full throughput. New connections ramp up over several RTTs via slow start."
Confusing flow and congestion control	"The receiver tells the sender to slow down"	That is only flow control (rwnd). Congestion control (cwnd) is the sender independently limiting itself based on packet loss. Both constrain the sending rate.	"Flow control prevents overwhelming the receiver. Congestion control prevents overwhelming the network. The effective limit is the minimum of both."
Assuming TCP is fast for small messages	"TCP adds minimal overhead"	SYN + SYN-ACK + ACK = 1 RTT before any data. For a 50-byte RPC, the handshake dominates. Add TLS and it is 2-3 RTT.	"For small messages, connection reuse is critical. The handshake overhead dwarfs the payload size."
Not knowing about TIME_WAIT	"We just close the connection"	The active closer enters TIME_WAIT for 60-120 seconds. On a high-churn server, this exhausts ports.	"The active closer stays in TIME_WAIT for 2xMSL. Use connection pooling and tcp_tw_reuse to manage this."

How I Would Communicate This in an Interview

Here is how I would actually say this:

"TCP provides reliable, ordered byte-stream delivery over an unreliable network. It achieves this through three key mechanisms.

Interview Cheat Sheet

Trigger: "How does TCP guarantee delivery?" → "Sequence numbers, cumulative ACKs, and retransmission timers. The sender tracks which bytes are acknowledged and resends anything unconfirmed after a timeout or three duplicate ACKs."
Trigger: "What happens during the three-way handshake?" → "SYN to propose client ISN, SYN-ACK to propose server ISN and acknowledge client's, ACK to confirm. One RTT of latency before data flows."
Trigger: "What is flow control?" → "The receiver advertises its buffer capacity via the window field. The sender never sends more than the receiver can buffer. This prevents receiver overwhelm."
Trigger: "What is congestion control?" → "The sender maintains a congestion window (cwnd) that limits bytes in flight. Slow start doubles cwnd each RTT. Congestion avoidance adds linearly. Loss halves the window. This prevents network collapse."
Trigger: "Why does HTTP/2 have head-of-line blocking?" → "HTTP/2 multiplexes streams over one TCP connection. TCP delivers bytes in order. If one packet is lost, all streams stall until the retransmit arrives. QUIC fixes this with per-stream delivery."
Trigger: "What is Nagle's algorithm?" → "Nagle buffers small writes until the previous segment is ACKed. Combined with delayed ACKs (40ms wait), it introduces 40ms latency on small messages. Disable with TCP_NODELAY for RPCs."
Trigger: "Why do TIME_WAIT sockets accumulate?" → "The active closer stays in TIME_WAIT for 2xMSL (60-120 seconds) to handle delayed packets. High-churn servers exhaust ports. Mitigate with connection pooling and tcp_tw_reuse."
Trigger: "TCP vs UDP for real-time systems?" → "TCP's retransmission and ordering add latency that real-time systems cannot tolerate. UDP lets the application decide what to retransmit. But UDP requires implementing your own reliability if you need it, which is what QUIC does."
Trigger: "Why does connection pooling matter?" → "TCP handshake + TLS handshake + slow start = hundreds of ms before full throughput. Pooling amortizes this across many requests. Every production database driver and HTTP client pools connections."
Trigger: "What is TCP window scaling?" → "The original TCP window field is 16 bits, limiting advertised window to 64KB. Window scaling is a TCP option negotiated in the handshake that allows windows up to 1GB. Required for high-bandwidth, high-latency paths."

Test Your Understanding

Quick Recap

TCP provides reliable, ordered byte-stream delivery using sequence numbers, cumulative acknowledgments, and retransmission timers.
The three-way handshake (SYN, SYN-ACK, ACK) synchronizes initial sequence numbers and costs one round trip before data can flow.
Flow control uses the receiver's advertised window to prevent the sender from overwhelming the receiver's buffer.
Congestion control uses slow start, congestion avoidance, and fast retransmit/recovery to probe network capacity and back off on loss.
Head-of-line blocking in TCP means a single lost packet blocks all multiplexed streams, which is why HTTP/3 uses QUIC over UDP.
Nagle's algorithm and delayed ACKs interact to cause 40ms latency on small messages; set TCP_NODELAY for latency-sensitive RPCs.
TIME_WAIT sockets last 60-120 seconds and can exhaust port ranges on high-churn servers; mitigate with connection pooling and tcp_tw_reuse.
TCP connections are expensive to establish and slow to ramp up, making connection pooling essential for any production distributed system.

How TLS handshake works: TLS runs on top of TCP. Understanding TCP's handshake latency explains why TLS 1.3's 1-RTT design matters (TCP 1 RTT + TLS 1 RTT = 2 RTT minimum connection setup).
How WebSockets work: WebSockets upgrade an HTTP connection (which runs over TCP) to a persistent, full-duplex channel. TCP's keepalive and flow control directly affect WebSocket behavior.
How HTTP/3 and QUIC work: QUIC was built to solve TCP's head-of-line blocking by implementing transport reliability in user space over UDP with per-stream loss recovery.
Networking fundamentals: The broader networking model that TCP sits within, including DNS resolution, IP routing, and how data moves across the internet.

"how-tls-handshake-works"
"how-websockets-work"
"networking"

The Problem Statement

Interviewer: "You are building a service that streams sensor data from thousands of IoT devices. Your team is debating whether to use TCP or UDP. Before we get to the tradeoff, walk me through what TCP actually does under the hood. How does it guarantee delivery, ordering, and flow control?"

What you will walk away with after reading this:

The full mechanics of the three-way handshake and why each packet exists.
How sequence numbers and acknowledgments provide reliable, ordered delivery.
How the sliding window and receiver flow control prevent overwhelm.
How congestion control algorithms (slow start, AIMD, fast retransmit) adapt to the network.
Why TCP behaviors like Nagle's algorithm, delayed ACKs, and TIME_WAIT matter in system design.
The production patterns (connection pooling, TCP_NODELAY, keep-alive) that mitigate TCP's overhead.

Clarifying the Scenario

You: "Great question. Let me make sure I scope this correctly."

You: "When you say 'what TCP does under the hood,' do you want me to cover connection setup, data transfer, and teardown as three phases? Or focus on one?"

Interviewer: "Cover all three, but go deeper on the data transfer phase, especially flow and congestion control."

You: "Got it. Should I also discuss how TCP behavior affects system design decisions, like connection pooling and keep-alive?"

Interviewer: "Yes, tie it back to real-world architecture at the end."

My Approach

I break this into four parts:

Connection lifecycle: The three-way handshake, data transfer, and four-way teardown.
Reliable, ordered delivery: How sequence numbers, ACKs, and retransmission timers guarantee that every byte arrives in order.
Flow and congestion control: The sliding window, receiver window, and congestion window working together to maximize throughput without causing collapse.
System design implications: Connection pooling, Nagle's algorithm, TCP_NODELAY, keep-alive, and head-of-line blocking.

Here is a quick reference for the core mechanisms:

Guarantee	Mechanism	How it works
Reliable delivery	Sequence numbers + ACKs	Sender retransmits unacknowledged segments after timeout
Ordered delivery	Sequence numbers	Receiver reassembles out-of-order segments using sequence offsets
Flow control	Receiver window (rwnd)	Receiver advertises how much buffer space it has, sender respects it
Congestion control	Congestion window (cwnd)	Sender probes network capacity gradually, backs off on loss

The Architecture

Here is the full lifecycle of a TCP connection, from handshake through data transfer to teardown.

Walk through what happens:

The Three-Way Handshake and Connection State Machine

A few states deserve special attention:

Congestion Control Algorithms

TCP maintains two windows that limit how much data can be in flight:

Receiver window (rwnd): Advertised by the receiver. "I have this much buffer space."
Congestion window (cwnd): Maintained by the sender. "I think the network can handle this much."

The effective window is min(rwnd, cwnd). The sender can have at most this many unacknowledged bytes in flight at any time.

Here is what each phase does:

Congestion avoidance: Once cwnd reaches the slow-start threshold (ssthresh), switch to linear growth. Increase cwnd by roughly 1 MSS per RTT. This probes for additional capacity carefully.

Timeout: If the retransmission timer expires (RTO, typically starting at 1 second), TCP assumes severe congestion. Reset cwnd to 1 MSS and restart slow start. This is the most aggressive backoff.

TCP in System Design: Head-of-Line Blocking and Nagle's Algorithm

Head-of-Line Blocking

This is the fundamental reason HTTP/3 moved from TCP to QUIC (which runs over UDP and implements its own per-stream reliability). With QUIC, a lost packet only blocks the stream it belongs to.

Nagle's Algorithm and Delayed ACKs

Delayed ACKs, also on by default, cause the receiver to wait up to 40ms before sending an ACK, hoping to piggyback the ACK on a response packet.

Here is the interaction in detail. Suppose a client sends a 100-byte RPC request in two writes (header + body):

First write (50 bytes): Nagle sends it immediately (nothing unacknowledged yet).
Second write (50 bytes): Nagle holds it because the first 50 bytes are not yet ACKed.
Server receives the first 50 bytes but delays its ACK by 40ms (delayed ACK timer).
After 40ms, the server sends the ACK, Nagle releases the second 50 bytes.
The server now has the full request, 40ms late.

The fix is a single socket option: setsockopt(fd, IPPROTO_TCP, TCP_NODELAY, &one, sizeof(one)). This disables Nagle entirely. Every write goes out immediately regardless of outstanding ACKs.

For real-time protocols (gaming, trading, interactive RPCs), always set TCP_NODELAY to disable Nagle's algorithm. The small-packet overhead is negligible compared to the latency penalty.

For bulk transfers (file uploads, streaming large payloads), keep Nagle enabled. It reduces the number of small packets, which is more efficient for the network.

The Tricky Parts

TCP keepalive is not the same as HTTP keep-alive. TCP keepalive is an OS-level probe that detects dead connections by sending empty ACKs at intervals (default: 2 hours on Linux). HTTP keep-alive controls whether the TCP connection stays open between HTTP requests. Confusing the two is a common mistake. For service meshes and load balancers, you need both: TCP keepalive to detect dead peers, and HTTP keep-alive to reuse connections.
The receive buffer size limits throughput on high-latency links. TCP throughput is bounded by window_size / RTT. If your receive buffer is 64KB and the RTT is 100ms, maximum throughput is 640KB/s (about 5 Mbps), regardless of the link's capacity. On transcontinental links (200ms RTT), you need large buffers (several MB) or TCP window scaling to saturate the pipe. Linux enables window scaling by default, but the actual buffer size is controlled by net.core.rmem_max.
Connection setup latency dominates short requests. For a request that takes 5ms to process, the TCP handshake (1 RTT) and TLS handshake (1 RTT) add 200ms on a 100ms link. The transport overhead is 40x the actual work. This is why connection pooling is not optional in distributed systems.
TCP does not preserve message boundaries. TCP is a byte-stream protocol. If you call send() twice with 100 bytes each, the receiver might read 50 bytes, then 150 bytes. Your application protocol must handle framing (length-prefix, delimiter, or fixed-size messages). This catches people who assume TCP "sends messages."
Retransmission timeout (RTO) calculation is adaptive but conservative. TCP computes RTO from smoothed RTT measurements using Jacobson's algorithm. The minimum RTO on Linux is 200ms. If a packet is genuinely lost (not just reordered), you wait at least 200ms before retransmitting. On a fast local network where RTT is 0.5ms, this 200ms floor is 400x the actual RTT. This is why fast retransmit (triggered by 3 duplicate ACKs, typically within a few RTTs) is so much faster than waiting for timeout.

What Most People Get Wrong

Mistake	What they say	Why it is wrong	What to say instead
Treating TCP as lossless	"TCP guarantees delivery so I do not need to handle failures"	TCP guarantees delivery within a connection. If the connection drops (timeout, reset), data in flight is lost. Applications still need retry logic.	"TCP retransmits lost segments within a connection, but connection failures still require application-level retry with idempotency."
Ignoring slow start	"Our service can handle 10 Gbps"	A new TCP connection starts at ~14KB/RTT. On a cold connection, it takes many RTTs to reach full speed.	"Warm connections via pooling reach full throughput. New connections ramp up over several RTTs via slow start."
Confusing flow and congestion control	"The receiver tells the sender to slow down"	That is only flow control (rwnd). Congestion control (cwnd) is the sender independently limiting itself based on packet loss. Both constrain the sending rate.	"Flow control prevents overwhelming the receiver. Congestion control prevents overwhelming the network. The effective limit is the minimum of both."
Assuming TCP is fast for small messages	"TCP adds minimal overhead"	SYN + SYN-ACK + ACK = 1 RTT before any data. For a 50-byte RPC, the handshake dominates. Add TLS and it is 2-3 RTT.	"For small messages, connection reuse is critical. The handshake overhead dwarfs the payload size."
Not knowing about TIME_WAIT	"We just close the connection"	The active closer enters TIME_WAIT for 60-120 seconds. On a high-churn server, this exhausts ports.	"The active closer stays in TIME_WAIT for 2xMSL. Use connection pooling and tcp_tw_reuse to manage this."

How I Would Communicate This in an Interview

Here is how I would actually say this:

"TCP provides reliable, ordered byte-stream delivery over an unreliable network. It achieves this through three key mechanisms.

Interview Cheat Sheet

Trigger: "How does TCP guarantee delivery?" → "Sequence numbers, cumulative ACKs, and retransmission timers. The sender tracks which bytes are acknowledged and resends anything unconfirmed after a timeout or three duplicate ACKs."
Trigger: "What happens during the three-way handshake?" → "SYN to propose client ISN, SYN-ACK to propose server ISN and acknowledge client's, ACK to confirm. One RTT of latency before data flows."
Trigger: "What is flow control?" → "The receiver advertises its buffer capacity via the window field. The sender never sends more than the receiver can buffer. This prevents receiver overwhelm."
Trigger: "What is congestion control?" → "The sender maintains a congestion window (cwnd) that limits bytes in flight. Slow start doubles cwnd each RTT. Congestion avoidance adds linearly. Loss halves the window. This prevents network collapse."
Trigger: "Why does HTTP/2 have head-of-line blocking?" → "HTTP/2 multiplexes streams over one TCP connection. TCP delivers bytes in order. If one packet is lost, all streams stall until the retransmit arrives. QUIC fixes this with per-stream delivery."
Trigger: "What is Nagle's algorithm?" → "Nagle buffers small writes until the previous segment is ACKed. Combined with delayed ACKs (40ms wait), it introduces 40ms latency on small messages. Disable with TCP_NODELAY for RPCs."
Trigger: "Why do TIME_WAIT sockets accumulate?" → "The active closer stays in TIME_WAIT for 2xMSL (60-120 seconds) to handle delayed packets. High-churn servers exhaust ports. Mitigate with connection pooling and tcp_tw_reuse."
Trigger: "TCP vs UDP for real-time systems?" → "TCP's retransmission and ordering add latency that real-time systems cannot tolerate. UDP lets the application decide what to retransmit. But UDP requires implementing your own reliability if you need it, which is what QUIC does."
Trigger: "Why does connection pooling matter?" → "TCP handshake + TLS handshake + slow start = hundreds of ms before full throughput. Pooling amortizes this across many requests. Every production database driver and HTTP client pools connections."
Trigger: "What is TCP window scaling?" → "The original TCP window field is 16 bits, limiting advertised window to 64KB. Window scaling is a TCP option negotiated in the handshake that allows windows up to 1GB. Required for high-bandwidth, high-latency paths."

Test Your Understanding

Quick Recap

TCP provides reliable, ordered byte-stream delivery using sequence numbers, cumulative acknowledgments, and retransmission timers.
The three-way handshake (SYN, SYN-ACK, ACK) synchronizes initial sequence numbers and costs one round trip before data can flow.
Flow control uses the receiver's advertised window to prevent the sender from overwhelming the receiver's buffer.
Congestion control uses slow start, congestion avoidance, and fast retransmit/recovery to probe network capacity and back off on loss.
Head-of-line blocking in TCP means a single lost packet blocks all multiplexed streams, which is why HTTP/3 uses QUIC over UDP.
Nagle's algorithm and delayed ACKs interact to cause 40ms latency on small messages; set TCP_NODELAY for latency-sensitive RPCs.
TIME_WAIT sockets last 60-120 seconds and can exhaust port ranges on high-churn servers; mitigate with connection pooling and tcp_tw_reuse.
TCP connections are expensive to establish and slow to ramp up, making connection pooling essential for any production distributed system.

How TLS handshake works: TLS runs on top of TCP. Understanding TCP's handshake latency explains why TLS 1.3's 1-RTT design matters (TCP 1 RTT + TLS 1 RTT = 2 RTT minimum connection setup).
How WebSockets work: WebSockets upgrade an HTTP connection (which runs over TCP) to a persistent, full-duplex channel. TCP's keepalive and flow control directly affect WebSocket behavior.
How HTTP/3 and QUIC work: QUIC was built to solve TCP's head-of-line blocking by implementing transport reliability in user space over UDP with per-stream loss recovery.
Networking fundamentals: The broader networking model that TCP sits within, including DNS resolution, IP routing, and how data moves across the internet.

Related Articles

Comments

Related Articles

Comments