TCP flow control and congestion control: windows, RTT and throughput
Separate receiver capacity from network congestion, estimate bandwidth-delay product and diagnose a slow transfer using window, loss and application evidence.
TL;DR: Flow control protects the receiver's available buffer space; congestion control limits pressure on the network path. A fast network link cannot deliver high throughput when the permitted in-flight data, round-trip time or application production rate is the tighter constraint.
Two windows represent different limits
The receiver advertises how much data it can accept through its receive window. The sender maintains a congestion window according to its congestion-control algorithm and observations. The usable in-flight allowance is constrained by both, along with data already outstanding and implementation details.
A receiver advertising a small or zero window may have an application that is not reading quickly enough. Loss or congestion signals can cause the sender to reduce its sending behavior even when the receiver has ample space. These diagnoses lead to different repairs.
RFC 5681 describes classic TCP congestion-control mechanisms. Implementations may use other algorithms, so do not apply one algorithm's precise window evolution to every modern connection. Linux's TCP reference describes relevant controls and behavior.
Estimate the in-flight data needed
Bandwidth-delay product estimates how much data must be in flight to fill a path at a target throughput. At an illustrative 100 megabits per second and 80 milliseconds round-trip time, the product is 8 megabits, or about 1 megabyte using decimal units.
If a transfer can keep only 64 KiB in flight, a simple window-over-RTT estimate gives about 819,200 bytes per second, or 6.55 megabits per second. That estimate ignores protocol overhead and other limits, but explains why a nominal 100-megabit path can appear mostly idle.
Do not immediately set every buffer to one megabyte. Check negotiated window scaling, socket behavior, actual RTT and memory effects across many connections. Larger buffers can absorb more data while also permitting longer queues. Queueing behavior still governs latency.
Worked diagnosis: backup throughput collapses
Suppose a backup stream previously moved data quickly, then falls to a fraction of its expected rate after the destination begins heavy disk compaction. The network interface is not saturated. Packet evidence shows the receiver advertising a shrinking window while its application spends longer waiting for storage.
Increasing the sender's congestion-control aggressiveness cannot make the receiver consume data faster. Inspect destination I/O, application read behavior and buffer pressure. Reschedule competing work or repair the storage bottleneck, then verify the receive-window behavior and end-to-end completion time.
In a different case, receive windows stay large but retransmissions and RTT rise under shared-path load. Investigate congestion, loss and competing traffic. The symptoms may justify pacing or a different path, but a packet capture from one side still needs corroboration before blaming a specific network device.
| Evidence | Likely limiting mechanism to investigate | Misleading first action |
|---|---|---|
| Small advertised receive window | Receiver buffer/application consumption | Increase sender retry count |
| Large RTT with insufficient in-flight data | Window and bandwidth-delay product | Assume link capacity alone decides speed |
| Loss and congestion-window reduction | Path congestion or loss source | Increase receiver buffer indiscriminately |
| Empty send queue and idle connection | Application is not producing data | Tune TCP before profiling the application |
| Rising queue delay under load | Excess buffering or capacity pressure | Optimize only peak throughput |
Distinguish throughput from useful performance
A bulk backup can tolerate different latency from an interactive request. Filling a link with a long transfer may improve its throughput while degrading response time for competing workloads. Define the performance objective and observation interval before judging a tuning change.
Measure application completion time alongside transport metrics. Compression can reduce bytes while adding CPU time; encryption or storage can cap throughput before the network. Multiple parallel connections may improve a particular transfer but can also obscure a per-connection limit or unfairly consume shared capacity.
Use a controlled before/after experiment with the same dataset and path where possible. Record concurrency and competing load. Without those details, a reported speedup can be caused by a quieter network rather than the changed socket setting.
Reason from the limiter
Is a zero window evidence of network congestion? It directly indicates receiver-advertised flow-control pressure. Investigate why the receiver cannot accept more data; network congestion may coexist but is a separate mechanism.
Does doubling bandwidth halve transfer time? Only if bandwidth was the controlling limit. RTT, windows, application work and destination storage can prevent the expected improvement.
Why inspect the handshake first? Connection establishment confirms the transport path and negotiated options before throughput analysis. A connection that repeatedly fails to establish has a different problem from an established stream constrained by its receiver.