TCP connections: handshakes, retransmissions, FIN and reset evidence
Read TCP connection establishment and teardown as evidence. Distinguish dropped SYNs, resets, orderly close and application timeouts without assuming an open port proves service health.
TL;DR: A TCP handshake establishes a transport connection, not application health. Locate the failed exchange and distinguish retransmission, orderly close and reset before changing network or timeout settings.
The handshake proves a limited claim
A typical connection starts with SYN, SYN-ACK and ACK, establishing sequence-number state in both directions. A successful handshake indicates that the endpoints reached the transport state needed to exchange bytes. TLS negotiation, application authentication and request processing happen afterward.
TCP provides an ordered byte stream rather than application message boundaries. One application write need not correspond to one packet or one read. RFC 9293 specifies the protocol and connection states; use application framing when deciding whether a complete message arrived.
This illustrates an ordinary active close. Packets can combine flags, both sides can initiate closure, and errors can terminate a connection differently.
Interpret missing packets cautiously
Repeated SYNs with no observed reply suggest loss, filtering, routing or an unavailable endpoint. A capture from one host only shows what reached that observation point. It cannot prove that the server never sent a response.
A reset is an explicit transport-level rejection or termination. It can originate from an endpoint or an intermediary, and the cause may include a closed port or an aborted established connection. Inspect direction, timing and surrounding state before assigning responsibility.
A FIN closes one sending direction orderly. Half-close behavior allows the opposite direction to continue when the application protocol supports it. A connection in TIME_WAIT can be part of correct teardown behavior; its presence is not evidence that a process is stuck.
Linux's TCP manual documents implementation behavior and relevant socket controls. Tune those controls only after the packet and application evidence identifies a mechanism.
Worked incident: port check passes, checkout times out
Suppose a TCP port check to an API succeeds, but checkout calls time out. The initial handshake completes quickly. A trace then shows the client sends application data and waits while the server's handler is blocked on a database pool.
The successful port check rules out some connection-establishment failures for that test path. It does not validate the handler or dependency. Inspect request traces and queueing before widening firewall rules. Increasing the client timeout might allow more blocked requests to accumulate and worsen saturation.
Now change the evidence: SYNs leave the client, but the server capture never sees them. The investigation shifts toward the route, security controls and intermediate devices. Compare the return path too; a SYN reaching the server with replies disappearing is a different fault from a missing forward packet.
| Observed evidence | Useful hypothesis | Next check |
|---|---|---|
| SYN retransmissions, no reply observed | Path loss, filter or unreachable listener | Capture at another boundary and inspect routes |
| Immediate reset after SYN | Explicit rejection or no listener | Confirm reset origin and server listening state |
| Handshake succeeds, TLS fails | Transport works but negotiation fails | Certificate, protocol and TLS error evidence |
| Request sent, response delayed | Application or dependency waiting | Trace handler and queue time |
| Many short-lived connections in TIME_WAIT | Frequent connection turnover | Pooling, connection rate and actual resource limits |
Preserve context in packet analysis
Record client/server addresses and ports, capture location and timestamp. NAT and load balancing can change identifiers between observation points. Match the translated connection rather than assuming both captures use identical tuples.
Encryption limits application visibility in a packet capture. You can still inspect timing and transport behavior, but do not infer response content you cannot observe. Packet capture offloading can also make host-side traces look different from wire-level segmentation; corroborate unusual packet shapes before declaring protocol corruption.
Use bounded captures with appropriate access controls because traffic can contain sensitive data. A compact excerpt around the failed handshake often answers more than an uncontrolled capture of all production traffic.
Connect transport to request policy
A TCP retransmission repairs transport loss when possible. It is distinct from an application retry that may repeat a business operation. Deadline propagation controls how long the request remains useful, while flow and congestion control explain throughput limits after connection establishment.
Does an open TCP port prove a database is accepting useful work? No. The listener can accept a connection while authentication, query execution or storage is unhealthy.
Should TIME_WAIT be eliminated to improve performance? First prove a resource constraint and examine connection reuse. Removing a normal protocol safety behavior without a workload model can create harder failures.