Container PID 1 and termination: signals, children and shutdown
Trace termination signals to the actual application process. Explain exec-form entrypoints, child reaping and Kubernetes grace periods with a shutdown timeline.
TL;DR: The container's main process must receive and handle the termination signal, stop accepting work and finish or cancel existing work within its grace period. Check the process tree; a shell wrapper can prevent the application from receiving the signal you expected.
PID 1 is the process the runtime starts
Inside a normal private container PID namespace, the initial process has PID 1. If that process is a shell launching another program, the application may be a child rather than the process the runtime addresses. Signal forwarding and child cleanup then depend on the wrapper's behavior.
Docker's Dockerfile reference distinguishes shell and exec forms. An exec-form entrypoint such as the following starts the named program directly:
ENTRYPOINT ["/usr/local/bin/worker"]
This line is a Dockerfile instruction, assuming the executable already exists in the image. A wrapper that finishes setup and then uses exec can replace itself with the application instead of leaving an intermediate shell.
#!/bin/sh
set -eu
exec "$@"
The wrapper must be invoked with the actual executable and its arguments. If it instead launches the program in the background and waits, it needs deliberate signal forwarding and child-reaping behavior. Keeping a shell alive as PID 1 without that design is a common source of confusing shutdowns.
Delivery of a signal is only the beginning
Receiving SIGTERM does not automatically finish an HTTP response or commit a job acknowledgment. The application needs a shutdown path. It should stop admitting new work, notify relevant internal components and wait for eligible work within a bounded deadline.
For a queue worker, admission means stopping new message acquisition. Finishing an existing message includes its durable effect and acknowledgment policy. If the worker exits after the effect but before acknowledgment, redelivery may occur; idempotent processing remains necessary.
Account for Kubernetes lifecycle time
Kubernetes documents termination in the Pod lifecycle reference. A configured preStop hook runs before the normal termination signal, and the hook consumes the termination grace period. After the applicable grace expires, remaining processes can be forcibly stopped. Sidecars and version-specific lifecycle features require checking the cluster's behavior.
In an illustrative Pod with a 30-second grace period, a preStop operation taking eight seconds leaves roughly 22 seconds in the main budget. Do not then promise the application another independent 30 seconds. Account for hook time, propagation and cleanup together, and avoid depending on exceptional extensions as the normal shutdown design.
| Observation | Likely investigation | Evidence to collect |
|---|---|---|
| App logs no shutdown event | Signal did not reach handler or handler absent | Process tree and entrypoint |
| App handles signal but stops immediately | Handler does not await active work | Shutdown code and active-request count |
| Hook runs until grace expires | Hook consumes available time | Hook duration and termination events |
| Process exits only after forced stop | Drain blocked or signal mishandled | Thread/task state and deadline behavior |
| Child processes accumulate | Parent does not reap exited children | Process table and parent relationships |
Zombie processes are exited children whose status has not been collected. A small init process can help with forwarding and reaping when the workload spawns children, but it cannot implement the application's business-level drain procedure.
Test a rolling replacement under load
Start a request or job that lasts near the service's supported upper duration, then terminate the container through the orchestrator. Record when admission stops, when the main process receives the signal and whether the original operation completes. Test cancellation too: some operations should be abandoned promptly when their caller disappears.
Repeat with the real entrypoint and runtime user from the production image. A local development command may bypass the wrapper that causes the failure in Kubernetes. Use multi-stage image verification to ensure tests exercise the final image's actual process configuration.
Coordinate this with load balancer draining. Endpoint changes and process termination can propagate concurrently. The application may still receive a request briefly after removal begins, so readiness alone cannot substitute for a shutdown handler.
Explain an interrupted request precisely
Self-check: a Pod has a 60-second grace period, but requests reset immediately when it is deleted. Should the first fix be increasing the period?
Inspect whether the application exits immediately, whether a shell intercepts the signal and whether an upstream connection is closed first. A longer grace period cannot help a process that has already exited. Change the component responsible for the early termination, then verify that the remaining duration is sufficient for the supported work.