Two pods both believed they held the lock, so the nightly job ran twice. Design locking that survives stalls, pauses, and clock lies.

Leases prevent dead holders but not overlapping ones. The staff-level answer is fencing tokens and a candid account of why quorum lock services without downstream checks are still probabilistic.

Updated Sep 2026 · Grounded in researched DevOps, SRE and platform engineering interview loops, written to a senior-engineer editorial bar, and never padded to hit a word count.