storage

A routine cloud operation with three layers that can quietly disagree. The strong answer is an ordered procedure you could run during business hours, without a reboot.

Copy-on-write freezes a point-in-time view while the database keeps writing. Fat snapshots overflow into invalid backups and thin pools exhaust into frozen volumes: size both deliberately.

S3 access failures are rarely about the bucket policy alone. The caller, object ownership and network path determine which authorization checks apply. Start with the failed request and trace changes across those boundaries.

Both run pods, so the answer sounds obvious until an interviewer asks what happens when you delete one. The guarantees are narrower and more specific than most candidates think.

Almost always a finalizer, and the fast fix is the one that loses data. The safe answer explains what the finalizer is protecting against before removing it.

Emergency mode after a storage change is usually one line in fstab. What interviewers score is your order of operations and whether your recovery preserves data-mount dependencies.

NFS waits can stall shutdown and inflate load average while CPUs stay idle. Check the blocked kernel path before deciding whether signals or mount recovery can release it.

Checkpoint too rarely and a failure costs hours of GPU time. Checkpoint too often and you pay for stalled GPUs writing terabytes to storage. There is an arithmetic answer and most teams never do it.

Expensive accelerators waiting on cheap storage is the most common form of waste in a training cluster, and it looks exactly like a code problem on every dashboard.