reliability

Large object uploads are a reliability design problem, not a bandwidth one. Parts, retries and the completion call decide whether a flaky link costs one chunk or the whole night.

A dead-letter queue with no owner is a landfill with a nicer name. Triage discipline separates teams that recover messages from teams that archive regret.

The arithmetic takes ten seconds and most candidates stall on it. The stall is the signal: it means error budgets are something you have read about rather than used.

Their green page, your red SLO. Separate their true failure rate from the amplification your own client adds, buffer at the boundary, then escalate with a client-side scorecard instead of feelings.

An SLA is a price list for failure signed before the failures. Bring the measured distribution, negotiate methodology and exclusions harder than the headline number, and phase targets with dates.

You have added a dependency with a latency distribution an order of magnitude wider than anything else in your stack, whose failure mode is a plausible wrong answer rather than an error.

Outbound delivery looks like a queue and a loop until one subscriber hangs for sixty seconds and takes your workers with it. Isolation, retry horizons, and signatures are where the marks are.

Every overloaded system dies the same way: queues turn load into latency, latency into timeouts, timeouts into retry storms. Tiered shedding, deadline propagation, and retry budgets are the design.

Asking nicely in the prompt gets you most of the way and fails on the tail, which is where the automation breaks. Constrained decoding can guarantee the shape of completed output for supported schemas. Interrupted generations still need a failure path, and compilation has a latency cost.

Checkpoint too rarely and a failure costs hours of GPU time. Checkpoint too often and you pay for stalled GPUs writing terabytes to storage. There is an arithmetic answer and most teams never do it.

Failure stops being an event and becomes a rate. Once mean time between failures drops below the length of a run, the design question is not how to avoid interruption but how cheaply you can absorb one.

A launch-versus-guardrail negotiation. Interviewers score whether you price the risk in the stakeholder's currency, offer a path that still serves the business goal, and leave the final call with a named owner.