spot

The difference between scaling node groups and provisioning nodes is the whole answer. One picks from shapes you defined in advance; the other reads the pending pods and buys what they need.

Four purchase instruments, one fleet, and a finance team asking why you did not commit to everything at maximum discount. The scoring rubric rewards the arithmetic of being wrong, not the discount table.

Everyone quotes the discount. Fewer candidates can describe what happens when a notice arrives late or a worker disappears without completing shutdown and why some workloads shrug it off while others lose hours of compute.

GPU capacity is not elastic the way CPU is. On-demand means whatever is left after the reservations, and in a tight region that is nothing, so the availability problem and the cost problem are the same problem.

A training job is an hours-long computation with no user watching. Whether an interruption costs four minutes or four days is decided entirely by checkpoint cadence and restart semantics designed before the run.

Nobody is waiting, which changes every constraint. Latency stops mattering, throughput per dollar becomes the only metric, and the design converges on something that looks like a data pipeline rather than a service.