capacity

Turns the standard autoscaling definition into the problem it actually causes. Almost every candidate can define an HPA; far fewer can say why theirs did not save them.

Priority is one number with three separate effects: scheduling order, preemption victim selection, and eviction order under node pressure. Conflating those is the trap; designing tiers that survive both a full cluster and a hot node is the pass.

The classic napkin-math question. The number in the prompt is the question, and candidates who draw the architecture before doing the arithmetic design something the physics forbids.

A mixed fleet is the normal state, not a transitional one, because capacity arrives when it arrives. The failure is a scheduler that treats two very different cards as the same countable unit.

It reads as a configuration change and it is a capacity change. Cache memory per request grows with the limit, prefill cost grows with what people actually send, and concurrency falls in proportion.

Scale to zero is correct for some workloads and an outage generator for others, and the deciding number is what the first request after idle experiences. Cold start turns an availability promise into a queue.

GPUs are the default for reasons that hold for large generative models and not for everything else. A small classifier on CPU can be cheaper, more available and easier to operate, and knowing where the crossover sits is worth more than knowing the newest card.