TL;DR: Model the fleet as distinct pools with distinct resource names rather than as one pool of interchangeable GPUs, so a workload's requirements are expressed in the request instead of implied by hope. Then decide per workload class: training pins to one generation because collectives run at the speed of the slowest member, while inference tolerates mixing if you route by capability and account for throughput differences in capacity planning.
How to approach it
Say up front that heterogeneity is permanent, because the answer changes if you treat it as debt to be paid off. Then split by workload class, since training and inference have opposite tolerances for it.
A strong answer
Why one pool fails. The device plugin advertises nvidia.com/gpu from every node regardless of what the card is. A job needing 80GB of device memory can be scheduled onto a 48GB card and fails at load with an out-of-memory error rather than staying Pending, which is a worse outcome because it consumes a scheduling cycle and looks like an application bug. The traditional workaround is labels from GPU Feature Discovery plus node affinity in every manifest, which works and pushes scheduling logic into workload specs where it drifts.
Separate pools, separate names. Give each generation its own node pool with its own taint, and where the platform supports it, advertise distinct extended resource names so a request states what it needs. Kueue resource flavors express this cleanly: one ClusterQueue can list flavors in preference order with quota per flavor. Workload node selectors constrain eligibility; flavor ordering and fungibility policies are configured on the ClusterQueue, not an arbitrary per-job preference-list API. See Kueue ClusterQueue flavor selection.
Training pins, and the reason is arithmetic. A data-parallel job synchronises every step. If one rank runs on a card 40 percent slower, every other rank waits for it at every all-reduce, so the whole job runs at the slow card's pace and you have wasted the difference on every fast GPU in the group. There is no configuration that fixes this. Mixed-generation training is only reasonable when the parallelism strategy deliberately assigns different work per rank, which is rare and hard.
Inference mixes well if you plan capacity properly. Replicas are independent, so a slower card simply serves fewer requests. Two things follow. First, route by capability: a model that needs FP8 hardware support cannot go to a generation without it, and a model that needs 70GB cannot go to a 48GB card. Second, capacity is not replica count. Size each pool from its measured tokens per second, and make the autoscaler aware that adding a replica in pool A is not the same amount of capacity as adding one in pool B. A fleet-wide replica target computed from a single throughput assumption will under-provision on the day traffic lands on the older pool.
Cost accounting has to follow the hardware. Charging one blended GPU-hour rate across generations means a team on old cards subsidises a team on new ones, and neither can see it. Rate per pool, reported per team.
What interviewers probe next
"How does DRA change this?" It replaces the countable resource with a claim describing attributes, so memory size, model and topology become expressible in the request rather than encoded in labels. That is precisely this problem, which is why the feature exists.
"What do you do with the oldest generation?" Give it the work that tolerates it: batch inference, evaluation runs, development, CI. Retiring it early wastes capacity you already own, and letting it into the training pool wastes capacity you just bought.
"How do you stop teams all requesting the newest?" Price it, or queue it. If the newest pool is free and unlimited, every job requests it and the older pools sit idle while the queue grows.
Common mistakes
One pool, one resource name, and node affinity copied between manifests until nobody remembers which constraints still matter for which job.
Mixed-generation distributed training, which silently runs at the slowest card's speed and gets diagnosed as a code regression.
Autoscaling on replica count across a mixed fleet, which produces a capacity number that is wrong by whatever the generation mix happens to be that hour.