memory
DevOps interview questions tagged memory, across every topic.
7 questions · 1 unlocked for you
Concepts behind "memory"
The concept pages behind what these questions are testing.
Foundational
The KV cache is what decides how many users you can serveModel weights, runtime memory and the KV cache compete for GPU memory. Estimate the cache budget and memory per sequence, then check whether that concurrency also meets your latency target.🤖 AI Infrastructure
Core
Why GPUs break the assumptions schedulers are built onGPU scheduling must account for device memory, sharing boundaries and interconnect topology. Device plugins, MIG, DRA and gang scheduling address different constraints; none makes accelerators behave like interchangeable CPU millicores.🤖 AI InfrastructureSign in