13How is rolling out a new model different from rolling out new code, and what does a model canary actually measure?▼expertNewMetaGoogleDatabricks○ sign inCode rollouts ask 'does it work'; model rollouts ask 'is it better', and the second question has no HTTP status code. Shadow traffic, offline eval gates and quality canaries replace the 500-rate dashboards you cannot use.Open full answer →
17Someone proposes serving the quantised model to halve your GPU bill. What has to be true before you ship it?▼hardNewNVIDIADatabricksMicrosoft2 replies○ sign inThe infrastructure win is real and easy to measure. The quality regression is real, harder to measure, and will not show up in your latency dashboards at all.Open full answer →
29How do you detect that a model in production got worse, when nothing in your infrastructure metrics changed?▼expertNewDatabricksDatadogMicrosoft2 replies◆ premiumLatency is flat, errors are zero, and the answers are worse. Quality regression is invisible to every metric a platform team normally owns, which is why it needs its own pipeline.Open full answer →
