04One node in your training fleet makes every job it touches 30 percent slower, but it passes health checks. How do you find and handle it?▼hardNewNVIDIAMetaMicrosoft2 repliesunlockedGPUs fail gradually before they fail loudly. Xid errors, ECC retirement, thermal throttling and a degraded NVLink all produce a node that is up, schedulable, and slowing every gang it joins.Open full answer →
27Your GPU dashboard shows 95 percent utilisation and finance wants to know why you need more GPUs. What do you tell them?▼hardNewNVIDIADatabricksGoogle2 replies◆ premiumThe utilisation number everyone quotes means at least one kernel was resident, not that the hardware was doing useful work. A GPU at 95 percent can be doing five percent of what it is capable of.Open full answer →
