22The column shipped with expand-and-contract and a billion rows still hold nulls. How do you run the backfill?▼hardNewRazorpayPhonePeDatabricks◆ premiumA billion-row backfill is a deployable, throttled, resumable job with checkpoints and verification gates. Ad-hoc scripts against prod are how it becomes an incident.Open full answer →
38One late task stalls forty downstream jobs every morning. Design a scheduler that owns dependencies, SLAs, and backfills.▼mediumNewDatabricksLinkedInGoldman Sachs◆ premiumCron fires at wall-clock times; a scheduler reasons about data intervals and blocked frontiers. The scored details are per-partition idempotency and catch-up semantics.Open full answer →
