DevOpsInterviewPrep logo
CI/CD, Release Engineering & GitOps / 04
hard★ EssentialNewStripeShopifyGitLab

Ship a schema change to a live service with no downtime. Walk me through the sequence.

The deployment question that has a real answer and a real trap. Code and schema deploy at different speeds, and any step where they must change together is the step that causes the outage.

Updated Sep 2026 · Grounded in researched DevOps, SRE and platform engineering interview loops, written to a senior-engineer editorial bar, and never padded to hit a word count.

TL;DR: Expand and contract. Never change a column in place while code depends on it. Add the new shape, deploy code that writes both and reads the old, backfill, switch reads, then remove the old shape in a later release. The early steps preserve code rollback. Contracting the schema removes that option and needs a deliberate recovery boundary.

How to approach it

State the constraint that generates the answer: during any rollout, old and new code run simultaneously, so the schema must be compatible with both. Then give the steps as separate deploys, and be explicit that they are separate.

A strong answer

The constraint is that a rolling deploy means both versions are live at once, sometimes for minutes and sometimes for hours if a rollout stalls. So a migration that renames a column breaks the old code the instant it runs, and rolling back the code does not roll back the schema. That asymmetry is why in-place changes cause outages.

Expand and contract, using a column rename as the example:

rendering diagram…

Step 1, expand. Add the new column, nullable, with no constraint. Check the engine’s DDL locks even for this additive change. PostgreSQL normally needs an ACCESS EXCLUSIVE lock for ADD COLUMN; set a short lock timeout and retry during a quiet window rather than queuing behind a long transaction.

Step 2, dual write. Deploy code that writes both columns and still reads the old one. Wait until all writers, including old workers and scheduled jobs, write both representations. Otherwise an old process can keep creating missing or stale values during the backfill. Reversible by redeploying the previous version.

Step 3, backfill. Populate the new column for existing rows, in batches, with a pause between them. This is where production incidents happen: a single UPDATE across a large table takes a lock, blocks writes, and fills the replication stream, so replicas fall behind and read traffic sees stale data. Batch by primary key, keep transactions short, and watch replication lag. Use conditional updates or version checks so a backfill cannot overwrite a newer live write. Reconcile values and missed rows after every writer has adopted the new format; row counts alone do not prove consistency.

Step 4, switch reads. Deploy code that reads the new column and still writes both. If validation fails, return reads to the old column while both representations are still maintained. Check whether bad data has already escaped to downstream consumers.

Step 5, stop writing the old. A deploy that writes only the new column.

Step 6, contract. Drop the old column, in a separate release, after enough time has passed that you are certain no rollback will need it. Days, not minutes.

Six deploys where a naive migration is one. That is the cost, and it is the correct trade for a system you cannot take down.

Two mechanics worth naming. Adding a column with a non-null default rewrites the whole table on some engines and versions and is instant on others; know which you are on, because that difference is a locked table for the duration. And long-running migrations must be interruptible, so a change that cannot be stopped halfway is one you cannot abort during an incident.

The wider framing that scores: separate schema changes from code releases entirely. Migrations run as their own pipeline step with their own approval and rollback plan, never as a side effect of application startup, because an app that migrates on boot will run the migration once per replica and race itself.

What interviewers probe next

"How do you roll back step 3?" You usually do not. The backfill wrote data, and that is fine because nothing reads it yet. Reversibility comes from the read switch, not the write.

"What about a foreign key or a unique constraint?" In PostgreSQL 17, a foreign key can be added with NOT VALID and checked later with VALIDATE CONSTRAINT. A unique constraint cannot use NOT VALID: on an ordinary table, first create a suitable unique index with CREATE UNIQUE INDEX CONCURRENTLY, then attach it with ALTER TABLE ... ADD CONSTRAINT ... UNIQUE USING INDEX. Resolve duplicates before building it and budget for the attachment lock; concurrent index creation does not mean every step is lock-free. Check partitioned-table restrictions and the exact engine version. PostgreSQL documents these separate paths.

"How would you canary a migration?" A shared schema affects all clients of that database. Canary its readers after a compatible expansion; independently isolated tenant databases can support a staged schema rollout. Which is the argument for the read switch being its own deploy.

Common mistakes

Doing it as one migration and one deploy. It works in staging with no traffic and fails in production.

Backfilling in a single statement, which is the most common way to cause the outage you were preventing.

Dropping the old column in the same release that stops writing it, which removes your rollback path exactly when you might need it.

That one was free, and so are 10 answers per topic without an account. Signing in doubles that to 20, keeps your bookmarks, and tracks which topics you keep getting wrong.one Google click · no card · nothing to cancel
HOW DID IT GO?
0
UP NEXT ON YOUR JOURNEY
DISCUSSION · 0

Nothing here yet. Say how you would answer it.