Every schema change looks small when you write it down. Add a column, split a field, tighten a constraint — a line or two of SQL, reviewed in a minute. What that line hides is the work that follows it: the rows that already exist and don’t have the new field, the ones whose old value doesn’t cleanly convert, the ones written by a version of the code that predates the convention everyone now assumes. The definition changes instantly. The data doesn’t, and until it does you have a table where the schema says one thing and a large fraction of the contents say another.

The backfill is where the real engineering lives, and it has properties the migration script usually doesn’t account for. It takes long enough that it can’t be a single transaction, which means it has to be resumable — interrupted halfway, it should pick up where it stopped rather than starting over or, worse, double-applying. It runs against a live system, so it has to be paced: a tight loop rewriting rows as fast as the database allows will happily saturate the very resource production traffic needs. And it runs concurrently with the application, which is still writing new rows in the meantime, so “backfill everything up to the current maximum id” quietly leaves a moving tail behind it.

The correctness question is the one that bites hardest, because a backfill is a bulk transformation applied to data nobody has looked at closely in years. The rows written last month are uniform and convert cleanly. The rows from three years ago were written under different assumptions, possibly by code that no longer exists, and include the values nobody anticipated: empty strings where null was expected, a timezone convention that changed midway, an enum value that was removed but never cleaned up. Running the transformation over a sample and inspecting the results — before running it over everything — is the cheap step that catches this. Discovering it afterward means a second migration to repair the first.

All of this is why the safe pattern is staged rather than atomic: add the new structure without removing the old, write both while backfilling the gap, switch reads only once the new field is complete and verified, and drop the old one much later, once you’re confident nothing still reads it. Each step is independently reversible, and the risky moment — the switch — happens when the data is already in place rather than as part of a change that has to succeed all at once. It’s the same reasoning that applies to deploys, for the same reason: the window where both shapes exist is unavoidable, so the job is to make that window safe rather than to pretend it away.

The thing to internalize is that the size of a schema change has almost nothing to do with the size of its diff. A one-line ALTER against an empty table is trivial; the same line against a table with years of accumulated history is a project with a plan, a verification step, and a rollback story. Estimating it by looking at the SQL is how migrations end up half-finished in production, with a column that’s populated for most rows and quietly null for the ones that predate everyone’s memory.