django-MacFly – Forward To The Past

Output sql migrations for ahead of time schema changes.

TL;DR

pip install django-MacFly

INSTALLED_APPS += ['django_macfly']

On a develop environment, deploy the old (version N) code and run migrations. Then, deploy the new code (version N+1) without running the migrations.

Then run:

python manage.py sqlmigratefwd

Read the result, think, hack, do.

Rationale

Here, at Peopledoc we found that:

• ZDD is great. We need to achieve at least "Almost Zero Downtime Deployments"
• ZDD is (quite) easy except when there are SQL Database schema migrations.
• Django migration system is useless in ZDD context, especially for large databases.

Our strategy for schema migration is to run migrations ahead of time, the database is migrated to schema N+1 while the code is at N.

This strategy requires coding discipline (do not rename a DB Column, do not change a column type...) and we needed a tool that checks the code changes, warn us if the changes are not ZDD-friendly, and gather all (safe) SQL commands to make the migration.

WARNING

This is NOT AT ALL an automatic migration tool. It's just a (dumb) HELPER. It outputs hints that you MUST read and understand. Your ahead-of-time migration will probably need DB triggers, query re-writing, default values and all sort of DBA voodoo stuff to work properly. MacFly DO NOT handle this. Candide use may eat your data and voids even the no-warranty-of-any-kind clause.

What it does

Basically it walks through pending migration (using django migrate internals).

For each migration:

• Check migration safety
  • if it's unsafe, MacFly writes commented sql and issues a warning It's your responsabilty to interpret the issue reading SQL and the migration script
  • it it's safe, MacFly prints a modified version of sqlmigrate:
    • sqlmigrate remove default values for NOT NULL columns. Here we need a SQL default because the code doesn't know the column, and inserts would fail. Hence, sqlmigratefwd doesn't drop defaults
    • sqlmigratefwd also issues a faked migration (an insert in django_migration table). So it's safe at deploy time to run migrate