Migopy 1.0 - Mongo Migrations for Python

Migopy is a simple python library which simply allows You to setup mongo migrations manager for Your fabfile (see fabfile.org). After that You will be able to run command like this:

fab migrations

which returns status of Your migrations, which are registered or not:

fab migrations:execute

which execute not registered migrations.

Each migration is treated as a seperate python file localized in mongomigrations directory.

Information which migrations are registered or not is stored in 'migrations' collection in mongo database.

Quick start

At first install migopy:

pip install migopy

then quickly configure mongo migrations in Your fabfile with basic informations

import migopy

class Migrations(migopy.MigrationsManager):
    MONGO_HOST = # host
    MONGO_PORT = # port
    MONGO_USER = # username
    MONGO_USER_PASSWORD = # password

migrations = Migrations.create_task()
# or for new style tasks:
# from fabric.api import task
# migrations = task(Migrations.create_task())

and You are done. Migrations files should be putted in 'mongomigrations' directory which is at the same level as fabfile.py and names of migrations files should fulfill default pattern:

(?P<migr_nr>[0-9]+)_[a-z0-9_]+\.py

Basic commands:

• fab migrations - show unregistered migrations
• fab migrations:execute - execute unregistered migrations
• fab migrations:ignore - register unregistered migrations without executing them
• fab migrations:help - show help about the migopy commands

Additional commands:

• fab migrations:execute,ex_1_ex.py - execute specyfic migration
• fab migrations:rollback,ex_1_ex.py - rollback specyfic migration (do down() function)
• fab migrations:ignore,ex_2_ex.py - ignore specyfic migration

Structure of migration file:

def up(db):
    db.notes.insert({'content': 'test'})
    pass

def down(db):
    db.notes.remove({'content': 'test'})

where up() function is executed during fab migrations:execute command, and down() during fab migrations:rollback. Under db variable pymongo database object is given (pymongo.database.Database).

Handling different environments

It's common for Fabric to execute tasks on different environments, for example You probably would like to do that:

fab staging migrations
fab production migrations

Example of fabfile.py:

import migopy
import settings # your settings

is_remote = False

def staging():
    is_remote = True

def production():
    is_remote = True

# Bind your settings with those in Migopy
class Migrations(migopy.MigrationsManager):
    MONGO_HOST = settings.MONGO_HOST

    @classmethod
    def task_hook(cls, subtask, option):
        if is_remote:
            run(cls.fab_command(subtask, option))
            raise migopy.StopTaskExecution()


migrations = Migrations.create_task()

In the case above when we want to run migrations on remote machines, under the hood we have to run for example fab staging migrations command by fabric run() method. Migopy is not handling remote mongo connections from local fabric script so we need to raise fab migrations itself on remote machines.

To do this we have to implement task_hook() class method. In the example task_hook simply recognize if we choose remote environment and if we does it runs itself by created string command, on remote machine and stop further execution (to stop raising migopy tasks on local).

More on migration files

Migration files are quite flexible, if special mongo connection is needed or better integration with Mongokit You can import mongokit models or pymongo in migration file directly.

Under the hood Migopy import each migration file as module and executes up/down functions giving pymongo database object as an argument.

import mymongokitmodel

def up(db):
    note = mymongokitmodel.Notes()
    note['name'] = 'test'
    note.save()

in the case above, mongokitmodel handle mongo connection by it's own.

Further customization

Additional configuration

class Migrations(migopy.MigrationsManager):
    MIGRATIONS_DIRECTORY = # directory where migrations files will be stored
    MIGRATIONS_FILE_PATTERN = # regex pattern of the migrations files
    DO_MONGO_DUMP = True # will do mongo dump before migrations execution
    MONGO_DUMP_DIRECTORY = # directory where database dump will be stored

For more, check migopy.MigrationsManager class attributes. You can override selected methods

class Migrations(migopy.MigrationsManager):
    @migopy.task
    def execute(self, spec_migr=None):
        super(Migrations, self).execute(spec_migr)
        ...

    @migopy.task
    def dbdump(self):
        ...

You can add, additional migrations subtasks

import migopy

class Migrations(migopy.MigrationsManager):
    @migopy.task
    def mytask(self, option=None):
        """Here should be a help doc which will be showed under
        fab migrations:help command"""
        pass

fab migrations:mytask
fab migrations:mytask,some_option

Setup for development

$ git clone https://github.com/clearcode/migopy.git
$ cd migopy
$ virtualenv venv
$ source venv/bin/activate
$ pip install -e .

Unit tests:

$ python -m unittest tests.test_units

Integration tests:

$ python -m unittest tests.test_integrations

All:

$ python -m unittest discover

Changes log

1.0 (2014-01-14)

• bug fixes for mongo user authorization
• bug fixes for working with remotes (logging, commands executing)

1.0 beta (2013-12-13)

• Initial version