Liquipy

A Python wrapper for Liquibase

Description

Liquipy is a small python wrapper around Liquibase that exposes a much simpler changelog format in YAML instead of XML. This module currently only works for MySQL migrations.

Installation

python setup.py install

YAML Changelogs

In Liquipy, a changelog is a list of database migrations in SQL that contain the DDL to update the schema. It should also contain the SQL to rollback to the previous state, which allows users to navigate to any spot within the changelog. Each changelog contains one or more numbered changesets, identified by an integer as its key.

Here's an example of a changelog with one changeset.

1:
  author: Richard Smoker
  comment: Creating happy table.
  tag: version-0.1
  sql: |
    CREATE TABLE `happy` (
      `happyGuid` char(36) NOT NULL DEFAULT '',
      PRIMARY KEY (`happyGuid`)
    ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
  rollback: |
    DROP TABLE happy;

This changelog contains only one changeset. The id of the changeset is 1. The author, comment, sql, and rollback properties of the changeset are required. The tag property is optional, but when present will create a navigable point within the entire changelog that is easy to rollback to. This tag usually corresponds directly with the version of the software product the database schema is associated with. The value of the tag can be any string. To use a value such as 0.1, you should enter this in quotes so it is not interpreted as a number (as seen in the additional sample below).

Here is another changelog with multiple changesets, comments, and more whitespace:

# Sample migration 2
2:
  author: Rocky Racoon
  comment: Creating sad table.

  sql: |

    CREATE TABLE `sad` (
      `sadGuid` char(36) NOT NULL DEFAULT '',
      PRIMARY KEY (`sadGuid`)
    ) ENGINE=InnoDB DEFAULT CHARSET=utf8;

  rollback: |

    DROP TABLE sad;

# Sample migration 3
3:
  author: Richard Smoker
  comment: Adding grumpy column
  tag: "0.2"

  sql: |

    ALTER TABLE happy ADD `grumpy` BOOL;

  rollback: |

    ALTER TABLE happy DROP grumpy;

Changelog Includes

Any changelog file may specify one inclusion at any point, which should point to a directory relative to that file that includes further changelog files. Any YAML files (with the .yml extension) within this directory will be read and processed as if they were a part of the parent changelog.

For example, imagine a directory stucture like this:

migrations
├── master_changelog.yml
└── changelogs
    ├── may.yml
    ├── june.yml
    └── july.yml

The contents of master_changelog.yml could simple contain an inclusion of the changelogs directory, and all the changesets within each file in that directory will be included in the master changelog:

master_changelog.yml

include:
  directory: changelogs

This could allow users to break apart their changesets into temporal or other logical units. Changelogs included in this fashion can also themselves include other directories.

Changeset ID Order Matters

Each changeset has an id, which is the only ordering that matters. All changesets within the changelog, no matter what order they appear sequentially, or what order they are included, will be executed in the sorted order of their integer ids.

Usage

import liquipy

db = liquipy.Database(
  host="localhost",
  port="3306",
  database="my_database",
  username="your_username",
  password="your_password"
  tempDir=".")

# Reads and validates the changelog file, as well as processing any included
# changesets.
db.initialize('my_changelog.yml')

# Performs the actual database migration by applying any unexecuted changesets
# within the changelog with Liquibase
db.update()

Temporary Directory

Liquipy needs a temporary directory to write XML files for Liquibase. If a tempDir is provided in the liquipy.Database() constructor, this directory will be used for this purpose. If not, liquipy will use /tmp. Specifying a tempDir is useful for easily inspecting the Liquibase XML generated by liquipy, especially for debugging. A file called liquipy_changelog.xml is created here, which contains all the changesets specified in the input YAML changeset. This file is overwritten each time an liquipy initializes a new changelog.

Rollbacks

You can rollback to a specific tag easily. The tag name specified must have been included as the tag property of a changeset within the changelog.

db.rollback('tagName')

If you specify an nonexistent tag name, you'll get a Liquibase error like this:

Error running liquibase! See output below:

INFO 9/18/13 10:23 PM:liquibase: Successfully acquired change log lock
INFO 9/18/13 10:24 PM:liquibase: Reading from test_liquipy.DATABASECHANGELOG
INFO 9/18/13 10:24 PM:liquibase: Successfully released change log lock
Liquibase rollback Failed: Could not find tag '0.2s' in the database
SEVERE 9/18/13 10:24 PM:liquibase: Could not find tag '0.2s' in the database
liquibase.exception.RollbackFailedException: Could not find tag '0.2s' in the database
  at liquibase.changelog.filter.AfterTagChangeSetFilter.<init>(AfterTagChangeSetFilter.java:28)
  at liquibase.Liquibase.rollback(Liquibase.java:325)
  at liquibase.integration.commandline.Main.doMigration(Main.java:875)
  at liquibase.integration.commandline.Main.main(Main.java:138)


For more information, use the --logLevel flag

The Liquibase XML changelog file used to perform this operation is here: ./liquipy_changelog.xml

At the end of the error message, the location of the actual Liquibase XML file is stated for debugging.

Listing Tags

You can also get a list of existing tags (after initialization).

db.getTags()

This returns a list of dicts with each tag's name and the changeset it is associated with.

[{'changeSet': 1, 'tag': '0.1'}, {'changeSet': 3, 'tag': '0.2'}]

Sample

For a sample of usage, see the sample directory. In contains examples of most of the concepts described above.

License

This software carries the Apache License V2.0 to adhere to the same licensing standard as the bundled Liquibase software.