A generic integration test tool utilizing docker-compose (for now)


Keywords
integrationtest, sysadmin, devops, ci, cd
License
MIT
Install
pip install intmaniac==0.18.1

Documentation

INTManiac

This python tool enables you to run a test program against a set of containers to smoke- / integration- / acceptance-test your program. docker-compose is used to create your program environment stack, while docker is used to run a container with a test program against that stack.

You can run a matrix test, which means you can check several combinations of containers with each other.

This tool is available as docker container as well. There is a constantly updated docker container available under:

  • quay.io/flypenguin/intmaniac

How this works

For most products you want to test them embedded in a system of connected components. For example if you use a database, it might be good to know with which versions of the database your product works, or that your latest build has still the same API functionality than your stable branch.

intmaniac enables you to write a parameterized docker-compose template which describes your full test environment, and then run tests against it. It uses a docker-compose template to create your application stack, and then uses docker to run a container with a test program. The container is linked against your application stack using the links key in the tester_config(s), so everything happens completely separate from your actual host network.

All tests are executed in sequence.

TODO

A lot basically ...

  • Unit tests are not too good (ha-ha...)
  • If a command blocks forever intmaniac will also block forever
  • More information about service outputs would also be nice (currently it's ONLY the tester which is printed, this will change soon I guess)
  • The text output sucks. Seriously. I use it with TeamCity only right now, so I did not put much effort into this.
  • The code quality is ... so-so, I guess. Much better than before, though. :)
  • And a lot more probably.

Super-quick-start

$ mkvirtualenv intmaniac
$ pip install intmaniac

# now run with the intmaniac.yaml file in the current directory
$ intmaniac

# or maybe even with an environment variable set
$ intmaniac -e TMP_IMAGE=my/tmp:image

Configuration reference

intmaniac.yaml

Use version 2 for a single test run (one testing container in one docker-compose setup).

---
version: 2          # can be string or int, for ONE single test combination


# required, if path is relative it will be interpreted as relative
# to this file
compose_template: "here/be_dragons.yml"


# required, one must be present.
tester_config:

  # required
  image: my/image:latest

  # required, format: [linked_service_from_compose_file, ...]
  # a service "myservice" will be linked into the test container under
  # that name (so that the host name "myservice" will resolve)
  links: []

  # optional, can be string, list of strings, list of lists of strings
  commands: "..."       # executed in the tester container
  pre: "..."            # executed on the local machine
  post: "..."           # same. (example: "sleep 10")

  # optional, format { key: value }
  environment: {}

  # optional. true / false. default is false.
  allow_failure: false  # allow test to fail

  # optional, boolean, default is true.
  pull: true            # pull containers before running

Use version 3 for a matrix test (multiple environments, set-ups, tests, etc.). The (non-working) example below will create two tests. Basically every possible combination is created for each test and then run.

---
version: 3          # can be string or int, allows for matrix test

# required, must have one entry, format: { name: template_path }
compose_templates:
  my_template: "..."        # as above

# required, must have one entry. format: { name: tester_config }
# tester_config is the same structure as defined above.
# if there is an environment definition it is merged UNDER the environment
# defined in the "tests" section (lower precedence)
tester_configs:
  my_tester: {}             # as above
  my_other_tester: {}       # as above

# optional.
environments:
  my_environment: {}        # as above

# required, must have one entry. see below for example
tests:
  default:

    # required. either "string" or [string, ...]
    # the key can also be named "tester_configs". DO NOT define both.
    tester_config: [my_tester, my_other_tester]

    # required. either "string" or [string, ...]
    # the key can also be named "compose_templates". DO NOT define both.
    compose_template: my_template

    # optional. either "string" or [string, ...]
    # the key can also be named "environments". DO NOT define both.
    environment: my_environment

Full example, with docker-compose file

---
# docker-compose-pg.yml

test-me:
  image: my_company_hub/%%TEST_CONTAINER%%
  environment:
    - DB=postgresql://db:5432
  links:
    - db:db

db:
  image: postgres:%%PG_VERSION%%

---
# docker-compose-mongo.yml

test-me:
  image: my_company_hub/%%TEST_CONTAINER%%
  environment:
    - DB=mongodb://db:27019
  links:
    - db:db

db:
  image: mongo:%%MONGO_VERSION%%

---
# intmaniac.yaml

version: 3

compose_templates:
  default: docker-compose-pg.yml
  next: docker-compose-mongo.yml

environments:
  postgres93: { PG_VERSION: "9.3" }
  postgres94: { PG_VERSION: "9.4" }
  postgres95: { PG_VERSION: "9.5" }
  mongodb24:  { MONGO_VERSION: "2.4" }
  mongodb32:  { MONGO_VERSION: "3.2" }


tester_configs:
  standard_tester:
    image: myhub.com/myproduct/acceptance:production
    links: ["test-me:testservice"]
    environment:
      TEST_URL: http://testservice:80
  prerelease:
    image: myhub.com/myproduct/acceptance:beta
    links: ["test-me:testservice"]
    allow_failure: true
    environment:
      TEST_URL: http://testservice:80

tests:
  databases:
    compose_template: default
    tester_config: [standard_tester, prerelease]
    environments: [ postgres93, postgres94, postgres95 ]
  prerelease-pg:
    compose_template: default
    tester_config: prerelease
    environment: [postgres94,postgres95]
  prerelease-mongo:
    compose_template: next
    tester_config: prerelease
    environment: [mongo24,mongo32]

In this case it is assumed that TEST_CONTAINER comes from a commandline -e TEST_CONTAINER=... setting.

Now you should have an idea about how this works.