tempest-zigzag

Add new information to xml generated by tempest


Keywords
zigzag
License
Apache-2.0
Install
pip install tempest-zigzag==0.1.0

Documentation

tempest-zigzag

Process xml output generated by tempest.

Quick Start Guide

  1. Install tempest-zigzag from PyPI with pip:

    $ pip install tempest-zigzag
    
  2. For more information on using tempest-zigzag launch help by:

    $ tempest-zigzag --help
    
  3. Here is an example of processing a tempest generated xml file:

    $ tempest-zigzag tempest_junit.xml test_list.txt

The reason for this tool

Tempest has the ability to generate junit xml thanks to the tools 'stestr' and 'subunit'. Unfortunately there are circumstances where the XML produced by Tempest does not conform to the contract for tests set out by Tempest itself. Specifically when a setUpClass or Teardown class fails the reports generated in xml format are inaccurate. When these failures occur data is omitted about the true state of the test case. If try to inspect a test that failed its setUpClass you will not find a record for it in the XML report, you will only find a record for the setUpClass which does not speak of any downstream tests that were intended to be executed.

This tool will help tempest be more honest about its errors in setup and teardown. The features described below adjust the behavior similar to that found in `py.test`_. Please see tempest_xml_failure_examples.rst for setUpClass and tearDownClass examples.

When a setUpClass fails this tool will look in the supplied test-list and find tests downstream from the setup, it will create new testcase records in the xml and mark them with an error tag. The original text from setUpClass failure is preserved and it is noted where the error occurred.

When a tearDownClass fails this tool will look at the supplied test-list and find all the upstream test cases mentioned in the supplied xml document. Any test upstream from the teardown failure will be marked with an error tag, preserving text from the originating failure.