UnderTest

What is UnderTest?

Goals

UnderTest is an open source testing toolkit for building world-class behavior driven (BDD) based enabled projects and culture.

UnderTest is focused on improving the life-cycle of test cases from requirements to test suite to reporting through continued iteration.

One of the key things we strive to enable is for team members building testing suites to reuse web developing skills. Developing with UnderTest hopefully feels like writing web API applications.

Examples

Below are a list of examples to learn from.

1. UnderTest.Examples.GettingStartedVerifyWebApp

Getting Started

Prerequisites

• .net core 2.2+

Project Setup

1. Install UnderTest.Templates
2. run dotnet new UnderTest --name "MyProject" - see Usage for other options
3. Open the project and have fun.

Test Suite Configuration

After we have a working project, we need to setup UnderTest. Add the following to the Main method in your Program.cs.

  return new UnderTestRunner()
    .WithCommandLineArgs(args)
    .WithProjectDetails(x => x
      .SetProjectName("YOUR PROJECT NAME Test Suite"))
    .WithTestSettings(x => x
      .AddAssembly(typeof(Program).Assembly))
    .Execute()
      .ToExitCode();

There is more capability available to configure UnderTest, but this is enough to get us started.

Fundamentals

UnderTest is made up of a few key principles.

• Feature files
• Handlers
• TestFilters

Feature Files

Gherkin is the lifeblood of any BDD project. UnderTest lives its best life by using gherkin to drive all portions of the pipeline.

Features are stored in the Features folder of the project.

An example feature, named say Example.feature (this name will matter more a little later) might look like:

Feature: Simple feature

Scenario: Simple scenario
  Given something is true
  When something happens
  Then this should now be true

Feature Handlers

Feature handlers are the plumbing that connect our features to the associated functionality.

If you are comfortable with web programming, and specifically MVC, Feature handlers are the controllers of the UnderTest suite. Requests are submitted to handlers and they handle them.

A handler for the above Simple Feature might look like:

using UnderTest.Attributes;

namespace YourCompany.YourProjectTestSuite.FeatureHandlers
{
  [HandlesFeature(@"Example.feature")]
  public class SimpleFeatureHandler : FeatureHandler
  {
    [Given("something is true")]
    public void SomethingExists()
    {
      // todo - we will learn more about what goes here later
    }

    [When("something happens")]
    public void SomethingHappens()
    {
      // todo - we will learn more about what goes here later
    }

    [Then("this should now be true")]
    public void ThisExists()
    {
      // todo - we will learn more about what goes here later
    }
  }
}

Test Filters

When we start building real test suites, we generally will want to start running slices or categories of our available tests. This is where Test Filters come in.

These filters are configured by default through code on the TestSettings object. You can layer and mix and match filters. A filter to run any feature tagged with @smoke would look like:

  .OnlyRunTheseTags("@smoke")

Our updated Program.cs with this new change might look like:

    return new UnderTestRunner()
      .WithCommandLineArgs(args)
      .WithProjectDetails(x => x
        .SetProjectName("YOUR PROJECT NAME Test Suite"))
      .WithTestSettings(settings => settings
        .AddAssembly(typeof(Program).Assembly)
        .OnlyRunTheseTags("@smoke"))
      .Execute()
        .ToExitCode();

and to ignore the same tests

  .DoNotRunTheseTags("@smoke")

We can also add custom filters using the AddTestFilter method:

  .AddTestFilter(x => x.Tags.All(t => t.Name != "@smoke"))

returning true from the passed lambda will cause the test to be ignored.

[DebugRunThis] and [DebugIgnoreThis]

Many times when building test suites, we will want to run one specific test.
With test filters you could add a tag, then write a filter to handle that - maybe something along the lines of .AddTestFilter(x => x.Tags.All(tag => tag.Name != "@just-this-one")).

This is a pattern we found ourselves repeating often, so we added two attributes to handle the most common cases.

• [DebugRunThis] will run the annotated feature(s) and ignore all other feature without this marker.
• [DebugIgnoreThis] will mark the annotated feature as ignored, without having to modify the feature file.

Related Projects

• UnderTest.FeatureLint - Linter for gherkin feature files.
  • VSCode extension - VS Code extension bringing UnderTest.FeatureLint into VS Code.
• UnderTest.FeatureTransform - A project for transforming feature files into various other format of living documentation.
• UnderTest.Nuke - Nuke extensions for UnderTest

License

UnderTest is a free open source project, released under the permissible MIT License. This means it can be used in personal and commercial projects for free. MIT is the same license used by such popular projects as Angular and Dotnet.

Known issues

1. Currently, to have a feature file update after saving, you must build the project to have the feature file copy to the bin folder.

If anyone knows any tricks to help with this, please create an issue outlining the steps.