Opinionated acceptance testing framework.
Install-Package UnderTest -Version 0.9.0-rc010
UnderTest is an open source testing toolkit for building world-class behavior driven (BDD) based enabled projects and culture.
UnderTest is focused on improving the life-cycle of test cases from requirements to test suite to reporting through continued iteration.
One of the key things we strive to enable is for team members building testing suites to reuse web developing skills. Developing with UnderTest hopefully feels like writing web API applications.
Below are a list of examples to learn from.
dotnet new UnderTest --name "MyProject"
- see Usage for other options
After we have a working project, we need to setup UnderTest. Add the following to the Main method in your Program.cs
.
return new UnderTestRunner()
.WithCommandLineArgs(args)
.WithProjectDetails(x => x
.SetProjectName("YOUR PROJECT NAME Test Suite"))
.WithTestSettings(x => x
.AddAssembly(typeof(Program).Assembly))
.Execute()
.ToExitCode();
There is more capability available to configure UnderTest, but this is enough to get us started.
UnderTest is made up of a few key principles.
Gherkin is the lifeblood of any BDD project. UnderTest lives its best life by using gherkin to drive all portions of the pipeline.
Features are stored in the Features
folder of the project.
An example feature, named say Example.feature
(this name will matter more a little later) might look like:
Feature: Simple feature
Scenario: Simple scenario
Given something is true
When something happens
Then this should now be true
Feature handlers are the plumbing that connect our features to the associated functionality.
If you are comfortable with web programming, and specifically MVC, Feature handlers are the controllers of the UnderTest suite. Requests are submitted to handlers and they handle them.
A handler for the above Simple Feature
might look like:
using UnderTest.Attributes;
namespace YourCompany.YourProjectTestSuite.FeatureHandlers
{
[HandlesFeature(@"Example.feature")]
public class SimpleFeatureHandler : FeatureHandler
{
[Given("something is true")]
public void SomethingExists()
{
// todo - we will learn more about what goes here later
}
[When("something happens")]
public void SomethingHappens()
{
// todo - we will learn more about what goes here later
}
[Then("this should now be true")]
public void ThisExists()
{
// todo - we will learn more about what goes here later
}
}
}
When we start building real test suites, we generally will want to start running slices or categories of our available tests. This is where Test Filters come in.
These filters are configured by default through code on the TestSettings object. You can layer and mix and match filters. A filter to run any feature tagged with @smoke
would look like:
.OnlyRunTheseTags("@smoke")
Our updated Program.cs
with this new change might look like:
return new UnderTestRunner()
.WithCommandLineArgs(args)
.WithProjectDetails(x => x
.SetProjectName("YOUR PROJECT NAME Test Suite"))
.WithTestSettings(settings => settings
.AddAssembly(typeof(Program).Assembly)
.OnlyRunTheseTags("@smoke"))
.Execute()
.ToExitCode();
and to ignore the same tests
.DoNotRunTheseTags("@smoke")
We can also add custom filters using the AddTestFilter
method:
.AddTestFilter(x => x.Tags.All(t => t.Name != "@smoke"))
returning true
from the passed lambda will cause the test to be ignored.
Many times when building test suites, we will want to run one specific test.
With test filters you could add a tag, then write a filter to handle that - maybe something along the lines of .AddTestFilter(x => x.Tags.All(tag => tag.Name != "@just-this-one"))
.
This is a pattern we found ourselves repeating often, so we added two attributes to handle the most common cases.
[DebugRunThis]
will run the annotated feature(s) and ignore all other feature without this marker.[DebugIgnoreThis]
will mark the annotated feature as ignored, without having to modify the feature file.UnderTest is a free open source project, released under the permissible MIT License. This means it can be used in personal and commercial projects for free. MIT is the same license used by such popular projects as Angular and Dotnet.
If anyone knows any tricks to help with this, please create an issue outlining the steps.