Benchmarkable

Easy benchmarks that are statistically accurate


Keywords
benchmark, profile, statistics, fluent, timing, performance
License
MIT
Install
Install-Package Benchmarkable -Version 1.0.1

Documentation

Benchmarkable

Simple easy .NET benchmarking for little bits of code that uses probability to give you accurate results regardless of how fast your CPU is. When you benchmark a method it will do a sample run to see how many iterations per batch it should run of your method. Then it will run batch after batch until the error percentage is below 1% (the default, this can be changed though).

The idea is that even if your CPU is insanely fast or really slow you will still get a statistically accurate result.

Install

Run the following command in the Package Manager Console (NuGet).

PM> Install-Package Benchmarkable

Or clone and include Benchmarkable.csproj directly

Use

Lets say you wanted to see if string.Contains was faster or slower than using a regular expression

var haystack = "abcdef";
Benchmark.This(() => haystack.Contains("ef"), "string.Contains")
    .Against(() => new Regex("ef").IsMatch(haystack), "Regex.IsMatch")
    .Print();
Label            |Runs      |Ops/Sec                  |Times slower
-----------------+----------+-------------------------+---------------
string.Contains  |27        |13,773,246 +/-3.460%     |1.000x
Regex.IsMatch    |10        |459,500 +/-9.674%        |29.974x

Single method

Benchmark.Just.This(() => Math.Sin(1.23))
	.Print();

Two methods

Benchmark.This(() => { })
    .Against(() => { }, "With a label")
    .Print();

Multiple methods

Benchmark.These(new(Action, string)[]{
    (() => Math.Sin(1.23), "Sin1"),
    (() => Math.Sin(1.23), "Sin2"),
    (() => Math.Sin(1.23), "Sin3"),
    (() => Math.Sin(1.23), "Sin4"),
    (() => Math.Sin(1.23), "Sin5"),
}).Print();

Note: You have to provide labels for each method you want to benchmark when using this syntax

Explicit declaration

var benchmark = new Benchmark();
    benchmark.Add(() => { }, "Label");
    benchmark.Add(() => { }, "Label 2");
    var results = benchmark.Run();
    results.Print();

Benchmark.Settings

The settings must be changed before creating a benchmark.

Value Default Description
Benchmark.Settings.InitialBatchTime 500ms The time to run a benchmark method for initially to calculate the batch size
Benchmark.Settings.MinimumErrorToAccept 1.0 Error % over the last 10 (default) batches to accept as a valid benchmark result
Benchmark.Settings.BatchesToWorkAcross 10 Number of batches to calculate statistics across (and thus error%)
Benchmark.Settings.MaxTime 5000ms Maximum time to run a single method for before terminating, this is if the error % has not yet been met
Benchmark.Settings.Verbose false Print outinformation as a benchmark is run.

Contributing

Any and all help is greatly appreciated. Especially around the statistical elements.

TODO

  • Printing of results, needs to be cleaned up made more versatile. Maybe some lightweight templates
  • Automatically calculate the batch size to one that will lower the error % as fast as possible.