Command-line tool to test the optimizations performed to a program

command, line, tool
pip install fast-cli==0.0.8



A command-line tool to test the optimizations performed to a program. With fast you can perform benchmarks to different executables in a really easy way!


$ pip install fast-cli

fast is still in a very early stage of development. This means that lots of changes are likely to occur, and your old benchmarks could need to be changed in order to work with the new versions.


fast tries to load a Python module located in the current working directory. In this module you need to define the different benchmarks that you want to perform to your executables!

Here is how you can define benchmarks in

from fast.benchmarks import benchmark, BenchmarkBase

class DocUsageBenchmark(BenchmarkBase):
    # The original executable from which to compare the optimizations
    target = 'my_executable'

    # A list of optimized executables
    # default: but with `_fast` appended before its extension
    candidates = ('my_executable_v2',)

    # Number of instances that you want for your benchmark samples
    # default: 20
    instances = 20

    # Number of executions per instance
    # The average time is taken as the total time of the instance
    # default: 1
    executions = 2

    # Main label for the generated inputs
    # default: "Input"
    xlabel = "Number of as"

    def args(self, instnace):
        Arguments that should be provided to the executables for the given instance
        return []

    def input(self, instance):
        Input that should be provided to the executables for the given instance
        return "a" * instance

    def label(self, instance):
        Label for the given instance
        return instance

The command benchmark will run the benchmarks in that are decorated with @benchmark, also it accepts the names of the benchmarks as arguments. The name of one benchmark is its class name but replacing CamelCase with underscores. For example, you could be able to run the previous benchmark doing: fast benchmark doc_usage

To make the executables, fast invokes make name_of_the_executable before performing each benchmark. Therefore, you need to have a Makefile with the correct rules to generate your executables.


Let's see how we can use fast in a real-world example.

Suppose that we want to optimize a program named swap that reads the input in pairs of bytes and writes them in the output in reverse order. If the number of bytes of the input file is odd, then the last byte it's left in the same position.

We could write as follows:

from fast.benchmarks import benchmark, BenchmarkBase

class DataSizeBenchmark(BenchmarkBase):
    target = 'swap'
    instances =  20
    executions = 3
    xlabel = "Data size (Mbytes)"

    def input(self, instance):
        return "abc" * ((instance * 1024 * 1024) / 3)

    def label(self, instance):
        return instance

We define a benchmark that is going to increase the input size from 1 Mbyte to 20 Mbytes targeting the swap executable. We have not defined any candidate, so by default fast is going to try to make a swap_fast executable to perform the benchmark.

Now, we perform the benchmark:

hector@Ubi:~/example/swap$ fast benchmark
Making executables...
make: `swap' is up to date.
gcc  swap_fast.c -o swap_fast
Benchmarking data_size...
  Checking differences between swap and swap_fast...
    Checking input with args []...
    Checking input with args []...
    Checking input with args []...
    Checking input with args []...
  Generating stats for swap...
    Data size (Mbytes)  Time (s)
    1                   0.2670
    2                   0.5348
    3                   0.7982
    20                  5.4169
  Generating stats for swap_fast...
    Data size (Mbytes)  Time (s)
    1                   0.0023
    2                   0.0024
    3                   0.0046
    20                  0.0192
  Generating plots for data_size...
    Generating plot: Data size (Mbytes) vs Time (s)
    Generating plot: Speedup

fast creates .stats files for each executable of the benchmark and two plots in .pdf: one that shows the Input vs Time of the target and the candidates, and another that shows the Input vs Speedup.


Data size (Mbytes) - Time (s) Data size (Mbytes) - Speedup

Our optimization worked. We got huge speedups for all the inputs!