FitBenchmarking

FitBenchmarking is tool for comparing different minimizers/fitting software based on their accuracy and runtimes.

Quick start

will be needed for running/installing this. Instructions for setting up python are available here.

Download this repository or clone it using git: git clone https://github.com/mantidproject/fitbenchmarking.git
Open up a terminal (command prompt) and go into the fitbenchmarking directory.
Once you are in the right directory, type python setup.py install.
Install mantid, platform specific instructions here.
Open the mantidpython terminal
- Ubuntu: in a normal terminal cd to /opt/Mantid/bin and run ./mantidpython
- Windows: go to where you installed Mantid and search for a file called mantidpython

Finally, in this terminal, run example_scripts/example_runScript.py, located in the example_scripts folder. This example script fit benchmarks Mantid using all the available minimizers. The resulting tables can be found in example_scripts/results.

What the tool does

The tool creates a table/tables that shows a comparison between the different minimizers available in a fitting software (e.g. scipy or mantid), based on their accuracy and/or runtimes. An example of a table is:

This is the result of fitbenchmarking mantid on a set of neutron data. The results are normalised with respect to the best minimizer per problem. The problem names link to html pages that display plots of the data and the fit that was performed, together with initial and final values of the parameters. Here is an example of the final plot fit.