A Python 3 gradient-free optimization library.


Keywords
optimization, multi-objective, parallel
License
MIT
Install
pip install fcmaes==1.3.36

Documentation

fcmaes - a Python 3 gradient-free optimization library

Join%20Chat

logo

fcmaes complements scipy optimize by providing additional optimization methods, faster C++/Eigen based implementations and a coordinated parallel retry mechanism. It supports the multi threaded application of different gradient free optimization algorithms. There are 34 real world tutorials showing in detail how to use fcmaes. See performance for detailed fcmaes performance figures.

fcmaes started as a fast CMA-ES implementation combined with a new smart parallel retry mechanism aimed to solve hard optimization problems from the space flight planning domain. It evolved to a general library of state-of-the-art gradient free optimization algorithms applicable to all kind of real world problems covering multi-objective and constrained problems. Its main algorithms are implemented both in Python and C++ and support both parallel fitness function evaluation and a parallel retry mechanism.

Features

  • Focused on optimization problems hard to solve utilizing modern many-core CPUs.

  • Parallel fitness function evaluation and different parallel retry mechanisms.

  • Excellent scaling with the number of available CPU-cores.

  • Minimized algorithm overhead - relative to the objective function evaluation time - even for high dimensions.

  • New multi-objective/constrained optimization algorithm MODE combining features from Differential evolution and NSGA-II, supporting parallel function evaluation. Features enhanced multiple constraint ranking improving its performance in handling constraints for engineering design optimization.

  • Selection of highly efficient single-objective algorithms to choose from.

  • Ask-tell interface for CMA-ES, CR-FM-NES, DE, MODE

  • Large collection of 34 tutorials related to real world problems: tutorials.

Optimization algorithms

To utilize modern many-core processors all single-objective algorithms should be used with the parallel retry for cheap fitness functions, otherwise use parallel function evaluation.

  • MO-DE: A new multi-objective optimization algorithm merging concepts from differential evolution and NSGA. Implemented both in Python and in C++. Provides an ask/tell interface and supports constraints and parallel function evaluation. Can also be applied to single-objective problems with constraints. Supports mixed integer problems (see CFD for details)

  • BiteOpt algorithm from Aleksey Vaneev BiteOpt. Only a C++ version is provided. If your problem is single objective and if you have no clue what algorithm to apply, try this first. Works well with almost all problems. For constraints you have to use weighted penalties.

  • Differential Evolution: Implemented both in Python and in C++. Additional concepts implemented are temporal locality, stochastic reinitialization of individuals based on their age and oscillating CR/F parameters. Provides an ask/tell interface and supports parallel function evaluation. Supports mixed integer problems (see CFD for details)

  • CMA-ES: Implemented both in Python and in C++. Provides an ask/tell interface and supports parallel function evaluation.

  • CR-FM-NES: Fast Moving Natural Evolution Strategy for High-Dimensional Problems, see https://arxiv.org/abs/2201.11422. Derived from https://github.com/nomuramasahir0/crfmnes . Implemented both in Python and in C++. Both implementations provide parallel function evaluation.

  • Wrapper for cmaes which provides different CMA-ES variants implemented in Python like separable CMA-ES and CMA-ES with Margin (see https://arxiv.org/abs/2205.13482) which improves support for mixed integer problems. The wrapper additionally supports parallel function evaluation.

  • Dual Annealing: Eigen based implementation in C++. Use the scipy implementation if you prefer a pure Python variant or need more configuration options.

  • GCL-DE: Eigen based implementation in C++. See "A case learning-based differential evolution algorithm for global optimization of interplanetary trajectory design, Mingcheng Zuo, Guangming Dai, Lei Peng, Maocai Wang, Zhengquan Liu", doi.

  • Expressions: There are two operators for constructing expressions over optimization algorithms: Sequence and random choice. Not only the five single objective algorithms above, but also scipy and NLopt optimization methods and custom algorithms can be used for defining algorithm expressions.

Installation

Linux

  • pip install fcmaes.

To use the C++ optimizers a gcc-9.3 (or newer) runtime is required. This is the default on newer Linux versions. If you are on an old Linux distribution you need to install gcc-9 or a newer version. On ubuntu this is:

  • sudo add-apt-repository ppa:ubuntu-toolchain-r/test

  • sudo apt update

  • sudo apt install gcc-9

Windows

For parallel fitness function evaluation use the native Python optimizers or the ask/tell interface of the C++ ones. Python multiprocessing works better on Linux. To get optimal scaling from parallel retry and parallel function evaluation use:

  • Linux subsystem for Windows WSL.

The Linux subsystem can read/write NTFS, so you can do your development on a NTFS partition. Just the Python call is routed to Linux. If performance of the fitness function is an issue and you don’t want to use the Linux subsystem for Windows, think about using the fcmaes java port: fcmaes-java.

MacOS

  • pip install fcmaes

The C++ shared library is outdated, use the native Python optimizers.

Usage

Usage is similar to scipy.optimize.minimize.

For parallel retry use:

from fcmaes.optimizer import logger
from fcmaes import retry
ret = retry.minimize(fun, bounds, logger=logger())

The retry logs mean and standard deviation of the results, so it can be used to test and compare optimization algorithms: You may choose different algorithms for the retry:

from fcmaes.optimizer import logger, Bite_cpp, De_cpp, Cma_cpp, Sequence
ret = retry.minimize(fun, bounds, logger=logger(), optimizer=Bite_cpp(100000))
ret = retry.minimize(fun, bounds, logger=logger(), optimizer=De_cpp(100000))
ret = retry.minimize(fun, bounds, logger=logger(), optimizer=Cma_cpp(100000))
ret = retry.minimize(fun, bounds, logger=logger(), optimizer=Sequence([De_cpp(50000), Cma_cpp(50000)]))

Here https://github.com/dietmarwo/fast-cma-es/blob/master/examples you find more examples. Check the tutorials for more details.

Dependencies

Runtime:

Compile time (binaries for Linux and Windows are included):

Optional dependencies:

  • NLopt: NLopt. Install with 'pip install nlopt'.

  • pygmo2: pygmo. Install with 'pip install pygmo'.

Example dependencies:

  • pykep: pykep. Install with 'pip install pykep'.

Citing

@misc{fcmaes2022,
    author = {Dietmar Wolz},
    title = {fcmaes - A Python-3 derivative-free optimization library},
    note = {Python/C++ source code, with description and examples},
    year = {2022},
    publisher = {GitHub},
    journal = {GitHub repository},
    howpublished = {Available at \url{https://github.com/dietmarwo/fast-cma-es}},
}