python-mle

A Python package for performing Maximum Likelihood Estimates.

Inspired by RooFit and pymc.

mle is a Python framework for constructing probability models and estimating their parameters from data using the Maximum Likelihood approach. While being less flexible than a full Bayesian probabilistic modeling framework, it can handle larger datasets (> 10^6 entries) and more complex statistical models.

To achieve maximum performance, this package (like pymc) uses Theano to optimize and compile statistical models. This also means that models can automatically be evaluated using multiple CPU cores or GPUs. Derivatives used for the likelihood optimization are calculated using automatic differentiation.

Currently, the package is only a basic prototype and will change heavily in the future.

Example

import numpy as np
from mle import *

# Define model
x = var('x', observed=True, vector=True)
y = var('y', observed=True, vector=True)

a = var('a')
b = var('b')
sigma = var('sigma')

model = Normal(y, a * x + b, sigma)

# Generate data
xs = np.linspace(0, 2, 20)
ys = 0.5 * xs + 0.3 + np.random.normal(0, 0.1, 20)

# Fit model to data
result = model.fit({'x': xs, 'y': ys}, {'a': 1, 'b': 1, 'sigma': 1})
print(result)

Optimization terminated successfully.
         Current function value: -21.632165
         Iterations: 25
         Function evaluations: 38
         Gradient evaluations: 38
   status: 0
  success: True
     njev: 38
     nfev: 38
 hess_inv: array([[  1.55949709e-04,  -2.06891597e-06,   4.52439923e-06],
       [ -2.06891597e-06,   8.94222021e-04,  -8.85856496e-04],
       [  4.52439923e-06,  -8.85856496e-04,   1.21017793e-03]])
      fun: -21.632165325132977
        x: {'a': 0.44739489680783401, 'b': 0.31133017710324606, 'sigma': 0.082040126713057424}
  message: 'Optimization terminated successfully.'
      jac: array([ -8.72776888e-07,   5.92010624e-08,   8.06620475e-08])