MuTaR: Multi-task Regression in Python

Description

MuTaR is a collection of sparse models for multi-task regression. Mutar models fit regularized regression on a sequence of related linear models (X_1, y_1) ... (X_k, y_k) and follows scikit-learn's API. Compared with scikit-learn's MultiTaskLasso, MuTaR allows for a different design data X for each task.

Mutar models include:

• Independent linear models:

  • Independent Lasso estimator
  • Independent Re-weighted (Adaptive) Lasso estimator

• Group-norms multi-task linear models:

  • GroupLasso: The Group Lasso is an l1/l2 regularized regression with identical feature supports across tasks (Yuan and Lin, J. R Statistical Society 2006).
  • DirtyModel: Dirty models are a generalization of the Group Lasso with a partial overlap of features. They are defined using a composite l1/l2 and l1 regularization (Jalali et al., NeurIPS 2010).
  • MultiLevelLasso : Multilevel Lasso is a non-convex model that enhances further sparsity and encourages partial overlap with a product decomposition (Lozano and Swirszcz, ICML 2012).

• Optimal transport regularized models:

  • MTW: Multi-task Wasserstein is a sparse regression model where relevant features across tasks are close according to some defined geometry. (Janati et al., AISTATS 2019).
  • ReMTW: Reweighted MTW is a non-convex variant of MTW that promotes even more sparsity and reduces the amplitude bias caused by the L1 norm. Both models are implemented with a concomitant argument for inferring the standard deviation of each task and adapting the amount of regularization accordingly.

Installation

To install the last release of MuTaR:

pip install -U mutar

To get the current development version:

git clone https://github.com/hichamjanati/mutar
cd mutar
python setup.py develop

We recommend creating this minimal conda env

conda env create --file environment.yml
conda activate mutar-env
git clone https://github.com/hichamjanati/mutar
cd mutar
python setup.py develop

Example

>>> import numpy as np
>>> from mutar import GroupLasso
>>> # create some X (n_tasks, n_samples, n_features)
>>> X = np.array([[[3., 1.], [2., 0.]], [[0., 2.], [-1., 3.]]])
>>> print(X.shape)
(2, 2, 2)
>>> # and target y (n_tasks, n_samples)
>>> y = np.array([[-3., 1.], [1., -2.]])
>>> print(y.shape)
(2, 2)
>>> gl = GroupLasso(alpha=1.)
>>> coef = gl.fit(X, y).coef_
>>> print(coef.shape)
(2, 2)
>>> # coefficients (n_features, n_tasks)
>>> # share the same support
>>> print(coef)
[[-0.8  0.6]
 [-0.  -0. ]]

Documentation

See the doc and use examples at the MuTaR webpage.