EmpiricalRisks

This Julia package provides a collection of predictors and loss functions, mainly to support the implementation of (regularized) empirical risk minimization methods.

Test Status:

Currently, the following higher-level packages are depending on EmpiricalRisks:

Regression: solving moderate-size problem using conventional optimization techniques.
SGDOptim: solving large-scale problem using stochastic gradient descent or its variants.

Overview

This package provides basic components for implementing regularized empirical risk minimization:

Prediction models u = f(x; θ)
- linear prediction
- affine prediction
- multivariate linear prediction
- multivariate affine prediction
Loss functions loss(u, y)
- squared loss
- absolute loss
- quantile loss
- huber loss
- hinge loss
- squared hinge loss
- smoothed hinge loss
- logistic loss
- sum squared loss (for multivariate prediction)
- multinomial logistic loss
Notes:
- For each loss function, we provide methods to compute the loss value, the derivative/gradient, or both (at the same time).
- For each (consistent) combination of loss function and prediction model (which together are referred to as a risk model), we provide methods to compute the total risk and the gradient w.r.t. the parameter.
Regularizers
- squared L2
- L1
- elastic net (L1 + squared L2)
Notes:
- For each regularizer, we provide methods to evaluate the regularization value, the gradient, and the proximal operator.