metriks is a Python package of commonly used metrics for evaluating information retrieval models.
Available Metrics
| Python API | Description |
|---|---|
| metriks.recall_at_k(y_true, y_prob, k) | Calculates recall at k for binary classification ranking problems. |
| metriks.precision_at_k(y_true, y_prob, k) | Calculates precision at k for binary classification ranking problems. |
| metriks.mean_reciprocal_rank(y_true, y_prob) | Gets a positional score on how well you did at rank 1, rank 2, etc. |
| metriks.ndcg(y_true, y_prob, k) | A score for measuring the quality of a set of ranked results. |
| label_mean_reciprocal_rank(y_true, y_prob) | Determines the average rank each label was placed across samples. Only labels that are relevant in the true data set are considered in the calculation. |
| metriks.confusion_matrix_at_k(y_true, y_prob, k) | Generates binary predictions from probabilities by evaluating the top k items (in ranked order by y_prob) as true. |
Installation
Install using pip
pip install metriks
Alternatively, specific distributions can be downloaded from the
github release
page. Once downloaded, install the .tar.gz file directly:
pip install metriks-\*.tar.gz
Development
1. (Optional) If you have virtualenv and virtualenvwrapper create a new virtual environment:
mkvirtualenv metriks
This isolates your specific project dependencies to avoid conflicts with other projects.
2. Clone and install the repository:
git clone git@github.com:intuit/metriks.git cd metriks pip install -e .
This will install a version to an isolated environment in editable mode. As you update the code in the repository, the new code will immediately be available to run within the environment (without the need to pip install it again)
3. Run the tests using tox:
pip install tox tox
Tox will run all of the tests in isolated environments
