Surrogate-based Hyperparameter Tuning System
Hydro is a system that automatically applies the hyperparameter transfer theory together with multiple system techniques to jointly improve the tuning efficiency. To learn more about how Hydro works, please refer our paper.
We highlight three key features of Hydro:
Getting Started
Refer to Getting started for complete instructions on environment setup, installation, and integration.
Installation
pip install hydro-tune
Docker Image
We also provide a Docker image fully equipped with all dependencies and environments.
docker pull tonyhao96/hydro
Examples
We provide working examples for end-to-end hyperparameter tuning inside examples
folder.
-
vision
: Image Classification Example -
language
: Language Modeling Example
Artifact Evaluation
Please check osdi23-artifact
branch for the artifact evaluation version of Hydro.
Citation
If you find this code useful in your research, please consider citing:
@inproceedings{Hydro,
author = {Qinghao Hu and Zhisheng Ye and Meng Zhang and Qiaoling Chen and Peng Sun and Yonggang Wen and Tianwei Zhang},
title = {Hydro: {Surrogate-Based} Hyperparameter Tuning Service in Datacenters},
booktitle = {17th USENIX Symposium on Operating Systems Design and Implementation},
year = {2023},
publisher = {USENIX Association},
series = {OSDI '23}
}
Acknowledgements
Hydro is built upon many fabulous open-source repositories, including
ray | mup | hfta | pytorch | transformers