A toolset for compressing, deploying and serving LLM


Install
pip install lmdeploy==0.2.1