lmdeploy

A toolset for compressing, deploying and serving LLM


Install
pip install lmdeploy==0.5.2.post1