A toolset for compressing, deploying and serving LLM


Install
pip install lmdeploy==0.4.2