Quantized MatMul in CUDA with a PyTorch interface
pip install quant-matmul==1.2.0
Original code from FasterTransformer / TensorRT-LLM: https://github.com/NVIDIA/TensorRT-LLM/tree/main/cpp/tensorrt_llm/kernels
Adapted to support a different quantization scheme.
The Tidelift Subscription provides access to a continuously curated stream of human-researched and maintainer-verified data on open source packages and their licenses, releases, vulnerabilities, and development practices.
Learn more →
Something wrong with this page? Make a suggestion
Export .ABOUT file for this package
Last synced: 2024-03-20 03:45:00 UTC
Login to resync this project