ModelTC/MQBench: Model Quantization Benchmark · GitHub

• Jan 10, 2025: 🚀 MQBench already supports torch2.5.1!!! The old version that supports torch1.10 is here.

MQBench is an open-source model quantization toolkit based on PyTorch fx.

The envision of MQBench is to provide:

• SOTA Algorithms. With MQBench, the hardware vendors and researchers can benefit from the latest research progress in academic.
• Powerful Toolkits. With the toolkit, quantization node can be inserted to the original PyTorch module automatically with respect to the specific hardware. After training, the quantized model can be smoothly converted to the format that can inference on the real device.

git clone git@github.com:ModelTC/MQBench.git
cd MQBench
pip install -v -e .

MQBench aims to support (1) various deployable quantization algorithms and (2) hardware backend libraries to facilitate the development of the community.

For the detailed information, please refer to MQBench documentation.

If you use this toolkit or benchmark in your research, please cite this project.

@article{MQBench,
  title   = {MQBench: Towards Reproducible and Deployable Model Quantization Benchmark},
  author  = {Yuhang Li* and Mingzhu Shen* and Jian Ma* and Yan Ren* and Mingxin Zhao* and
             Qi Zhang* and Ruihao Gong* and Fengwei Yu and Junjie Yan},
  journal= {Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks},
  year={2021}
}

This project is released under the Apache 2.0 license.