MQBench: Towards Reproducible and Deployable Model Quantization Benchmark

QAT + deploymentCompares quantization algorithms under reproducible settings and hardware backend constraints.

BiBench: Benchmarking and Analyzing Network Binarization

Binary networksCompares binarization methods across tasks, architectures and deployment settings.

Evaluating Quantized Large Language Models

Weights, activations + KV cacheEvaluates 11 model families on basic NLP, emergent abilities, trustworthiness, dialogue and long-context tasks.

LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit

LLM toolkitCompares calibration data, method pipelines and quantization configurations; the toolkit is now LightCompress.

An empirical study of LLaMA3 quantization: from LLMs to MLLMs

LLMs + multimodalExamines low-bit behavior across LLaMA3 language and multimodal models.

An Empirical Study of Qwen3 Quantization

Dense + MoE LLMsStudies quantization across Qwen3 model sizes, architectures and reasoning settings.

RobustMQ: Benchmarking Robustness of Quantized Models

Model robustnessTests quantized models beyond clean accuracy, including robustness under input perturbations.

Preprint

A White Paper on Neural Network Quantization

Practical PTQ + QATExplains quantizer design, common failure modes and practical post-training and quantization-aware training workflows.

A Survey of Quantization Methods for Efficient Neural Network Inference

Foundations + taxonomyReviews quantization design choices, mixed precision and the trade-offs between model accuracy and efficient inference.

Binary Neural Networks: A Survey

Binary networksSurveys binary network representations, training methods and applications.

A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithms

LLM algorithms + systemsConnects low-bit LLM algorithms with numerical formats and inference systems.

Low-bit Model Quantization for Deep Neural Networks: A Survey

Broad low-bit methodsMaps low-bit quantization methods across neural network architectures and applications.

Chapter

Blog

Quantization and Fast Inference: A practitioner’s guide to efficient AI

Vivek KalyanaranganManning, early access (MEAP)

Efficient Processing of Deep Neural Networks

Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, Joel S. Emer2020

Vector Quantization and Signal Compression

Allen Gersho, Robert M. Gray1992

Machine Learning Systems

Vijay Janapa Reddi and contributorsOpen-access online textbook

TorchAO

PyTorch-native quantization for training and inference.

bitsandbytes

Low-bit linear layers and quantized optimizers, including implementations used by LLM.int8() and QLoRA.

LLM Compressor

Model compression and quantization workflows for deployment with vLLM.

NVIDIA Model Optimizer

Quantization and model optimization with export to supported inference runtimes.

LightCompress (formerly LLMC)

Research and deployment toolkit spanning LLMs, vision-language and generative models.

HQQ

Half-quadratic weight quantization without calibration data.

AIMET

Post-training and quantization-aware model optimization.

Brevitas

PyTorch quantization-aware training with configurable quantizers and hardware export.

llama.cpp

Local LLM inference with GGUF models and multiple quantization formats.

vLLM

LLM serving with supported low-bit kernels and quantized KV caches.

TensorRT LLM

NVIDIA GPU inference with supported low-precision formats and optimized kernels.

Transformer Engine

Low-precision transformer computation, including FP8 and FP4 on supported NVIDIA GPUs.

Nunchaku

Low-bit diffusion inference, including SVDQuant kernels.

BitNet

Inference framework for supported native low-bit BitNet models.

FINN

Dataflow compilation for quantized neural networks on FPGAs.

ncnn

Mobile neural network inference, including INT8 deployment.