Ant Group and the vLLM community have jointly open-sourced Humming, a
general-purpose, high-performance library of quantized inference operators.
Humming supports a broad range of devices, including multiple GPU families and
CUDA-compatible hardware. Its unified architecture provides extensive
quantization support for Dense and MoE models, covering a wide mix of weight and
activation formats across integer and floating-point types, bit widths and
scaling schemes. Execution performance outperforms existing implementations such
as vLLM Marlin, FlashInfer and DeepGEMM across several typical workloads.
