Paper Proposes Message-Passing Method For Compressing LLM Weights

Post-training quantization is the standard trick for shipping a large language model that does not need a rack of accelerators to answer a single prompt: weights get rounded down to a handful of bits after training, with no retraining step. The paper, posted to arXiv on October 23, 2026 by Yanshuo Chen, Chenxi Liu, Reza Shirkavand and Heng Huang, argues that the common solvers for this step, including GPTQ, LDLQ and block coordinate descent, all share a weakness. Each one locks a block of weights to its single best codeword before looking at the neighboring blocks, so any near-optimal alternative that a later block’s needs would have favored is thrown away the moment it is picked.

The authors reframe row-wise quantization as inference over a posterior distribution across discrete codebooks, and show that the existing solvers are what that framework looks like at zero temperature, where only the single top codeword per block survives. Their method, Message-Passing Quantization, instead carries soft posterior information between blocks for as long as possible and only commits to hard codewords at the very end. It does this in two stages: a vector approximate message passing step computes rough blockwise posteriors, and a second pass refines a hard-decision starting point with exact block coordinate descent.

The numbers

Testing covered 2-bit lattice quantization and 3- and 4-bit scalar quantization, run across dense language models drawn from the Llama, Qwen and Mistral families. The paper reports that MPQ improves both perplexity and zero-shot accuracy over the strongest hard-decision baselines at each precision level, and that it stacks cleanly with preprocessing steps such as activation-aware scaling and incoherence processing rather than competing with them. The submission itself runs to 316 KB and has been accepted at NeurIPS 2026, with code released alongside the paper.

For a reader outside the research community, the practical question is what a method like this changes about deploying a model rather than training one. Quantization quality at low bit widths sets how much memory and compute a served model needs per query, which in turn sets how cheaply a company can run an answer engine or a chat product at scale. A technique that narrows the accuracy gap between, say, 2-bit and full precision weights lets the same hardware serve a larger model, or the same model serve more users, without a retraining bill. None of that is visible in a product’s feature list, but it shows up eventually in what a plan costs or how fast it answers.