NanoQuant: Efficient Sub-1-Bit Quantization of Large Language Models

A team of researchers has introduced NanoQuant, a method that compresses large language models to binary and sub-1-bit precision, enabling a 70-billion-parameter model to run on a consumer-grade 8 GB GPU [1][2]. The work, posted to arXiv and last revised in June 2026, describes NanoQuant as the first post-training quantization technique capable of compressing LLMs below the 1-bit threshold [1][2]. The method formulates quantization as a low-rank binary factorization problem, compressing full-precision weights into low-rank binary matrices and associated scaling factors [1][2]. An alternating direction method of multipliers solver initializes the latent binary matrices, after which a block and model reconstruction process tunes the parameters [2]. In one benchmark, NanoQuant compressed Meta's Llama2-70B model by a factor of 25.8× in 13 hours on a single Nvidia H100 GPU [1][2]. The resulting footprint allowed the 70-billion-parameter model to operate within 8 GB of GPU memory, hardware commonly found in consumer devices [1][2]. The code has been released on GitHub under the SamsungLabs organization [2]. Weight-only quantization has become a standard approach for serving LLMs efficiently, but prior methods struggled to reach binary levels without requiring large amounts of data, compute, or additional storage [2]. NanoQuant establishes what the authors call a new Pareto frontier in low-memory post-training quantization [1][2]. The release arrives amid intensifying competition to shrink frontier models. Chinese firm DeepSeek, founded in 2023, drew attention in early 2025 when it released the DeepSeek-R1 model, which it said was trained for roughly US$6 million — a fraction of the reported cost of comparable Western models [7]. DeepSeek's models are described as open-weight, with parameters shared but training data kept proprietary [7]. Alibaba Cloud's Qwen family of LLMs, distributed under Apache 2.0 and other licenses, has also contributed to the open-weight ecosystem [9]. Large language models are defined as machine learning models with many parameters, trained via self-supervised learning on vast text corpora [8]. The ability to compress such models to sub-1-bit levels without sacrificing usability could lower the hardware barrier for organizations that lack access to datacenter-scale compute [1][2]. The NanoQuant paper is available on arXiv and has been indexed on Hugging Face's paper pages, a feature that links preprints to associated models, datasets, and interactive demos [4][5].