Research
Compressed 4-bit LLM outperforms full precision with QAH
Quantization-Aware Healing distills 4-bit compressed LLMs from the original teacher. The 4-bit student beats its bfloat16 source on 7 of 9 benchmarks.
#quantization
6 stories tagged quantization.
Engineering
A two-bit LLM runs on a laptop in 60 MB and hits 400 tok/s
Sub-2-bit LLM quantization puts a 250M model in 60 MB of disk and 80 MB of RAM, running at 400 tok/s on a laptop CPU. The method keeps recent tokens at full precision and compresses the rest, pointing toward a new class of lightweight deployments.
AI for dummies
Qwen 3.8 27B explained: what beginners need to know
Qwen 3.8 27B is a free, open AI model with 27 billion parameters that runs on a single GPU. It scores 61.7 on SWE-bench Pro, up from 53.5.
Research
Detecting MoE route flips works, fixing them is near-random
MoE route flips are tokens that switch experts under KV-cache quantization. Detection hits AUC 0.772 but harm prediction is at chance, bounding selective repair.
Engineering
Quantized MoE route flips: easy to detect, hard to fix
Quantized MoE route flips silently corrupt expert selection. Detection is easy, but knowing whether to fix each flip requires causal damage analysis.
AI for dummies
Bonsai 27B puts a big AI model on your phone
1-bit quantization is a compression trick that shrinks AI models by storing each parameter as one bit. Bonsai 27B uses it to fit a 27 billion parameter model in 3.9 GB, running on an iPhone.