Tag

#knowledge-distillation

1 story tagged knowledge-distillation.

Dumbbell chart comparing 60B bfloat16 versus 60B MXFP4 QAH scores across four benchmarks: AA-LCR 35.3 to 42.7, AIME 2025 70.7 to 76.3, Aider 38.2 to 40.9, and tau-squared-bench 59.4 to 61.7. The 4-bit QAH model wins on all four.

Compressed 4-bit LLM outperforms full precision with QAH

Quantization-Aware Healing distills 4-bit compressed LLMs from the original teacher. The 4-bit student beats its bfloat16 source on 7 of 9 benchmarks.

Lars Cornelissen · Aug 25, 2026