moe news, stories and analysis

Horizontal bar chart comparing three AUC scores for MoE route flips: flip detection at 0.772, harm prediction given a flip at 0.490, and benefit prediction at 0.499. The last two are at chance level. Source: arXiv 2608.11212.

Detecting MoE route flips works, fixing them is near-random

MoE route flips are tokens that switch experts under KV-cache quantization. Detection hits AUC 0.772 but harm prediction is at chance, bounding selective repair.

Lars Cornelissen · Aug 14, 2026

Bar chart of Qwen3.8-2.4T-A95B benchmark scores: GPQA Diamond at 92.6 percent, AIME25 pass rate at 93.33 percent, GSM8K flexible at 90.52 percent, and SWE-bench Pro at 67.7 percent. Qwen3.8 open weights.

AI for dummies

Qwen3.8 2.4T open weights: what beginners should know

Qwen3.8 open weights give you 2.4 trillion parameters with 95 billion active per token, Alibaba's largest. Here is what beginners should know.

Lars Cornelissen · Aug 13, 2026

Donut chart showing that on fragile tokens, the standard MoE route is the best option on only 0.8 percent of tokens while a better alternative exists on 99.2 percent. Quantized MoE route flips concentrate damage on these fragile tokens.

Engineering

Quantized MoE route flips: easy to detect, hard to fix

Quantized MoE route flips silently corrupt expert selection. Detection is easy, but knowing whether to fix each flip requires causal damage analysis.

Lars Cornelissen · Aug 13, 2026

Bar chart showing Hy3 has 295 billion total parameters but only 21 billion active parameters per token, a Mixture-of-Experts design. Hy3 hallucination rate dropped to 5.4 percent in internal evaluations.

AI for dummies

Hy3 295B open weights: what 21B active means

Hy3 is Tencent's 295-billion-parameter open-weights model with 21 billion active parameters per token. It uses a Mixture-of-Experts architecture to rival larger models at lower cost, cutting hallucination to 5.4 percent.

Lars Cornelissen · Jul 7, 2026