Research
Detecting MoE route flips works, fixing them is near-random
Under KV-cache quantization, MoE route flips are detectable at AUC 0.772, but predicting their harm is at chance, which limits selective repair.
#moe
3 stories tagged moe.
AI for dummies
Qwen3.8 2.4T open weights: what beginners should know
Qwen3.8 open weights give you 2.4 trillion parameters with 95 billion active per token, Alibaba's largest. Here is what beginners should know.
AI for dummies
Hy3 295B open weights: what 21B active means
Tencent's Hy3 open-weights model has 295 billion parameters but only 21 billion active per token, rivalling larger models and cutting hallucination to 5.4 percent.