Tag

#llm training

2 stories tagged llm training.

Comparison of LoRA versus full fine-tuning accuracy on multi-step procedural tasks. LoRA accuracy degrades from 96.3 percent at step 1 to 33.2 percent at step 5, while full fine-tuning maintains 97.8 percent at step 1 and 94.7 percent at step 5.

Why LoRA fails at multi-step procedural tasks

LoRA fine-tuning is the go-to method for teaching LLMs new skills cheaply. A 2026 arXiv paper shows it fails to internalize multi-step procedures, degrading sharply when chains exceed four steps.

Lars Cornelissen · Jul 28, 2026

GRPO standard deviation chart showing baseline extreme gradient mass at 13.9 percent, GRPO at 24.7 percent, and silent prompts at 44 percent for G=8.

Research

GRPO standard deviation is the reasoning RL dial now

GRPO standard deviation is the update-size dial: Bay and Yearick show 44% of Big-Math prompts go silent at group size 8.

Lars Cornelissen · Jul 2, 2026