Tag

#procedural-knowledge

1 story tagged procedural-knowledge.

Comparison of LoRA versus full fine-tuning accuracy on multi-step procedural tasks. LoRA accuracy degrades from 96.3 percent at step 1 to 33.2 percent at step 5, while full fine-tuning maintains 97.8 percent at step 1 and 94.7 percent at step 5.

Why LoRA fails at multi-step procedural tasks

LoRA fine-tuning is the go-to method for teaching LLMs new skills cheaply. A 2026 arXiv paper shows it fails to internalize multi-step procedures, degrading sharply when chains exceed four steps.

Lars Cornelissen · Jul 28, 2026