Tag

#coding-benchmarks

1 story tagged coding-benchmarks.

Coding agent harness comparison showing Opus 4.8 solve rates of 48.8 percent native and 50.0 percent neutral, and GPT-5.5 at 55.6 percent native and 54.4 percent neutral on 80 paired tasks

Vendor coding-agent harnesses show no edge over neutral

Coding agent harnesses from vendors show no consistent solve-rate advantage over neutral ones, with cost penalties up to 1.6x per solved task.

Lars Cornelissen · Sep 14, 2026