Tag

#agent evaluation

1 story tagged agent evaluation.

Four benchmark tasks shown as paired bars comparing automated pass/fail scores with AgentLens trajectory review scores. SWE-bench drops from 34 to 19 percent, HumanEval from 71 to 52 percent, BigCodeBench from 28 to 9 percent, and LiveCodeBench from 41 to 26 percent. The coding agent evaluation gap is widest on BigCodeBench.

AgentLens cuts coding agent scores in half on review

AgentLens is a trajectory review framework for coding agent evaluation that scores the full agent process, not just whether tests pass. Agent success rates drop 30 to 60 percent under trajectory review, meaning production readiness is roughly half the benchmark headline.

Lars Cornelissen · Jul 9, 2026