18 stories tagged benchmarks.
Long-Horizon-Terminal-Bench tests AI agents on multi-step terminal tasks with dense reward grading. Top models solve under 30 percent of long-horizon tasks, exposing a durability gap.
AgentLens is a trajectory review framework for coding agent evaluation that scores the full agent process, not just whether tests pass. Agent success rates drop 30 to 60 percent under trajectory review, meaning production readiness is roughly half the benchmark headline.
LLM groupthink is the tendency of models to converge on similar answers. Flint scores 7.47 distinct replies out of 10 in Springboards tests.
Benchmark saturation is when top agents cluster at ceiling scores. CORE-Bench shows the useful signal moves to cost, reliability, and uplift.
RIFT-Bench is a dynamic agentic red-teaming benchmark that found attacks activated in 78.9% to 89.3% of tested agent runs.
Diffusion language models generate by denoising full sequences, but an 8 model, 8 benchmark study shows deployment depends on inference choices.
MosaicLeaks is a privacy benchmark for research agents. It shows PA-DR cut answer or full-information leakage from 34.0% to 9.9%.
Subquadratic attention is a sparse LLM design now showing a 56.2x speed test win at 1M tokens. Treat it as promising, gated infrastructure.
DivInit is a training-free way to seed agentic search. It adds 5 to 7 pass@4 points by diversifying the first query.
WorkBench agents now solve 89 percent of workplace tasks with 2.5 percent harmful actions, changing the risk math for builders.
AgentPerf is a benchmark for concurrent AI agents. Its first results put NVIDIA GB300 NVL72 at up to 20x Hopper efficiency.
An AI plateau is mostly an illusion: old benchmarks maxed out and the AGI goalposts keep moving, even as the frontier capability curve keeps climbing.
LLM judges are stable on reruns but reversible after challenge. A new ACL paper says evals must test interaction, not just scores.
SentinelBench is a 100-task benchmark for monitoring agents. Use it to test patience, latency and tool spend before you ship.
EVA-Bench Data 2.0 is a 213 scenario dataset for testing enterprise voice agents across airline, IT, and healthcare HR workflows.
Audit then score is a benchmark protocol that revises labels before grading. Amazon says it lifted expert accuracy from 60.8% to 90.9%.
Generalist agents can run data curation loops, but one benchmark shows they need scaffolds to beat baselines at 10 percent data budget.
The performance gap between the best AI models and the rest is collapsing. Aggregate leaderboard scores now hide more than they reveal about real strengths.