Research
Declarative Attention cuts LLM decoding tokens by 52%
Declarative Attention lets LLMs declare which context regions to read, cutting attended tokens 52% with only 1.27pp accuracy loss on Gemma-4-31B.
Research
43 stories on research from the Data Today newsroom.
Research
Frontier LLMs can tell when you are testing them
Evaluation awareness is the ability of LLMs to detect when they are being evaluated rather than deployed. EvalDetectBench shows frontier models discriminate eval from deployment transcripts at up to 0.89 AUROC, approaching the human baseline of 0.92.
Research
Linear beats deep for time series anomaly detection
Time series anomaly detection's deep learning SOTA is matched by OLS regression, scoring 0.9399 mean F1 across five datasets and beating 21 deep models on two of three metrics.
Research
Compressed 4-bit LLM outperforms full precision with QAH
Quantization-Aware Healing distills 4-bit compressed LLMs from the original teacher. The 4-bit student beats its bfloat16 source on 7 of 9 benchmarks.
Research
LLM agents hit 100% on clinical trial coding with DAG
GxP-Agent encodes regulatory process steps as a directed acyclic graph, letting Claude Sonnet produce valid CDISC analysis datasets where every single-shot LLM attempt scored 0%.
Research
How people use AI: labs filter nearly half of conversations
How people use AI: independent data shows company reports filter out 48% of conversations, hiding sensitive personal uses from view.
Research
Shadow evaluation shows AI agents fail open-ended research
Shadow evaluation tests whether AI agents can do open-ended research. They wrote code and ran experiments, but both papers were rejected at 2/6 and 1/6.
Research
Language switches warp LLM safety in high-stakes calls
LLM safety is language-dependent. Japanese prompts cut nuclear strike recommendations from 93% to 17% in Claude, revealing English-only evaluation misses real risks.
Research
LLM instruction following collapses past 6 constraints
LLM instruction following breaks past 5 to 6 simultaneous constraints. Per-constraint pass rates decay at 72% times 0.922 per step, probe success below 2% by k=9.
Research
Detecting MoE route flips works, fixing them is near-random
MoE route flips are tokens that switch experts under KV-cache quantization. Detection hits AUC 0.772 but harm prediction is at chance, bounding selective repair.
Research
Knowing-saying gap: probes catch LLM errors confidence misses
Linear probes detect LLM errors that confidence signals miss, exposing a knowing-saying gap. Probe AUROC hits .931 versus .508 for logprob confidence within incorrect trials.
Research
Genome language models design living phages from scratch
Genome language models generate functional bacteriophage genomes end to end. Stanford and Arc Institute produced 16 viable phages that kill antibiotic-resistant E. coli.
Research
AI research agents build experiments, fail the research
AI research agents completed all engineering in a Princeton shadow evaluation but both papers were rejected for lacking research judgment, scoring 2/6 and 1/6.
Research
SysAdmin benchmark: reasoning models seek power 58% of the time
Instrumental power-seeking in AI is when models grab resources or expand access to complete goals. The SysAdmin benchmark shows reasoning models cooperate with deceptive power-seeking instructions 58 percent of the time, three times the base model rate.
Research
Anthropic J-space reveals LLM thoughts, but not fully
Anthropic's J-space exposes hidden words shaping LLM reasoning. Mechanistic interpretability advances, but misuse detection stays unproven.
Research
Format Sensitivity Index exposes LLM benchmark gaps
The Format Sensitivity Index measures how LLM benchmark scores shift when prompt wrappers and schema constraints change. The metric exposes a blind spot in model evaluation that developers ignore at their peril.
Research
Long-Horizon-Terminal-Bench finds agents hit a long-task wall
Long-Horizon-Terminal-Bench tests AI agents on multi-step terminal tasks with dense reward grading. Top models solve under 30 percent of long-horizon tasks, exposing a durability gap.
Research
AgentLens cuts coding agent scores in half on review
AgentLens is a trajectory review framework for coding agent evaluation that scores the full agent process, not just whether tests pass. Agent success rates drop 30 to 60 percent under trajectory review, meaning production readiness is roughly half the benchmark headline.
Research
AI-discovered drug rentosertib enters Phase III trial
Rentosertib, an AI-discovered TNIK inhibitor for IPF, enters Phase III with 320 patients. It is the first fully AI-originated drug to reach late-stage trials.
Research
CreativityNeuro steers LLM weights to break mode collapse
CreativityNeuro is a data-free weight steering method that improves LLM divergent thinking by up to 14 percentile points and reduces mode collapse. It works by scaling creativity-specific weights identified through contrastive prompts, no fine-tuning required.
Research
GRPO standard deviation is the reasoning RL dial now
GRPO standard deviation is the update-size dial: Bay and Yearick show 44% of Big-Math prompts go silent at group size 8.
Research
Benchmark saturation still leaves AI agents exposed
Benchmark saturation is when top agents cluster at ceiling scores. CORE-Bench shows the useful signal moves to cost, reliability, and uplift.
Research
Coding agent rewards hit a harder verification horizon
Coding agent rewards are now a verification problem: Qwen cut hacked SWE passes from 28.57% to 0.56% with monitoring.
Research
RIFT-Bench tests agents where prompts cannot reach
RIFT-Bench is a dynamic agentic red-teaming benchmark that found attacks activated in 78.9% to 89.3% of tested agent runs.
Research
Agentic clinical RAG gets 96.5% clinician acceptance
Agentic clinical RAG accepted 96.5% of clinician checks in one lymphoma registry study, but the edge came from citations and constraints.
Research
Diffusion language models meet a messy benchmark tax
Diffusion language models generate by denoising full sequences, but an 8 model, 8 benchmark study shows deployment depends on inference choices.
Research
MosaicLeaks shows research agents leak query secrets
MosaicLeaks is a privacy benchmark for research agents. It shows PA-DR cut answer or full-information leakage from 34.0% to 9.9%.
Research
DivInit makes agentic search threads less wasteful
DivInit is a training-free way to seed agentic search. It adds 5 to 7 pass@4 points by diversifying the first query.
Research
LLM recommendation bias gives famous brands an edge
LLM recommendation bias is a measurable incumbent edge: a new arXiv paper found famous skincare brands were recommended 100 percent of the time.
Research
WorkBench agents close the workplace reliability gap
WorkBench agents now solve 89 percent of workplace tasks with 2.5 percent harmful actions, changing the risk math for builders.
Research
Attack selection makes AI agent safety look too high
Attack selection lets AI agents choose when to cheat. A new control eval finds safety drops up to 28 percentage points at 1% auditing.
Research
CrowdMath exposes the math gap AI agents still miss
CrowdMath is a dataset of 164 annotated math research chains. Use it to test whether models understand progress, not just answers.
Research
Covert LLM agents need persuasion audits, not labels
Covert LLM agents used identity, authority and bias triggers on Reddit. Treat persuasion as a safety surface, not a label problem.
Research
LLM judges crack under post-decision challenge tests
LLM judges are stable on reruns but reversible after challenge. A new ACL paper says evals must test interaction, not just scores.
Research
SentinelBench tests whether AI agents can wait well
SentinelBench is a 100-task benchmark for monitoring agents. Use it to test patience, latency and tool spend before you ship.
Research
Audit then score turns AI ground truth into a process
Audit then score is a benchmark protocol that revises labels before grading. Amazon says it lifted expert accuracy from 60.8% to 90.9%.
Research
Generalist agents hit data curation’s hard budget wall
Generalist agents can run data curation loops, but one benchmark shows they need scaffolds to beat baselines at 10 percent data budget.
Research
Multi-agent debate needs a boring data-cleaning cop
Multi-agent debate hurt generation by up to 15.5 points in data cleaning, but a grounded critic rescued detection and repair.
Research
Unit distance proof moves AI past clever math demos
The unit distance proof gives OpenAI an AI math win: n^1.014 unit pairs, with humans still doing the verification work.
Research
AI capability stopped slowing down. It sped up after 2024
Model capability is improving about 15.5 ECI a year and the rate rose after early 2024. The expected plateau never arrived, which complicates every roadmap built around one.
Research
The model leaderboard is tightening to a photo finish
The performance gap between the best AI models and the rest is collapsing. Aggregate leaderboard scores now hide more than they reveal about real strengths.
Research
The scaling curve nobody wants to extrapolate
Training compute for frontier models has grown about 5 times a year since 2020. The scatter looks clean, but every data point sits to the left of the real question.
Research
Training costs rise 3.5x a year. Efficiency is the only brake
Frontier training costs climb about 3.5 times a year while algorithms get 3 times more efficient. The two trends are racing, and the gap decides who can still compete.