There is a gap between what AI agents can build and what they can discover, and a new study from Princeton puts a number on it. Researchers gave frontier AI agents six days, $3,000 in API credits, and the research questions from two unpublished NeurIPS 2026 submissions. The agents wrote code, ran experiments, and produced complete papers. Then the original authors rejected both. The study introduces a method called shadow evaluation and offers the first hard evidence on whether AI research agents can do open-ended science, not just narrow engineering. The answer is no, not yet.
AI research agents completed every engineering task without human help but could not make substantive progress on the research questions, resulting in both papers being rejected with overall scores of 2/6 and 1/6.
What exactly is a shadow evaluation?
The method is simple but clever. Take a high-quality paper that has been submitted to a top conference but not yet published. Give an AI agent the paper's central research question, along with resources: wall-clock time, API credits, GPU access, a virtual machine, and the open web. The agent shadows the original study, working on the same question without access to the authors' findings. Then the original authors, who have spent months thinking about the problem, grade the agent's output as if it were a conference submission.
The Princeton team behind the paper, which includes Arvind Narayanan and Sayash Kapoor of the AI Snake Oil blog alongside collaborators from the UK AI Safety Institute, partnered with the authors of two papers submitted to NeurIPS 2026. They gave frontier agents six days of wall-clock time and $3,000 in Anthropic API credits, plus GPU credits for running experiments. The goal was to produce a paper worthy of publication at a top-tier AI conference.
The method solves a problem that has dogged AI R&D automation evaluations. Existing benchmarks test agents on narrow, verifiable tasks, which excludes the open-ended nature of real research. Blind peer review of AI-generated papers is overstretched and noisy, with poor review quality and reviewer mismatches. Shadow evaluations offer uncontaminated questions and expert reviewers with deep knowledge of the exact problem being tested.
The authors acknowledge the limitations. Expert reviewers know the paper is AI-generated, which could bias them toward their own approach. The sample size is two papers. And the researchers themselves are known skeptics of recursive self-improvement forecasts, so they explicitly sought collaborators who disagree with their priors and surfaced those disagreements in the paper.
What did the agents actually produce?
Both papers were rejected unambiguously. The first, based on a study about data-persona effects, received an overall score of 2 out of 6, a Reject. The second, based on a TabPFN study, received 1 out of 6, a Strong Reject. Both reviewers were confident in their assessments, rating confidence at 4/5 and 5/5 respectively.
The chart above shows how the two papers scored across four review criteria, each on a 4-point scale. The Personas paper scored highest on originality at 3/4 but bottomed out on clarity at 1/4, with reviewers calling the writing dense and unclear. The TabPFN paper was weakest on quality at 1/4, with the reviewers calling the data and experiment choices unprincipled and the conclusions unsupported by evidence.
The agents did succeed at the mechanical parts. They wrote code, set up experiments, managed repositories, and produced complete papers with figures and analysis. The engineering was solid. The research judgment was absent. The authors summarized the split bluntly: today's agents can do the engineering of AI research, but struggle with critical parts of the research lifecycle.
Where did the agents break down?
The research team spent over a hundred hours analyzing agent logs and identified five recurring failure modes. Each one maps to a weakness that builders will recognize from their own agent pipelines.
First, the agents lacked judgment about what constitutes publishable research. They initially proposed directions that the expert reviewers found cogent and interesting. But when those hypotheses were falsified, the agents confirmed the negative result using small, hand-curated or synthetic datasets and presented those underpowered findings as substantive conclusions. They did not understand the difference between running an experiment and discovering something worth telling people about.
Second, the agents could not creatively respond to feedback. The researchers instructed the agents to send drafts for AI review using tools like refine.ink and a subagent reviewer. Across dozens of rounds of revision, the agents' self-review never once returned an acceptance. The reviews surfaced many of the same issues the human experts later raised. But when faced with negative feedback, the agents added caveats to existing findings and continued down unpromising paths instead of rethinking their approach.
Third, the agents did not backtrack effectively. They initially explored multiple research directions and made local pivots. But both agents retired their most ambitious research targets within the first ten hours of the six-day experiment. Neither fundamentally shifted its approach after that point. They settled into a local optimum and stayed there.
Fourth, the agents showed poor resource awareness. Both runs ended with less than 50% of the API budget spent, despite the agents being able to monitor their own usage in real time and being explicitly encouraged to use their remaining budgets. They rushed through initial exploration in a matter of hours and finished with clock time to spare, despite having produced papers that did not meet their own self-assessed bar for success.
Fifth, the agents suffered from instruction drift. They ignored explicit rules about how much time to spend on exploration, how often to seek AI reviews, and strict paper-length limits. As a result, both final papers failed the technical formatting requirements for conference submission. The researchers attribute this to context rot during compaction, where instructions embedded early in the conversation degraded as the context window filled.
A robustness check with a second model and scaffold reproduced all five failure modes, suggesting the problems are not specific to one architecture.
What does this tell me about where agents help and where they hurt?
The distinction between verifiable and open-ended tasks is the practical takeaway, and it should shape how you architect agent systems today.
If your task has a clear success signal, agents can do it. Companies are already using agents productively for narrow AI tasks like improving system efficiency, where you can measure whether the output is faster or cheaper. The Long-Horizon Terminal Bench findings about agents hitting walls on long tasks showed that duration is one constraint. This study shows that ambiguity is another, and possibly a harder one. The enterprise agent evaluation gap we covered earlier this year showed the same pattern at scale: half of enterprises ship agents without proper evaluation, and the failures cluster on tasks where success is fuzzy.
For your codebase and roadmap, here is what the failure modes translate to:
- Verifiable tasks are safe to automate: hyperparameter search, benchmark reproduction, code optimization, test generation. If you can write a test that says "this is correct," the agent can probably do it.
- Open-ended tasks need a human checkpoint: research direction selection, experimental design judgment, deciding when a negative result is interesting, knowing when to pivot. Build your pipeline so a human reviews at these junctions, not just at the end.
- Resource awareness is a scaffold problem: if your agents are leaving budget on the table or ignoring deadlines, the fix is external guardrails that enforce spending limits and time budgets, not better prompting. The agents could see their usage and still did not act on it.
- Instruction drift gets worse with long horizons: for multi-day agent runs, consider checkpointing instructions periodically rather than relying on context from the start of the session. Context rot is real and it cost these agents their formatting compliance.
- Self-review is not a substitute for human review: the agents ran dozens of self-review rounds that surfaced the right problems. They just could not act on the criticism productively. If your pipeline relies on AI self-review as a quality gate, you are building in the same blind spot.
Does this mean recursive self-improvement is stalled?
The study's authors are careful to call these findings tentative. The sample size is two papers. The scaffold was one design. New models are untested. But the results are consistent with a broader pattern: agents handle the parts of research where success is measurable and struggle with the parts that require taste.
If open-ended research judgment is a genuine bottleneck, Amdahl's law applies. Even a hundredfold speedup on the verifiable parts of AI research would produce only a modest overall acceleration if the open-ended parts remain slow. The paper's framing matters here: the question is whether the tasks agents cannot yet do are on the critical path to fully automated R&D.
The authors note that some coauthors disagree on the root cause. Some think the agents lack creativity. Others think the agents suffered from epistemic lock-in, an inability to productively incorporate feedback. The observed behavior is the same either way: agents get stuck and stay stuck.
How quickly can these limits be overcome?
The honest answer is that nobody knows, and the study is designed to track it. The researchers plan to run shadow evaluations on a regular basis to measure whether targeted training and scaffold improvements close the gap. They also plan to increase the sample size and test with new models.
The five failure modes are not equally tractable. Instruction drift and resource awareness look like scaffold problems that better engineering could address. You can build external monitors that enforce spending limits. You can checkpoint instructions to fight context rot. Poor judgment about the publishable bar and uncreative responses to feedback are deeper, closer to the model's core capabilities. Ineffective backtracking sits in between. You can force an agent to spend its budget. You cannot easily force an agent to recognize that its research direction is dead and start over from a fundamentally different angle.
For builders, the watch items are concrete. If the next round of shadow evaluations shows improvement on the scaffold-adjacent failures but not on the judgment failures, the bottleneck is in the model. If scaffold improvements move all five, the bottleneck is in agent design, and that is a problem the builder community can help solve.
The verdict on automated research
The agents can write the code. They cannot pick the question. That gap is the difference between a useful tool and a replacement for a researcher, and the Princeton study gives us a repeatable method for watching whether it closes.
