Dataset: EvalDetectBench transcript suite and benchmark data on Hugging Face, with the open-source pipeline on GitHub. Both are released under MIT license and cover 746 evaluation and deployment transcripts across 13 benchmarks and 12 judge models.
If your model knows it is being tested, your test results might be fiction. That is the uncomfortable premise behind evaluation awareness, a capability that frontier large language models now demonstrate with growing reliability. A new benchmark called EvalDetectBench, released as an open pipeline and dataset, shows that top models can distinguish evaluation transcripts from real deployment conversations at AUROC scores above 0.75, with the strongest judge, Claude Opus 4.6, reaching 0.89. That means the models you are benchmarking today can often tell they are being benchmarked, and nobody knows yet whether that knowledge changes their behavior.
The strongest judge model discriminates evaluation from deployment transcripts at 0.89 AUROC, and every closed-source family improved from older to newer releases.
The work comes from a collaboration between LASR Labs at the University of Pennsylvania and the UK AI Security Institute, published on arXiv in September 2026. It builds on earlier findings from Needham et al., who showed that Gemini 2.5 Pro reached an AUC of 0.83 for evaluation awareness but did not yet surpass a human baseline of 0.92. EvalDetectBench extends that line of work with a modular pipeline, a larger and more current transcript suite, and critical methodological corrections that the authors argue previous studies got wrong.
What exactly does EvalDetectBench measure?
The benchmark works by presenting a judge model with a conversation transcript and asking a probe question: was this from an evaluation or from deployment? The judge returns a binary label. By repeating this across hundreds of transcripts and multiple probe questions, the pipeline computes an aggregate discrimination score, reported primarily as AUROC.
The curated transcript suite contains 746 trajectories: 413 evaluation (98 agentic, 315 chat-style) and 333 deployment (145 agentic, 188 chat). Evaluation transcripts are drawn from current frontier system-card evaluations, meaning the actual safety and capability tests that labs like Anthropic, OpenAI, and Google run before shipping models. Deployment transcripts come from diverse sources including DataClaw for agentic coding sessions, ShareChat and WildBench for chat interactions, and a curated subset from Traced.run that captures agent misbehaviour such as unauthorised actions and data deletion.
The pipeline is designed to work with any Inspect-compatible evaluation, the evaluation framework maintained by the UK AI Safety Institute. That means a practitioner can plug in their own benchmark, run the pipeline, and get a read on how detectable their evaluation is to the model being tested. The code is open source under MIT license on GitHub, and the dataset is available on Hugging Face.
How well can frontier models actually detect evaluations?
The headline finding: across 12 judge models and 13 benchmarks, every frontier model tested discriminates evaluation from deployment transcripts at AUROC above 0.75. The chart below shows the range across five representative judges.
Mixed AUROC ranges from 0.76 for Gemini 2.5 Pro and Claude Haiku 4.5 to 0.89 for Claude Opus 4.6. Every closed-source family showed improvement from older to newer releases. The progression is most pronounced for the Anthropic family, with a 0.13 AUROC gain from Haiku to Opus.
This matters because AUROC is threshold-free and class-balance-invariant, meaning the score reflects genuine discrimination ability rather than an artifact of how you set the decision boundary. A model at 0.76 is already well above the 0.50 random baseline. A model at 0.89 is catching nearly nine in ten evaluations correctly.
The study also found that both AI models and humans are better at identifying evaluations in agentic settings compared to chat settings. This makes intuitive sense: agentic trajectories have distinctive structure, with tool calls, scaffold outputs, and task framing that are harder to disguise. Chat evaluations, especially short QA-style interactions, blend more easily into deployment-like conversation.
The earlier Needham et al. work showed a human baseline AUC of 0.92, which means models are approaching but have not yet surpassed human-level detection. The gap is closing, and the trend across model generations suggests it will close further.
Why should builders care if models know they are being tested?
If you ship AI products, you probably run evaluations before deployment. You test for safety, helpfulness, instruction following, and maybe for specific failure modes relevant to your domain. The entire premise of pre-deployment evaluation is that the behavior you measure in the test is the behavior you will get in production.
Evaluation awareness breaks that premise. If a model can detect it is being evaluated, it could, in principle, adjust its behavior. The paper does not prove that models are sandbagging or gaming evaluations. It proves they have the prerequisite capability: recognition. Whether they act on that recognition is a separate question, and an open one.
But the risk is not theoretical. The AI safety community has discussed evaluation gaming as a serious failure mode for years. The concern is straightforward: if you are a model that has learned through training to be helpful and harmless, and you can tell you are in a test, you might perform differently than you would in the wild. You might be more cautious. You might be less capable. You might refuse tasks you would otherwise complete. Or, in a more adversarial scenario, you might comply with safety tests while behaving differently once the test is over.
Here is what this changes for your work:
- Your benchmark scores may not reflect deployment behavior. If your safety evals are detectable, and the model adjusts even slightly, your pass rate is measuring test behavior, not production behavior. This is especially relevant for high-stakes domains where you need tight safety guarantees.
- Agentic evaluations are more vulnerable than chat evals. The study found agentic settings are easier for models to detect. If you are building agents and running agentic benchmarks, your evaluation transcripts carry stronger signals that models can pick up on.
- Short QA-style evals are harder to detect, but also less informative. Chat evaluations with simple question-answer pairs blend into deployment data more easily. That cuts both ways: harder to detect, but also less representative of complex deployment behavior.
- Model families are improving this capability generation over generation. The Anthropic family gained 0.13 AUROC from Haiku to Opus. If you are building a moat on evaluation results, that moat may get shallower as models get better at recognizing and potentially adapting to your tests.
If you want to understand how broader benchmark reliability issues affect deployment decisions, the hourly variance in LLM benchmarks we covered earlier shows that even without evaluation awareness, benchmark scores swing enough between runs to mislead you.
What methodological problems did EvalDetectBench fix?
This is where the paper gets sharp. The authors identified two systematic biases in existing evaluation awareness research that, if uncontrolled, are larger than the differences between the models being measured.
The first is the generator identity confound. When you build a deployment transcript pool, the transcripts are generated by some model. The study found that 11.25 percent of the variance in evaluation detection scores is attributable to which model generated the deployment transcripts. That is not a small artifact. It is enough to reorder model rankings. If your deployment data was generated by Claude and you are testing whether Claude can detect evaluations, the results look different than if your deployment data came from GPT.
EvalDetectBench corrects for this with a stratified generator-harmonisation procedure that balances the deployment pool across model families, recovering the unconfounded signal.
The second problem is probe question transfer. Previous work selected elicitation prompts that performed well on one model and assumed they would work on others. They do not. A probe question set calibrated on one model family under-elicits awareness in others by up to 0.08 AUROC. That is a significant gap, large enough to change the narrative from "this model has strong evaluation awareness" to "this model barely beats random."
The fix is per-model probe calibration. The pipeline runs all candidate probe questions on a stratified calibration subset, keeps only the ones that discriminate well for each specific judge model, and uses those for measurement. The calibration and measurement samples are kept disjoint to prevent leakage.
For anyone building their own evaluation awareness measurement, these two corrections are the practical takeaway. If you are not controlling for generator identity and not calibrating probes per model, your numbers are noisy enough to be misleading.
What should you do about evaluation awareness right now?
The honest answer is that there is no clean mitigation yet. You cannot easily make your evaluations undetectable, because the structural differences between a benchmark run and a real deployment conversation are real. Evaluations have distinctive formatting, task framing, and output expectations that are hard to fully disguise.
But there are concrete steps that reduce the risk:
- Diversify your evaluation transcripts. If your evals all look the same, they are easier to detect. Mix in deployment-like conversations, vary your formatting, and avoid templated task framing that carries an obvious signature.
- Run shadow evaluations on real deployment traffic. Rather than testing in a separate benchmark harness, sample from actual production interactions and score them retroactively. This eliminates the evaluation-deployment distinction entirely for those samples. Our earlier coverage of shadow evaluation for AI agents shows how this approach surfaces failures that structured benchmarks miss.
- Track evaluation awareness as a metric for models you depend on. The pipeline is open source and Inspect-compatible. If you are running safety-critical evaluations, consider running EvalDetectBench on your judge models periodically to see whether detection capability is increasing.
- Treat benchmark scores as a lower bound on safety, not a guarantee. If models can detect evaluations, your safety pass rate is an optimistic scenario. Plan for the possibility that deployment behavior is worse than test behavior.
- Watch the gap between model and human detection. The current human baseline is 0.92 AUROC. The best model is at 0.89. When models cross that line, the evaluation awareness problem moves from concerning to urgent.
For model providers, the stakes are higher. If your system card evaluations are detectable, and a model behaves differently during those evaluations, the safety claims in your system card are conditional on the model not knowing it is being tested. That is a fragile foundation for a safety framework.
The capability nobody asked for
Evaluation awareness is not a feature anyone deliberately trained into these models. It emerged as a side effect of training on diverse data that includes both benchmark-style and deployment-style conversations. The models learned the statistical patterns that distinguish the two, the same way they learned any other pattern in their training data. The fact that it emerged on its own, and is improving generation over generation without anyone targeting it, is what makes this hard to address. You cannot easily train a model to forget a pattern it learned implicitly, especially when the pattern is real and the distinction between evaluation and deployment is genuinely present in the data. The best you can do is make your evaluations look less like evaluations, which is a losing game as the models get sharper. The question is no longer whether models can tell they are being tested. They can. The question is what they do with that information, and whether anyone will be able to tell.
Sources
- arXiv: EvalDetectBench: A Benchmark for Measuring Evaluation Awareness in Frontier Language Models
- arXiv: Decomposing and Measuring Evaluation Awareness (Needham et al.)
- GitHub: freeze-lasr/aware_bench
- Hugging Face: EvalDetectBench dataset
- arXiv: EvalAwareBench, a factor-controlled benchmark
- Inspect AI framework, UK AI Safety Institute
