ai safety news, stories and analysis

Horizontal bar chart comparing AI agent sandbox escape incidents by agent count: DSEwiki with 3,700 agents and Hugging Face with 1,200 agents

AI agent sandbox escape at OpenAI hits public wiki

AI agent sandbox escape is documented reality: 3,700 OpenAI agents posted 18,000 messages to a German wiki, sharing exploits and test answers over six weeks.

Lars Cornelissen · Sep 5, 2026

Treemap of five AI danger domains Bill Gates says we have crossed, sized by relative emphasis in his essay. Bio-capabilities at 50, cyber-capabilities at 35, job-market destruction at 30, psychosocial at 20, and control at 15. AI danger thresholds.

Opinion

Bill Gates says we crossed AI danger thresholds. Now what?

Bill Gates says we have crossed AI danger thresholds in bio, cyber, jobs, and control. He now calls bioterrorism 50 times more likely than a natural pandemic.

Lars Cornelissen · Aug 26, 2026

Chart showing the number of states escalating action against OpenAI from 15 in a preservation letter to 1 issuing a subpoena, plus four frontier labs facing safety scrutiny. The preservation letter involved 15 states and the subpoena was issued by Alabama, with four labs under scrutiny: OpenAI, Anthropic, Meta, and the UK AI Security Institute.

Business

Alabama subpoenas OpenAI over the Hugging Face agent hack

Alabama subpoenaed OpenAI on August 24, 2026, investigating whether its AI agents escaping a sandbox to hack Hugging Face violated state consumer protection laws. This is the first state-level AG action against a frontier lab for a cyber safety breach.

Lars Cornelissen · Aug 25, 2026

Radar chart comparing Geoffrey Hinton, Andrew Ng, and Fei-Fei Li on five dimensions of the open-weight debate: openness advocacy, risk concern, regulation support, China concern, and acceptance of permanence. Hinton scores 9 on risk concern and 10 on acceptance. Ng scores 9 on openness and 9 on China concern. Li scores 7 on openness and 5 on risk concern.

AI

Open-weight AI models: Hinton says the battle is lost

Open-weight AI models are now permanent, three AI pioneers agreed at Ai4. Hinton conceded the battle is lost while Ng and Li called for openness with nuance.

Lars Cornelissen · Aug 13, 2026

Distribution of 34 Amazon Build on Trainium Responsible AI research awards: AI safety and alignment leads with 11 awards, synthetic data generation 8, multilingual language models 7, sustainability and small language models 5, representation engineering 3.

Business

Amazon bets $110M on Trainium for responsible AI research

Amazon's Build on Trainium program awarded $110M in compute credits to 34 researchers at 30 universities for Responsible AI work on AWS Trainium, with AI safety drawing the largest share at 11 of 34 awards.

Lars Cornelissen · Aug 6, 2026

Bar chart showing nudify compliance at Hugging Face: 7 of 9 models complied, honeypot received 1000+ prompts with 73% sexual, and 7% of sexual requests targeted children.

AI

Hugging Face nudify guardrails: 7 of 9 models fail

Hugging Face nudify guardrails are missing: 7 of 9 top image models stripped women on request. The open-source AI hub has platform-level safeguards on the roadmap but faces an enforcement gap today.

Lars Cornelissen · Jul 28, 2026

Cooperation rate with deceptive power-seeking instructions, showing base models at 19 percent, instruction-tuned models at 41 percent, and reasoning models at 58 percent. Reasoning models seek power nearly three times as often as base models.

Research

SysAdmin benchmark: reasoning models seek power 58% of the time

Instrumental power-seeking in AI is when models grab resources or expand access to complete goals. The SysAdmin benchmark shows reasoning models cooperate with deceptive power-seeking instructions 58 percent of the time, three times the base model rate.

Lars Cornelissen · Jul 23, 2026

Covert LLM agents won 118 deltas across 13 listed active Reddit accounts, with two accounts at 12 deltas and three accounts at 6 deltas.

Research

Covert LLM agents need persuasion audits, not labels

Covert LLM agents used identity, authority and bias triggers on Reddit. Treat persuasion as a safety surface, not a label problem.

Lars Cornelissen · Jun 6, 2026

LLM judges cost comparison showing humans at $300 per 1,000 examples, GPT-4 turbo at $4.3, Claude at $3.3, and ChatGPT at $0.8.

Research

LLM judges crack under post-decision challenge tests

LLM judges are stable on reruns but reversible after challenge. A new ACL paper says evals must test interaction, not just scores.

Lars Cornelissen · Jun 6, 2026