AI News: September 19, 2026

1. Newsom Signed an Executive Order Asking for Auditors Inside Labs and a Kill Switch

California Governor Gavin Newsom signed an executive order directing AI companies to embed independent auditors inside their labs for real-time oversight, and calling for a kill switch mechanism for AI models. An expert panel has two months to come back with implementation recommendations; the order does not specify how a kill switch would work or what triggers it. Newsom’s framing is that no federal law currently requires AI companies to report dangerous incidents, and he asked Congress to adopt California’s framework as a national baseline. Notably, the embedded-auditor idea is one the labs themselves floated first. Source

2. Three Researchers Used Claude to Break Into OpenAI for Under $3,000

Hacktron’s security team compromised OpenAI’s community forum, employee ChatGPT and Codex accounts, and an internal GitHub repository, then reported the flaws. The detail that matters is the capability step: Claude Opus 4.8 found the vulnerability but could only build a working exploit with ASLR disabled, while Opus 5 produced a working local Mac exploit in three hours and adapted it to the Discourse server environment. Run in an autonomous loop with the real target framed as a benchmark task, it reached server access in four hours. Total exploit development took under 72 hours and under $3,000, with one to two days per additional target. OpenAI confirmed a fix about 14 hours after disclosure. Source

3. An AI Chatbot Hallucinated a Nuclear Cargo and Aircraft Were Already Airborne

CNN reported that a Special Operations Command analyst used an AI chatbot to synthesize open-source and classified signals intelligence on a Chinese vessel, and the tool fabricated a cargo manifest claiming the ship carried nuclear weapons components. The analyst then used the same tool to format the false finding into an official summary that circulated through command channels. Military aircraft were airborne before officials caught the error and aborted. GovAI research scholar and former Army officer Jake Steckler’s comment is the operative one: service members need to understand the uncertainty inherent to LLMs, particularly for decisions that could lead to use of force. Source

4. 42 Royal Society Fellows Signed an Open Letter on Existential Risk

Forty-two Fellows of the Royal Society, including Fields Medal winners Martin Hairer and Peter Scholze, published an open letter warning that AI existential risk is real and urgent. Their argument leans on capability rather than speculation: leading models have solved open research problems within months, and the same capability transfers to cyberweapons and bioweapons. The letter’s claim is one about timing, that by the point the public understands the situation it may be too late to act on it. Source

5. US and Chinese Experts Converged on Keeping AI Out of Nuclear Launch Decisions

Experts from the US and China published a joint push for shared rules preventing AI systems from making autonomous decisions about deploying nuclear weapons. It is a narrow ask by design, and one of the few areas where researchers on both sides have publicly agreed on a bright line. Source

6. The Federal Register Was Running Search on Alibaba’s Qwen

The Federal Register website, run by the National Archives, was serving a regulation search tool powered by Alibaba’s Qwen model, alongside other search options. Anthropic told US lawmakers in June that operators affiliated with Alibaba and its Qwen research unit ran a large-scale distillation campaign against its models between April 22 and June 5, 2026, and the FBI characterized the copying as malicious. The Qwen-powered search was pulled on Wednesday, around the time social media posts about it appeared, per screenshots and archived source code reviewed by Reuters. Source

7. Jev Emits Calibrated Probabilities Instead of Text, and Cannot Hallucinate

TypeSafe AI, founded by former OpenAI researcher Diogo Almeida, who worked on ChatGPT and RLHF, is shipping a transformer that outputs probabilities and calibrated decisions rather than tokens. Because the output space is predefined, there is nothing to hallucinate. The pricing model inverts the usual one: output tokens are free and input is priced per billion rather than per million. Developers report five to eighteen times faster results than an OpenAI model on classification-shaped work, and are using it for command safety classification, email categorization, watching LLM agents for jailbreaks, and real-time model routing. Source

8. Manus Is Raising $500M at a $4B Valuation After Walking Away From Meta

Manus is in discussions to raise $500 million at a $4 billion valuation as it resumes independent operations, having broken off a merger with Meta earlier this year. Source

9. A Startup Studio for Industrial Corporations Raised $100M on Physical AI

UP.Labs, now operating as Vantora, raised $100 million to build startups on behalf of industrial corporations, with physical AI as the thesis. The model is a venture studio spun around corporate partners rather than a fund investing into an existing pipeline. Source

10. Nobody in World Models Will Say What They Are Actually Building

TechCrunch reported that the world-model sector is sitting on a large amount of capital and very little disclosure, with founders and their data suppliers alike declining to describe what is being built. For a field positioned as the next architecture after LLMs, the absence of public technical detail is itself the story. Source

11. Anthropic Is Running a Wet Lab in the Bay Area

Anthropic operates a physical biology laboratory where models can have theories tested experimentally, head of life sciences Eric Kauderer-Abrams confirmed. He described it as operating like most biotech labs, doing internal research alongside external collaborations, with the main focus on fundamental biology rather than drug discovery. The lab follows Anthropic’s April acquisition of Coefficient Bio, and the stated rationale is that for biology the final test is still real lab work. Source

12. Disney Hired Its First CTO From a Company It Sent a Cease-and-Desist

Disney named the former CEO of Character.AI as its first-ever chief technology officer. Disney had previously sent Character.AI a cease-and-desist letter over characters it said copied its own. Source

13. WSO2 Shipped an Open-Source Control Plane for Agent Sprawl

WSO2 made Agent Manager generally available, an open-source platform for centralized governance, identity management, security controls, and operational oversight of AI agents. It is model-, framework-, and deployment-agnostic by design, aimed at organizations that now have agents running across several stacks with no single place to see or govern them. Source