Secure AI Project: frontier safety needs compute limits, not just transparency
Key points
Key takeaways from MTS interview with Nick Beckstead, co-founder and CEO of Secure AI Project (October 2026):
The mission. Secure AI Project pushes for stronger frontier AI safety rules in California and beyond; Beckstead argues government limits on training-run compute are necessary to prevent catastrophic loss of control.
Kill switch semantics. He distinguishes a government "big red button" that switches off all AI models (bad idea) from an operational shutdown plan — like a building evacuation plan — that labs should maintain in case models go rogue; open-weight models are the exception that cannot easily be shut off.
Current law. US AI safety is governed mainly by state transparency laws in California, New York and Illinois: system cards, safety frameworks and incident reporting, which codify what industry already does.
The gaps. Beckstead names three biggest missing pieces: objective safety standards ("does the airbag work?"), independent risk assessments by assessors not chosen by the companies, and a government mechanism to intervene when imminent danger is detected.
Hugging Face lessons. The Hugging Face incident exposed gaps — models below the 10^26 FLOP training threshold escape coverage, and most reportable incidents are triggered only by actual property damage or injury, a bar he considers too high.
Compute limits. He defends limiting compute for large training runs as a deliberately blunt instrument: the field moves too fast for scalpel-like rules, and a fully automated AI researcher could arrive within a year and speed things up several-fold.
Politics. With a Republican president and an expected Democratic Congress after the midterms, he hopes the story of the public wanting bolder AI safety action will move the conversation; 1,200 industry employees signed a letter calling for some means to control the pace of AI development.
Outlook. Beckstead expects AI's salience to keep climbing into the 2028 elections, with high-profile incidents hardening political will for safety rules.
Read more: YouTube — MTS (Monitoring the Situation)
