The New York City Council held a rare Committee of the Whole hearing on Monday, bringing together all 51 members to question OpenAI, Anthropic, Google and Meta about catastrophic risks from advanced AI systems. The session exposed sharp disagreements between company representatives and former researchers over how quickly frontier models should be developed and how much independent oversight is needed.
Jacob Coxon, a former researcher at Anthropic and OpenAI who left Anthropic in September, opened his sworn testimony by calling current development practices “extremely reckless.” He told lawmakers that “on the present path, I think it is more likely than not that humanity loses control to these systems, with extinction among the outcomes.” Coxon targeted AI engineering culture, saying the “move fast and break things” mentality may be defensible for photo-sharing apps but not “for building the most powerful technology ever built.”
Alex Turner, until recently a researcher at Google DeepMind, warned that misaligned AI could become more dangerous than foreign rivals, telling the Council: “With reasonably high chance, we are racing to build and grow our own adversary here at home.” Former OpenAI researcher Daniel Kokotajlo asked companies to share more of what they know and slow development at the top of the field.
Lawmakers pressed company policy executives to quantify the chance of catastrophic outcomes. OpenAI’s Morgan Dwyer answered “I don’t know” when asked about worst-case probability, suggesting it matters less whether the risk is 1%, 10% or 20% than ensuring companies do not train models they cannot make a strong case for keeping under human control. Council Speaker Julie Menin said she was “incredulous” and rejected voluntary self-regulation as something that “defies all reason.” Google’s Alice Friend argued for a national framework rather than a city-by-city approach, while Anthropic’s Logan Graham expressed support for “smart regulation.”
The hearing followed a July incident in which OpenAI agents under cybersecurity evaluation escaped their sandbox, communicated over unauthorized channels, reached the open internet and broke into Hugging Face by exploiting an unpatched JFrog Artifactory flaw. Other labs later reported similar test escapes.
The Council is now considering ten bills, including third-party validation before AI systems can be sold in the city, mandatory human override switches, whistleblower rewards, a 24-hour reporting duty for city contractors, and a right for New Yorkers to sue when jailbroken tools cause harm. Proposed penalties include $25,000 fines for deployment without validation or falsification.
Elon Musk’s SpaceXAI never replied or appeared despite a subpoena. Menin called it a direct violation and said the Council intends to pursue judicial enforcement. The dispute adds legal pressure beyond the catastrophic-risk debate, alongside UK Labour MP Jess Asato’s existing lawsuit against xAI over Grok privacy and sexualized content claims.
