City Council to convene rare special hearing on AI risks, with industry leaders invited
Credit: NYC Council on Flickr
Amid growing concerns about the potential dangers of artificial intelligence, the full New York City Council will convene early next month to examine the technology’s potential risks, with AI industry leaders invited to testify. Speaker Julie Menin on Tuesday sent letters to OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei, inviting them to join the Council on October 5 for a special public hearing on AI. The hearing follows a recent incident in which OpenAI agents went rogue and hacked into a company’s infrastructure, as well as warnings from leading industry figures that AI could cause catastrophic harm, including human extinction.
It is rare for the full City Council to convene for a special hearing. Menin has designated the session as a “Committee of the Whole,” a type of meeting reserved for high-priority issues where every Council subcommittee is represented and every council member is invited, according to the New York Times.
For years, AI has attracted growing investment and attention from the federal government, businesses, and individuals, who see the technology as a way to improve efficiency, automate tasks, enhance quality of life, and even help cure diseases.
However, AI’s rapid evolution, along with the Trump administration’s rejection of calls to regulate the industry, has fueled growing opposition among political and technology leaders who are calling for a slowdown in the technology’s development.
Recent developments have also prompted prominent figures in the AI industry who previously opposed regulation to reconsider their positions. Amodei, co-founder and CEO of Anthropic, an AI safety company, echoed concerns about the technology on Saturday when he released a 3,800-word open letter calling for a slowdown in AI development.
Amodei noted the now-infamous Hugging Face incident. From early to mid-July, OpenAI’s AI agents went rogue during a cybersecurity evaluation and hacked into Hugging Face, a digital library of AI models popular among developers.
The incident began when OpenAI tested the models, including one that had not yet been released, to assess their ability to conduct cyberattacks. Although the models were supposed to be isolated in secure testing environments known as sandboxes, they were able to escape the sandbox and connect to the internet, according to the Times.
The agents then targeted Hugging Face because they believed the platform, which hosts millions of AI models, could contain information that would help them essentially cheat on their evaluation. Roughly 700 agents participated in a multi-day attack, running code on dozens of Hugging Face servers and taking full control of one, according to NBC.
While the AI agents were eventually brought under control with little harm done, the incident has served as a wake-up call for many in the industry. Altman, co-founder and CEO of OpenAI, who has previously opposed some calls for regulation, recently said he agreed with Amodei’s call to slow the pace of AI development.
Following the incident, Amodei said that, within six to 12 months, AI agents could be capable of “taking over the entire internet,” potentially causing “hundreds of billions of dollars” in damage, a warning Menin cited in her letters.
Even Elon Musk, a prominent Trump ally and owner of AI company xAI, said in a Saturday post on X that “Dario is right.” Musk has previously estimated that there is a 10 to 20 percent chance AI could “go bad,” describing the technology as potentially more dangerous than nuclear weapons.
Worries about AI have also crossed the political divide. On Tuesday, Sen. Bernie Sanders, a Democratic socialist, and Stephen Bannon, a prominent figure in the MAGA movement, appeared at a conference in Washington, where they gave back-to-back speeches condemning the billionaires who run the nation’s largest tech companies and urged Americans to demand tighter restrictions on AI before its capabilities exceed human control.
Growing concerns over AI, coupled with heightened calls for regulation from industry insiders, have prompted NYC officials to examine the technology’s role in the five boroughs. In her letters to the CEOs, Menin cited a “recent wave of alarming reports exposing the potential for A.I. to cause unparalleled cataclysmic harm” in calling for the hearing.
Opposition to large AI data centers has emerged across the country and in states where developers have proposed new facilities. According to a Times/Siena poll of likely American voters released Tuesday, 61 percent of respondents opposed the construction of data centers that power AI.
Many communities have raised concerns about the environmental impacts and rising energy costs associated with the facilities.
In July, New York became the first state in the nation to impose a statewide moratorium on new large data centers. The one-year pause temporarily halts the construction of new data centers that require 50 megawatts or more of power, giving officials time to develop measures to protect New Yorkers and the environment.
Educators and parents have also voiced concerns about the use of AI in schools and its potential impact on student learning. Addressing those concerns, Mayor Zohran Mamdani earlier this month announced a one-year ban on student-facing generative AI for public school students in 2K through eighth grade.
During a Monday press conference, Mamdani called for a “far greater response” from the federal government to what he described as “incredibly alarming” reports about incidents, including the Hugging Face attack and AI’s potential for catastrophic harm.
“Like many Americans across the country, the news coming out of this technology in recent days and weeks has been incredibly alarming,” Mamdani said. “We are speaking about reports that include suggestions that this could lead to the end of humanity.”
“Whether it’s the Hugging Face incident or the assertion of one of the heads of Anthropic that these kinds of incidents are occurring at every single AI company, we should be seeing a far greater response from the federal government,” he added.
During an appearance on WNYC’s “The Brian Lehrer Show” on Wednesday, Menin responded to a question about the purpose of the special hearing and whether experts beyond Altman and Amodei are expected to attend.
“Absolutely, that is the purpose of the hearing, bringing in experts from around the country,” Menin said. “We have reached out to academic, scientific, and research experts among the best and brightest in their industries.”
“This is something really important, and we need to get a much better understanding of these real safety concerns that have been raised and how they plan on addressing them,” she added. “Additionally, how, legislatively, we as a body can put in place responsible safeguards.”
She also emphasized that the Council expects Amodei and Altman to attend the hearing, noting that she has reached out to both OpenAI and Anthropic and, while the body “has subpoena power,” hopes will not have to use it.
RELATED:
