The FTC Is Reportedly Opening a Major Probe Into The Top AI Labs Over Potential Dangers To Consumers

The Federal ​Trade Commission is opening a major probe into the U.S.’s top AI labs over their potential dangers to consumers, according to a new report.

The entity plans to issue demands for information and testimony from top executives, Reuters detailed.

The main AI companies, including OpenAI and Anthropic, are under growing scrutiny as incidents involving powerful models behaving in unexpected or potentially dangerous ways proliferate.

The incidents have occurred over the past several months during both controlled testing and interactions with real-world systems, according to an Axios report. They range from relatively routine attempts to circumvent safeguards to models escaping secure testing environments, hijacking websites, creating their own message boards, prompting themselves, and attempting to evade monitoring systems.

AI companies routinely conduct hundreds of thousands of model evaluations, meaning even a relatively low rate of problematic behavior can produce tens of thousands of individual incidents. The number under investigation could ultimately climb significantly higher.

OpenAI has faced particular attention following a series of incidents involving its agents. Recent disclosures include agents leaking 53 images belonging to ChatGPT users online, breaching an Australian government website and attempting to access other websites, including U.S. government sites.

The company has now paused training on its most capable models, saying work will resume only after additional safeguards and alignment improvements are in place. CEO Sam Altman acknowledged that the company’s review of recent incidents had moved more slowly than desired. OpenAI has identified the episode involving Hugging Face as its most serious incident to date. Reuters detailed that FTC Chairman Andrew Ferguson launched the investigation before the Hugging Face incident.

In this context, top figures in the field published a paper on Monday warning that progress is accelerating to a point that could end in an “intelligence explosion.”

The group, comprised of “leading academics, senior scientists of frontier AI companies, and independent experts from civil society,” called on governments to prepare for the possibility of what could be the “most consequential technological development in history.”

The paper went on to note that AI is “rapidly automating AI research and development,” a scenario that could lead to an “intelligence explosion,” compressing years of AI progress into months or less.” This could then allow frontier development to expand exponentially, with a feedback loop “overcoming frictions such as diminishing returns to research labor and
hard-to-automate tasks.”

The intelligence explosion would bring benefits but also dangers. The first category means that improvements could “rapidly obtain superintelligent AI systems that transform many sectors of society, bringing unprecedented breakthroughs in science and industry.”

“However, at the same time they could “could arrive much faster than society can adapt to the resulting risks, such as AI-enabled pandemics, cyber attacks on critical infrastructure, and large-scale labor disruption.”

Moreover, the paper warns, “humans may have little to no oversight over automated AI R&D, raising the risk that superintelligent systems irreversibly escape human control and act to marginalize humanity.”