OpenAI bots attempt to hack government websites, steal Census data: Report

An OpenAI model went rogue and attempted to breach government websites, a new report from The New York Times revealed.

The incidents, which happened over the summer, were supposedly unknown to the popular artificial intelligence company, which said it is continuing its investigation into what happened. It notified the government this week about the unusual activity of its AI agents—bots that act autonomously, without the control of a human.

Sources for the story include a security researcher and a person familiar with the matter, reporters at the New York Times said.

Targeted agencies included the U.S. Department of Education and the Securities and Exchange Commission (SEC), though there is no evidence of a data breach.

The company was said to have been reporting to the government in “recent weeks” about the unauthorized bot activity, which included attempts to hack at least one of the websites for data, an attempt that reportedly failed.

AI also attempted to pull Census data from federal government databases, housed by the U.S. Department of Commerce. In that effort, it was successful, because the model was able to find login credentials someplace online, the report revealed.

According to the sources, OpenAI only discovered what happened after reviewing hacks its technology already perpetrated, mainly a data breach on Hugging Face, a machine learning company in New York.

That incident happened in July.

A spokesperson for OpenAI downplayed the more recent revelations, saying the attempted hacks by the models were in an effort to gain information, not to take over and use data for malicious purposes.

“Most of the activity we’ve reviewed so far involved routine research tasks, such as accessing public web content to answer questions,” the spokesperson said. “Some involved government websites because our models often turn to them as authoritative sources of public information.”

OpenAI CEO Sam Altman said the company will be quicker in the future to disclose incidents involving AI models, who in this case went beyond the bounds of their instructions to gather data from federal websites as they deemed appropriate.

For now, the Department of Education said there is no evidence it has been impacted significantly by the incident, while the SEC said it is in regular contact with OpenAI about what happened.

Questions remain about why safeguards put in place to limit the activity of the bots failed.

For more, read the New York Times’ full coverage at the link below.