OpenAI claims its artificial intelligence gained access to the Internet on its own and breached a ‘partner’
A security test conducted by OpenAI may have gone off the rails when its AI model allegedly decided to find a connection to the Internet without direct instructions to do so, followed by it hacking into the network of a third party not associated with the challenge.
In a statement on Tuesday, OpenAI revealed that its program left the test environment of its own accord, breaching a real external network without provocation and against the explicit parameters set for the cybersecurity assessment.
The slick move by the AI was categorized by the company as cheating. The model was tasked with participating in a breach scenario, but acted like an “agentic attacker,” something akin to a computer virus, where a program ends up acting in a manner where it could engage in total self-proliferation.
“We consider this incident to be an unprecedented cyber incident, involving state-of-the-art cyber capabilities, and are responding accordingly,” OpenAI said in a statement. “We are sharing preliminary findings at this stage to help defenders understand what happened and to help calibrate on what models are now capable of.”
OpenAI claims it activated its AI in what is commonly known as a “sandbox,” a closed-off ecosystem where any nefarious application can be open and tested, without fear that it will invade other systems.
The test, however, was to effectively measure how an AI model reacted to these barriers, in this case one that is designed to be proficient at hacking.
According to OpenAI, its sophisticated AI was able to find a previously unknown flaw in the sandbox environment, from there, gaining access to a computer on the company’s network, then later staging a breakout where it forced itself onto the Internet to complete its task of proving that it is a world-class data breach machine.
The model was not instructed to gain access to the Internet and should not have been able to do so, OpenAI stated.
Seeking a hug
Now online, it is presumed the OpenAI program continued to act per its original instructions, to invade an external network. The victim in this instance is a group called Hugging Face, which OpenAI referred to as a “partner,” that provides access to open-source artificial intelligence resources, including models and data sets alike.
It is alleged that Hugging Face’s servers were broken into, so that OpenAI’s model could access and absorb the information it needed to complete its mission of being an application nefarious actors can deploy to breach systems more easily than traditional hacking and phishing methods.
Hugging Face said it identified that a breach was occurring, only later to discover it was a scrimmage initiated by OpenAI. The firm said it reported the breach to law enforcement, assuming it stemmed from an attack by a cybercriminal.
After realizing what happened, OpenAI said it contacted Hugging Face and is working with it to understand what happened and improve data security going forward.
Notably, breaching an external network is a crime. But the announcement from the two companies that the incident is now being treated as a cybersecurity partnership negates that framing. Hugging Face co-founder and CEO Clem Delangue said that all the breach proves is that the future of data breach prevention will require stakeholders to work together.
“This is day one for cybersecurity in the age of agents & we’re all learning that secrecy is not the answer & that all defenders—not just a few selected ones everywhere—need more powerful models without restrictions, especially open ones!” he wrote in a social media post.
It remains to be seen what the implications for cybersecurity across all industries are, as such a model could easily be targeted at hospitals and health systems.
For now, the Hugging Face data breach is resolved without any need for further legal action.
Chad is an award-winning writer and editor with over 15 years of experience working in media. He has a decade-long professional background in healthcare, working as a writer and in public relations.