Google Says Criminal Hackers Used A.I. to Find a Major Software Flaw

A criminal hacking group recently attempted to launch a widespread cyberattack that appeared to rely on artificial intelligence to detect a previously unknown bug, Google said in research published Monday, highlighting the potential threat that A.I. poses to digital security.

Security experts have feared for years that malicious hackers could eventually rely on A.I. models to identify undisclosed flaws in computer code to launch crippling attacks that are difficult to guard against. That fear was largely theoretical until now.

“We have high confidence that the actor likely leveraged an A.I. model to support the discovery and weaponization of this vulnerability,” the report said.

The tech giant did not say precisely when the thwarted attack happened, whom it was targeting or which A.I. platform the hackers used, but the company added that it did not believe it was its own Gemini chatbot.

Google’s research arrives as the technology industry and governments, including the Trump administration, re-evaluate how, and whether, to police advanced versions of A.I., in large part because of growing concerns over what they mean for cybersecurity.

Flaws like the one identified by Google and the hacking group are known as “zero-day vulnerabilities” — security holes that are unknown to the software makers. They were once considered so rare and powerful that they could fetch millions of dollars on black markets used to sell hacking tools.

But new A.I. models like Anthropic’s Mythos, which was announced last month, appear to be so good at finding such holes that Anthropic shared it only with a limited number of firms and government agencies in the United States and Britain. When Mythos was announced, Anthropic said it had identified thousands of zero-day vulnerabilities “in every major operating system and every major web browser,” including many that were decades old.

A.I. models are rapidly upending cybersecurity. Late last year, Anthropic said that state-sponsored Chinese hackers had used its technology in an effort to infiltrate the computer systems of about 30 companies and government agencies around the world. It was the first reported case of a cyberattack in which A.I. had gathered sensitive information with limited help from human operators.

The zero-day flaw was detected by the Google Threat Intelligence Group within the past few months and was exploited by “prominent cybercrime threat actors” in a script of the Python programming language. It would have allowed the hackers to bypass two-factor authentication on “a popular open-source, web-based system administration tool,” though the hackers also would have needed access to valid credentials like user names and passwords to be successful, the company said.

Google declined to identify the administration tool but said it notified the software maker quickly enough to allow for a patch before the attack could do damage. It also declined to identify the hackers.

Google and independent security researchers said the attempted attack was the first known example of a zero-day bug being put to malicious use by hackers enabled chiefly by A.I.

“It’s a taste of what’s to come,” John Hultquist, the chief analyst at Google Threat Intelligence Group, said in an interview. “We believe this is the tip of the iceberg. This problem is probably much bigger; this is just the first tangible evidence that we can see.”

Rob Joyce, the former cybersecurity director of the National Security Agency, said that it can be difficult to know whether a human or machine wrote computer code, adding that, “A.I.-authored code does not announce itself.”

But Google’s clues linking the hack to A.I. — which included excessive explainer text and other curiosities that human coders would have no reason to include — appeared compelling, said Mr. Joyce, who reviewed the findings ahead of their public release. “It is the closest thing yet to a fingerprint at the crime scene,” he said.

Mr. Hultquist said that Google possessed other indicators that bolstered its conclusion that the hacking code was written by A.I., but he declined to discuss them.

The zero-day flaw announced by Google could bolster international calls for controlled releases of the latest A.I. models so specialists can patch problems first. The Trump administration has been assessing ideas that could include a formal government review process for new models, The New York Times reported last week.

Some experts believe A.I. will ultimately strengthen cybersecurity in the long run by allowing the production of flawless software code. But in the short term, they say, governments and companies need to work together to limit the damage models can do to the current internet, which was crafted by imperfect human hands.

“The bleeding-edge models will allow us to build the safest code we’ve ever built,” Mr. Hultquist said. “That is an absolute win for cybersecurity. The challenge is that we have just begun that process, and we have to contend with a world of code that is already out there.”