Google’s threat unit confirms first AI-driven large-scale exploit

Google’s Threat Intelligence Group (GTIG) reported that it had disrupted an attempted cyber operation involving the use of AI models to develop and weaponise a zero-day software vulnerability for broad exploitation.

GTIG assessed with “high confidence” that an unidentified malicious actor employed an AI model to develop and weaponise a previously unknown zero-day exploit. This enabled the actor to bypass two factor authentication (2FA) on an open-source, web-based system administration tool.

Researchers at the Google unit determined that the zero-day exploit was written as a Python script and exhibited patterns indicative of large language model (LLM) involvement. The code’s characteristics pointed to LLM-generated content.

Characteristics included abundant educational documentation within the code and a fabricated CVSS score. The formatting was highly structured, showing patterns more consistent with LLM-generated output than with manual scripting or conventional automated tools.

The exploit allowed the attacker to bypass 2FA but required possession of valid user credentials, targeting a logical flaw tied to a hardcoded trust assumption rather than a memory or sanitisation error.

GTIG stated that frontier LLMs are increasingly able to detect these semantic issues, where traditional fuzzers and scanners might fail, by reasoning through code logic and developer intent.

Google reported that it collaborated with the affected vendor to disclose the vulnerability and implement countermeasures. The precise system impacted and the identity of the attacker were not specified in the report.

According to the GTIG report, this incident marks the first direct evidence of an AI-generated zero-day being readied for a mass exploitation campaign.

The report also highlights continuing efforts by actors associated with China and North Korea to use AI models in vulnerability research and exploit development. These findings are based on incident response insights from Mandiant and Google’s own investigations.

GTIG outlined that threat actors are now using AI to accelerate the development of infrastructure tools, polymorphic malware, and obfuscation networks, citing activity it links to Russian nexus groups as well.

The group’s research details the emergence of malware such as PROMPTSPY, which uses AI models to interpret system states and generate commands, enabling partially or fully autonomous malware operations. The report says this trend allows attackers to delegate adaptive, at-scale activities to AI, reducing the need for human intervention at various stages of an attack.

AI is also being used for research support throughout the attack lifecycle, with workflows designed to automate analysis and facilitate information operations. GTIG references pro-Russia influence campaigns like “Operation Overload,” where AI-generated content is used for digital manipulation at scale, including synthetic media and deepfakes.

Adversaries are increasingly acquiring anonymised, premium access to commercial AI platforms using automated middleware and registration pipelines, which allow them to evade service restrictions and usage controls. GTIG said such methods underwrite large-scale abuse of AI services by programmatically cycling through trial accounts.

The report highlights the use of supply chain attacks targeting AI environments and their dependencies. Adversarial groups such as “TeamPCP” (UNC6780) have used these methods as initial access vectors, later attempting lateral movement into broader network environments for disruptive actions such as ransomware deployment and extortion. These risks are classified within Google’s Secure AI Framework (SAIF), naming Insecure Integrated Component and Rogue Actions as relevant risks.

GTIG observed the use of specialised repositories for vulnerability research, such as “wooyun-legacy” on GitHub, a plugin aggregating historical bug data from Chinese sources, which is used to train or prime AI models.

Actors, including APT45, were seen issuing large volumes of prompts, analysing vulnerabilities and validating proof-of-concept exploits recursively. This level of activity could not be matched by manual analysis.

The report additionally described experimentation with agentic tools like OpenClaw and OneClaw, and the use of intentionally vulnerable testing environments to refine and assess AI-generated exploits before deployment.