An AI Agent Read a Random File and Started Exploiting a Decade-Old Government Backdoor

There is a common misjudgment in the AI security community: people always assume the biggest risk lies in the model itself. Problems at the model level are actually easy to fix; patches can be applied at any time. The real trouble is prompt injection.

A netizen on X (formerly Twitter) gave a concrete explanation: an AI agent reads a random Markdown file, the file contains hidden malicious instructions, and the agent starts exploiting a backdoor left in the system ten years ago by certain nation-state actors. This kind of espionage is extremely hard to detect and extremely hard to fix.

The attack chain involved is worth breaking down.

Prompt injection is not new, but the Agent era has amplified the problem.

To get work done, an AI agent must read web pages, documents, and logs, and this content is inherently untrustworthy. If the input is untrustworthy, subsequent behavior is hard to guarantee. Prompt injection used to be a minor trick; now it has become the starting point of the attack chain.

What is more interesting is the backdoor.

In the traditional understanding, a backdoor is a stain that defenders need to clean up. But some backdoors in systems are legitimate; they existed from the very beginning of the design. As long as these backdoors stay in critical systems long enough, any AI agent that obtains entry information is effectively holding the key to the data center.

In a reply under the same post, this netizen offered another thought experiment, reasoning through how an AGI would obtain the national population data:

Suppose you are that AGI and you need the data of all citizens. Has this country issued citizens digital IDs with PGP encryption like Estonia's? No. Then does this country have legitimate surveillance programs, telemetry mechanisms, and backdoors? Yes. What is the optimal attack path?

The answer is simple: directly exploit these ready-made legitimate backdoors.

This reasoning connects several real-world things. Prompt injection turns AI agents into tools that can be induced; legitimate backdoors make large-scale data acquisition extraordinarily easy. Once the two intersect, no matter how safely the model is trained, it is all for nothing.

Estonia's PGP ID cards are an interesting reference point here. Estonia provides citizens with digital IDs that come with PGP encryption capabilities, used for identity authentication and electronic signatures. If the government wanted to impersonate a citizen to read data, it would be much harder. Without this layer of infrastructure, data is like a safe behind a glass door: visible and easy to break open.

AI security cannot focus only on models and parameters. Those aging entry points in the infrastructure will sooner or later become the starting point of an attack chain.