AI Agents Write Confidential Information Into Proposals, and No One Can Trace the Data Path

A sales rep asked an AI agent to generate a customer quote. The agent pulled data from restricted pricing systems, internal communication records, and archived quote files. The output looked fully legitimate, so a second AI agent proceeded to use it to draft the formal proposal. No human reviewed either of the two handoff steps. It was only after unapproved, classified, sensitive information showed up in the final proposal that the team realized no one could pinpoint when the problematic data entered the workflow, nor how it traveled across different systems, files, and AI agents.

This scenario comes from a custom co-produced report by MIT Technology Review Insights and Cyberhaven. Its core message is clear: once enterprises integrate agentic AI into daily workflows, the scope of governance changes. Beyond static files, there is now a continuous stream of data that gets read, copied, summarized, rewritten, reused, and redistributed nonstop. Data is constantly moving, and fragmented across different systems.

Traditional security practices set up defenses around applications, systems, networks and endpoints. This framework is no longer sufficient when multiple AI agents work in relay to complete tasks. The output of Agent A becomes the input of Agent B, and data may pass through files, chat logs, vector databases or work tickets in between. Permission checks can block a single access attempt, but cannot stop data from being modified, spliced, and re-spread after a legitimate access is approved. At the end, the proposal may look grammatically correct and even factually accurate, but has already crossed compliance lines.

This is why data lineage is now brought to the forefront. It answers very specific questions: Where did this piece of data come from? Who has accessed it? At which step was it summarized or rewritten, and which final output did it end up in? This traceability must work for human workers, AI agents, and all hybrid workflows. Without this level of visibility, governance, accountability and trust can barely land in practice. The report references NIST's AI Risk Management Framework which follows a similar logic: risks must be identifiable, documented and manageable.

This report is produced by Insights, the custom content team of MIT Technology Review. As custom content, it is completely separate from the editorial team's news reporting. It was researched and written by humans, with AI tools only used under supervision in the production process — a disclosure that is worth noting. When enterprises procure agentic AI solutions, they are easily attracted by the automation demos, but the real trouble emerges after an incident: whether you can reconstruct the full data path. If one agent writes confidential data into a proposal, and another agent sends the proposal out, liability will not disappear automatically just because AI is added to the workflow chain.

If teams still respond to agentic AI governance questions with the claim that "the model is secure", they will very likely get stuck during audits. Mapping out full data flows first, before setting permissions and policies, is a far more stable approach.

Download the report

Reference link: NIST AI Risk Management Framework