AI Doesn't Lack Context—It Lacks Filtering
Scribe ran a set of experiments: adding a wiki to AI made performance worse. Company documents are just the “digital exhaust” of work. What AI really lacks is not more context but the ability to filter—to know what to ignore.
In 1953, British engineer Colin Cherry conducted an experiment. He put headphones on participants, played different speech in the left and right ears, and asked them to repeat the content from one ear. The participants did this easily, but when the speech in the other ear switched from English to German, no one noticed.
Later, Daniel Simons and Christopher Chabris created a more famous version: people were asked to count passes, a gorilla walked through the center of the screen, and half the people didn't see it.
This is called selective attention. The human brain processes information by first setting a goal and then filtering out all irrelevant information. New parents can sleep through thunder but be awakened by a single cough from their baby. Employees leave a meeting room remembering only the parts relevant to their own team.
LLMs don't have this filter.
The More Context, the Dumber the Agent
In the first week of November, Notion, Box, and Snowflake released in quick succession products that provide business context to Agents. Agents are AI programs that can independently execute tasks; context is the background information they need to do the work. These moves were a response to disappointment over the first year of Agent deployment: Agents made some individual employees faster, but did not make the company better. The market's consensus judgment is that giving Agents more context can solve this problem.
Jennifer Smith disagrees. She is CEO of Scribe, a company that makes process recording tools. In the article The context layer has a clutter problem, she says directly: this assumption is wrong.
Documents, databases, and Slack messages in a company record how work is coordinated, or the output after work is done. They provide clues to models, but only clues. Smith uses an analogy: you can drive to a place but may not be able to explain the exact route. The truly important process knowledge in a company is often tacit, scattered in everyone's head, and has never been written down.
Experiment: More Context, Worse Results
Smith's team ran a set of tests. 30 business tasks from sales, marketing, and product, each involving judgment and not answerable by simply checking rules. Four conditions: the model completes the task alone; add an internal wiki; add a Scribe-recorded workflow; add both the wiki and the workflow recording.
Results:
- When the model outputs alone, it is basically empty talk with placeholders
- Adding the wiki or the workflow recording improves performance by more than 50%
- Workflow recording comprehensively outperforms the wiki across 7 scoring dimensions
- Giving both the wiki and the workflow recording performs worse than giving the workflow recording alone
Why Workflow Recording Is Better Than Wiki
Because workflow recordings are information that has already been filtered by humans. The recording contains only the steps of how the task is actually completed, without distractors that look relevant but are actually irrelevant. It bears the imprint of human attention: what is left after choices have been made.
A wiki is the opposite. A wiki is stuffed with all kinds of topics, most of which are useless for the current task. Giving a model a wiki is like searching for the correct answer amid a pile of distractors.
Last year, Chroma tested 18 models, placing two kinds of distracting content next to the question: completely irrelevant content, and highly relevant content that did not answer the question. The result: even one relevant distractor significantly degraded model performance. Google's 2023 research found the same.
The documents and messages piled up in companies are exactly the source of this kind of distractor.
Better Search Is Just Better Guessing
One example illustrates the point especially well. When an experienced salesperson prepares for a renewal call, they do not read the company's guide on how to do renewals. Instead, they look through Slack to see whether the client's account owner has mentioned leaving. The account owner leaving is the biggest threat to renewal, even if the client has always been loyal and satisfied. This is not in any document, but the salesperson knows to look for it.
An Agent cannot do this. The retrieval system will find all renewal-related materials—sales guides, last year's quote—but it will never find that Slack message about the account owner leaving, because its content has nothing to do with renewals. Smith says that even the best search cannot solve this problem, because company records do not mark which records are important.
What the Community Is Discussing
This analysis sparked quite a discussion on X. Matt Stockton, who works in applied AI, tweeted that he strongly agrees with the core point. He said he has no conflict of interest with Scribe and has not used it, but has been trying similar things. Agents need company context to solve problems, but connecting Slack, Notion, and Linear by default is far from enough. Those are just digital exhaust; after Agents get them, they still have to fill in the blanks themselves. What really needs to be done is turn tacit processes into a format that Agents can read. This is both a UX problem and, even more, an organizational problem. Context is a very sensitive and strange thing, and this field will become increasingly important.
The comments also offer different perspectives. Someone said that when writing SOPs as skills, they often discover that no one ever agreed on the process itself, and Agents have not become a place to resolve those disagreements. Another person said that screen recordings paired with narration, task descriptions, and expected outputs let the Agent get it right the first time. Someone else called this a recipe and suggested having Agents analyze user interactions to generate it automatically.
Smith's response was: still experimenting with best practices, and more data will be shared later.
This experiment was small in scale, used the same model for scoring, and had a limited sample. But the direction is worth paying attention to.
In the first year of Agent deployment, enterprises are beginning to realize a counterintuitive fact: AI does not lack data; what it lacks is knowing what to ignore. And ignoring things is exactly the work humans do every day but have never recorded.
发布时间: 2026-11-22 14:44