Agents Are Not Working Well? Don't Rush to Switch Models—LangChain Says Build a Knowledge Factory First
If Agents are not being used well, what's missing may not be the model, but the fact that the company's knowledge has not been collected and organized. In practice, LangChain built a knowledge factory: Agents flag gaps, and a background curator updates regularly, forming a cycle that becomes more complete the more it is used. Models can be changed; accumulated knowledge is the real moat.
Many companies have tried Agents, and then abandoned them after using them for a while. LangChain's answer is somewhat counterintuitive: what's missing may not be the model, but the company's own knowledge.
Which metric is calculated how, where the data is stored, and what format reports should be in—much of this lives in the heads of long-tenured employees. When an Agent takes on a task without this context, it naturally gets it wrong. In an article, LangChain CEO Harrison Chase put the problem directly: the most effective employees have this knowledge in their heads, but it is tacit and fragmented—sometimes it exists only in one person's head, sometimes it is scattered across various documents.
They built a 'knowledge factory' for this. The idea is simple: rather than having Agents guess from scratch, continuously collect and organize knowledge such as how the company operates, operating standards, and where data lives, and then feed it to Agents. This cycle has three parts:
- Skills: the steps and scripts for completing tasks, doing things the way the company requires.
- Domain knowledge: facts and definitions about products, customers, competitors, and the organization, which do not go stale easily.
- Data access guidance: where data is, how to query it, and which source to use.
The key is the mechanism behind it: documents do not need to be complete from the start. LangChain's approach is to have the working Agent notice when it is missing information or when information is wrong, and flag it along the way; then a curator agent runs in the background, periodically reviewing these flags and conversation records, and adds the missing knowledge to Skills and documents. Harrison gave an example: while producing a revenue report, an Agent discovered that a column in BigQuery had been renamed. It found the correct column name itself and flagged it; the finance lead fixed several formatting issues at the same time; that evening, after reviewing the records, the curator updated the new column name and new formatting requirements into the skills repository. The next run went smoothly.
This design has an obvious benefit: knowledge does not have to be written all at once. The more Agents are used, the more gaps are exposed, and the more complete the knowledge base becomes. Swapping models back and forth makes little difference; what really creates separation is the experience accumulated over time. One netizen's comment was quite accurate: 'The loop is the moat; the model is rented.'
But there are pitfalls in the details. A netizen shared their practice in a reply: subagents should not directly modify the knowledge base; they should only report results to the orchestrator, and the orchestrator should decide centrally what to change. Otherwise, if every Agent makes changes, the knowledge base quickly turns into a mess. Another netizen mentioned that when two Agents give different corrections to the same piece of knowledge, this often indicates that the knowledge itself is ambiguous, not necessarily outdated. If the curator quietly picks one in the middle of the night, it essentially hides the disagreement. Such conflicts should be handed over to humans, while individual flags can be merged automatically. These two suggestions are worth writing into the plan.
At the end of the article, Harrison made a point: there is no need to wait until all knowledge is written before letting Agents start work. Build the system first, let the work process expose knowledge gaps, and after they are filled, things will be done better in the future. LangChain has already run this model internally and has begun deploying it with customers.
It seems that whether Agents can get work done is increasingly bottlenecked not by the model, but by whether companies are willing to accumulate their own experience.
发布时间: 2026-11-22 09:33