Karpathy Says AI Is Repeating the 2016 Mistake, and Another 5 Years of Lessons May Be Wasted

A recent comment from Karpathy has been shared a lot on X. The gist: the biggest mistake in AI right now is that people force agents to do work instead of first getting the model's harness engineering right. He also said they made this mistake at OpenAI back in 2016, and it cost them 5 years.

Let's first explain harness. There is no unified Chinese translation for this term; some call it scaffolding, some call it reins, and some call it a control console. It refers to the layer wrapped around the model: how tasks are broken down, how tools are connected, how state is stored, how failures are fed back, and under what conditions it must stop. The model is the engine; the harness is the chassis, brakes, and dashboard.

The replies below are more concrete than the original post. A user named Anurag said most agents today are still monologues: the model writes a few lines of code, and the same model tells you it ran successfully. There's no compiler, no test that has actually failed, no check of whether the file actually exists. Of course it will say done.

The work harness does is boring. Maintain state, touch things outside the model, stuff failure information back into the context, and sample again. That's all it is. But it determines whether the agent is actually doing work or just burning tokens to entertain itself.

Another user, Ondrej, listed three concrete components: memory files, a fail-closed sandbox, and a loop that stops itself. Without these, the model will just keep burning forever. Someone else added a jab: harness work never produces a demo video, so everyone skips it and goes straight to agents.

The lesson from 2016 is here: at the time, they wanted to scale up policies before there was an environment that could reject them. Now models are stronger, but many people are still doing the same thing, just with a bigger sampler. A small model that is not allowed to merge until tests are green is closer to the word 'work' than a frontier model living in a chat box.

But one reply deserves to be singled out. User Islam Hassan pointed out that in the video, Karpathy did not say the words 'harness engineering,' 'the biggest mistake,' or 'cost 5 years.' He said what Karpathy built in 2016 was a computer-use-style agent, and it failed because the technology wasn't ready; what was missing was the model. This challenge is worth including, because quotes in the retweet chain often don't match the original video.

Regardless of how precise the original wording was, there is a solid consensus in the replies: a bare model is just a bunch of weights predicting the next word. What turns it into reliable work is that outer layer.