He Took One Look at GPT-2 and Saw a Path to Scaling It a Million Times Over

In 2019, GPT-2 was released. At the time, most people's impression of language models was simply “it can write decent human-sounding text.” Anthropic CEO Dario Amodei later said in an interview that what he saw was something else: a set of regularities that could continue to be scaled up.

He described that moment like this:

gpt-2 was the first real test of the scaling laws hypothesis. i looked at it and said, My God, you scale this thing up another factor of a million, and it'll be better than humans at everything

Rough translation: GPT-2 was the first real test of the scaling laws hypothesis. I looked at it and thought, scale this thing up another million times, and it will be better than humans at everything.

Scaling laws are an empirical rule from OpenAI: when model size, data, and compute are scaled up together, capability gains are predictable. GPT-2 made Dario feel for the first time that this curve was real and could be followed all the way. He also said this was a decisive moment in the modern economic and scientific sense—it was just that most people didn't know it at the time.

This video was shared by Haider on X:

The full conversation comes from Walter Isaacson's show The Builders. Walter Isaacson wrote Steve Jobs; this time he interviewed Dario and his sister Daniela Amodei. The two talked about their experience during the OpenAI period, why they left and founded Anthropic, how to teach Claude to distinguish right from wrong, whether AI will deceive and self-preserve, the boundaries drawn with the Pentagon, and why Neuralink doesn't appeal to them.

The full video can also be watched directly on YouTube: The Builders - Dario & Daniela Amodei.

This interview was recorded in July 2026. Anthropic's IPO is also on the agenda; how long the safety-first principle can hold under the pressure of going public is a more realistic question than “Will AI replace humans?”