From 100 Lines of Backprop to All-Night Experiment Runs: Karpathy Breaks an AI Degree into 7 Repositories
Karpathy has published an ordered AI learning path on GitHub, with 7 repositories that go from backpropagation all the way to agents that can modify training code all night. nanochat brings GPT-2-level training cost down to about $48, and all the links are here.
Andrej Karpathy is a former Tesla AI director and one of OpenAI's founding members. He has put an ordered AI course on GitHub, split into 7 repositories. The path starts with backpropagation and ends with agents that can run research all night. For developers, it is an executable roadmap; for non-technical readers, each repository also makes clear what key stage is solving.
Some online users summarized this path in one sentence: from differentiation to agents, building GPT from scratch like stacking blocks.
The order itself has a teaching logic. First understand how neural networks learn, then understand how text becomes tokens, then train your own GPT, then go down to C and CUDA, and finally build a chat model and automated research. A hundred links in your bookmarks are usually less useful than getting the first 100 lines running first.
micrograd: How neural networks actually learn. Backpropagation in about 100 lines of Python code.
github.com/karpathy/microgradnn-zero-to-hero: Video course and lecture notebooks, from a single neuron to GPT.
github.com/karpathy/nn-zero-to-herominbpe: How text becomes tokens. Every large language model starts with this tokenizer, and the code is clean and concise.
github.com/karpathy/minbpenanoGPT: The simplest repository for training and fine-tuning your own GPT.
github.com/karpathy/nanoGPTllm.c: The same training, with no hidden-layer wrappers. Pure C and CUDA, with no dependency on PyTorch lower layers.
github.com/karpathy/llm.cnanochat: A complete end-to-end chat model. GPT-2-level training cost about $43,000 in 2019; here it is about $48.
github.com/karpathy/nanochatautoresearch: An agent that can do research. The agent edits training code, trains for 5 minutes, keeps the improvements, and repeats all night.
github.com/karpathy/autoresearch
The value of this set is not the word "free"; it is that it runs. Each repository compresses abstract concepts down to an amount of code you can actually run. If you really want to learn from scratch, go in order, and don't jump to step 6 to start.
发布时间: 2026-11-01 10:27