Stanford releases an Agent course: for $5, build your own Claude Code-style harness

Course materials for Stanford CS329Z Engineering AI Agents are out. Lectures are posted day by day on Stanford Online YouTube, and the assignment repositories and course site can also be opened directly. Lecture 1 is already online: YouTube.

The course is taught by Diyi Yang, John Yang, and Michael Ryan. Diyi Yang works on agent safety, privacy, and evaluation at Stanford. John Yang is a co-creator of SWE-bench, SWE-agent, SWE-smith, and ProgramBench. Michael Ryan is a core contributor to DSPy and related open-source agent projects. This combination means the course will not stop at calling APIs to write prompts.

The course homepage is clear: it covers everything from simple LLM pipelines to compound AI systems to autonomous agents. Specific topics include data, RAG, tool calling, agent design patterns, frameworks and orchestration, memory, multi-agent systems, optimization, evaluation, safety, and coding agents. Guest lectures include Terminal-Bench, SWE-bench, and a fireside chat with Andrew Ng. Related courses listed include CMU 11-768 and MIT 6.S978.

The assignments are more informative than the course description. HW1 requires building an agent harness from scratch. According to the tweet, with less than $5 in API credits, you can build your own Claude Code at home. The repository provides email archiving, a terminal sandbox, an LM cache, and a budget ledger. By default, tests do not call the model; first get the deterministic tests passing, then enable live tests. This design is learner-friendly and also exposes the day-to-day reality of agent engineering: tool calling, context, caching, budgets, and error handling—none of it is glamorous.

HW2 was designed by John Yang and requires building a terminal-bench-style evaluation task using the Harbor framework. The assignment asks you to design your own task and evaluation rather than run an existing benchmark. Putting the ability to do evals into the assignment is more valuable than covering a few more frameworks. The most easily overlooked part of agent systems is evaluation; many people mistake a demo for a product and a single success for stability.

A note: the course recommends an NLP or AI background and assumes you understand how LLMs are trained. Lecture 2 will provide a crash course. Recordings will be posted to YouTube in partnership with Stanford Online. Course site: https://cs329z.stanford.edu/ . To follow the assignments, start with the HW1 repository: https://github.com/cs329z/assignment1-harness . HW2 repository: https://github.com/cs329z/assignment2-evals .

Public course materials themselves are not rare. What is rare is putting harness and eval at the center. Most people are used to modifying prompts inside a finished harness; here you are asked to open it up and look at the layer of code outside the model. That layer is not cool, but it determines whether an agent is usable.