Don’t draw boxes yet in AI system design: once models come in, slow, expensive, and wrong all get redefined

Two tweets, one repository, one course. The main thread is clear: AI system design first treats the model as a failure source and a cost item in the request path, then talks about how to place RAG, Agents, and vector stores. Four facts—models are slow, expensive, probabilistically correct, and can be induced—will change every arrow in the architecture diagram.

Pallavi’s tweet primarily recommends the GitHub repository ai-system-design, positioned as learning system design for LLMs, RAG, and AI Agents. It is suitable for looking at the skeleton first and knowing which modules need to be filled in.

In the reply, Ritesh Sonawane recommended Pidoku’s AI System Design. The course page is very detailed: 6 modules, 25 lessons, about 20 hours 35 minutes. The path starts from “after models enter the request path, what slow, expensive, correct, and trustworthy become,” then goes to building blocks, building AI infrastructure from scratch, Agentic systems, security design, and complete worked designs. The last two parts are an Agent platform and an Enterprise RAG Assistant.

The reference architecture on the material page, simplified, is like this:

flowchart LR
U[用户与应用] --> GW[AI网关]
GW --> AG[Agent运行时]
AG --> MR[模型路由]
MR --> API[托管模型]
MR --> SELF[自托管模型]
AG --> MCP[工具/MCP]
AG --> SB[沙箱]
AG --> MEM[状态与记忆]
SELF --> K8S[GPU集群]
AG -.-> OBS[遥测与评估]

This order is worth looking at. Many materials start with RAG or Agents; this one puts “models change design semantics” in the first module. The page states that design follows requirements → numbers → architecture → failure → cost → security, with tools chosen last. First calculate tokens, latency, capacity, and failure modes, then decide how to fill in gateways, caching, sandboxes, and GPU clusters. Conversely, collecting a bunch of tool names first and then assembling an architecture diagram easily produces a diagram that can be presented but cannot run.

The course page also has a constraint: it assumes you know Go and have seen ordinary web backends; it does not require having run GPUs, built RAG, or written Agents. Each lesson has a 1-minute concept, diagrams, real tools, a small Go program, and self-check questions. The Go program uses only the standard library, runs offline, and does not need a GPU, cloud account, or API key. This design is also friendly to ordinary readers: the concept part is understandable, and the code part is left to developers.

The page notes that the protocol version, project status, and publication date were checked on October 4, 2026. This field changes every month; tool names will expire, while design methods are relatively stable. When reading this kind of material, first look at how it calculates and how it decomposes failures, then look at which tools it recommends. Only when you can point out why every arrow in the diagram exists have you really taken system design in.