Inference Revenue Is Less Than 5% of Model Companies, Yet Some Are Still Doing the Math on 70% Gross Margin
In a discussion on 20VC, Rory O'Driscoll gave a striking ratio: major inference providers have combined annualized revenue of about $5 billion to $6 billion, while OpenAI and Anthropic together may exceed $100 billion. Measured by dollar amount, inference is less than 5%. He raised a more practical question: if the tokens are roughly the same, why give Claude Code a 70% gross margin instead of buying a 30% gross-margin service from a platform like Modal?
Rory O'Driscoll, General Partner at Scale, gave a set of numbers on 20VC. Four or five major inference providers together have annualized revenue of roughly $5 billion to $6 billion. OpenAI and Anthropic combined revenue may now be $100 billion to $110 billion. Measured by dollar amount, inference is less than 5%.
This line is worth pausing on. Inference is the layer that actually runs tokens and performs generation after a model has been trained. Over the past year, when people discussed AI infrastructure—GPUs, training clusters, model capabilities—inference was often treated as a utility. But Rory broke it down into a more direct question: for the same batch of tokens, would you rather let Claude Code earn a 70% gross margin, or buy from Modal and let the platform earn a 30% gross margin?
He said: “If the tokens are roughly the same, do I want to give Claude Code a 70% gross margin, or buy it on Modal and give them a 30% gross margin? It points to the upside there.”
There is a framing detail here that is easy to overlook. The $5 billion to $6 billion refers to the revenue of a few independent inference providers. The $100 billion is the total revenue of model companies. Model companies also sell inference themselves, and API revenue is booked on their books. So “inference is less than 5%” cannot be directly understood as meaning the inference market is only 5%. More precisely: the revenue pool of independent inference suppliers is still small, and most inference revenue is captured by the model companies themselves. One commenter also noted in a reply that the two largest inference sellers are also model companies, making it hard to treat this as a fully independent market.
Another reply asked a more practical question: who would actually switch to save 40 points of gross margin? That depends on switching costs. Model quality, latency, stability, compliance, and migration effort—each one could eat up those 40 points. Rory’s premise is that “the tokens are roughly the same.” That premise itself is exactly what most needs to be tested. If the tokens are not exactly the same, the gross-margin difference is not a net gain.
The same 20VC episode also discussed Modal raising $750 million at a $15.75 billion valuation, Replit switching its default model from Claude to OpenAI, and why code review may be bigger than code generation. Taken together, the problem with the inference market is not just that it is small; who books the revenue matters even more. The closer the application layer gets to the model, the more it will do this math. Whether model companies take a 70% gross margin or inference platforms take a 30% gross margin will ultimately turn into migration of product defaults.
The full episode is on YouTube and Apple Podcasts. The original tweet is here.
发布时间: 2026-10-24 20:40