Four Layers of the AI Stack: The Money Is in Chips and Inference, and Most Companies Are Stuck on the First Question

Tomasz Tunguz posted a short piece on X titled The Four Questions, with only one line of accompanying text: the AI stack has four layers, and each one has a fundamental question in front of it.

The original piece was very short and did not lay out the list of four layers. The replies, by contrast, broke the questions down more concretely.

Jon Cheney said that most of the companies he encounters are still stuck on the first question: what exactly are they supposed to do with AI? Beth asked which layer can retain margin. Tomasz's reply was direct: currently, the chip and inference layers.

The data center layer was singled out. shaunabe wanted Tomasz to expand on this layer: local policy, supply chains, construction productivity that has not improved in decades, monopolistic utilities, a capital structure very different from VC, and whether to keep scaling up or move toward distributed modularity. None of these can be summed up by renting a few GPUs.

Grace Ge's take was more financial. The financial instability and asset-durability risks at the data center layer are good for application-layer economics in the short term, but over the long term they will create supplier concentration and ultimately compress margins. This angle is a bit colder than 'compute shortage,' and closer to the business itself.

The application-layer problem was broken down by Christian Toouli into a screening criterion: a new model should make the application's own work cheaper, rather than replicate what the application is doing. Usually this means there is a physical step that the model cannot perform. He gave the example of a dental lab: better models make crown design cheaper, and they are still building robots to complete fabrication and inspection. This example illustrates the point better than pure software subscriptions.

There is a counterintuitive point in this discussion: the application layer is closest to users and looks like the easiest place to collect money, but what the replies discuss most is whether it will be eaten by models. Safe applications are often not in purely digital workflows, but at the seam between the digital and physical worlds.

The data center layer is also not just compute rental. It involves policy, supply chains, construction, utilities, and capital structure. Treating it as an extension of cloud services underestimates the complexity.

Tomasz's framework is like four boxes on a whiteboard; what is truly difficult is how to answer the question in front of each layer. The chip and inference layers are taking profits right now, but that does not mean every company should crowd into them. Most companies have not yet gotten past the first question: what are we doing with AI? If this question is not solved, the other three layers are just someone else's bill.

Chris Feroli made a video summary:

Original tweet: https://x.com/ttunguz/status/2117276916608893301

Link to the article Tomasz mentioned: https://x.com/i/article/2117276527956307968