OpenRouter Ran 5.2T Tokens in a Day; 64 B300s Handled One-Third—Where Are the GPUs Actually Going?

OpenRouter ran 5.2T tokens yesterday. In a tweet, Chaofan Shou said they used 64 B300s and 64 M890Ps to do about 1.8T, roughly one-third of OpenRouter's total for the day. The original tweet also included a model bill that contained more information than the headline.

The most striking entry in the table is deepseek-v4.1-flash-uncensored: 11 million requests a day, 1.82T input tokens, 9.69B output tokens, and a daily cost of $19,546.20. Another model, deepseek-v4.1-flash, had 1.1 million requests, 140.1B input tokens, 797.1M output tokens, and a daily cost of $2,066.30. glm-5.3-flash had only 130,000 requests and 8.64B input tokens, for a daily cost of $381.20. apex-flash-1 and seedance-2.0-fast-uncensored had even lower volumes.

The table shows two things. First, inference demand on the public API is highly concentrated: a single model variant can consume the vast majority of tokens. Second, input tokens and output tokens differ by several orders of magnitude; the bill looks large, but the actual generation volume is not that dramatic. In the replies, someone asked whether these requests were cache hits—a question more worth examining than the '5.2T' figure itself.

Chon Tang ran the numbers on this set of figures. He said all of OpenRouter's DeepSeek demand accounts for roughly 30% of its total token traffic and could be served by 5 to 10 GB300 NVL72 racks. Even if demand grows tenfold in a year, only 50 to 100 racks would be needed. He cited NVIDIA's projection to ship about 90,000 racks next year, then asked: What am I missing?

The replies did not provide a unified answer, but they offered several caveats. One commenter said OpenRouter is tiny in the enterprise market and that enterprises do not use this routing platform. Another said it is more like an exit poll: it can show trends but cannot serve as a global demand table. Someone else pointed out that even if OpenRouter accounts for only 1% of compute demand, working backward implies total supply of only around 10,000 racks. Chaofan Shou also added a self-deprecating line: the better explanation is that no one uses OpenRouter anymore.

These replies break the question down. OpenRouter is a model-routing platform, not a census of global inference demand. It counts requests that pass through it and does not cover cloud providers' long-term contracts, enterprise private deployments, internal R&D, or training clusters. Token volume also cannot be directly converted into GPU counts. Input tokens can be batched and can hit cache; MoE models activate only a subset of parameters each time; quantization, inference optimization, and utilization all change how much compute corresponds to the same number of tokens. Comparing 5.2T tokens with NVIDIA's rack shipments leaves out too many layers of conversion.

But Chon Tang's puzzlement is still valid. If the top traffic on the public API needs only a few dozen rows of racks, then the projected demand for 90,000 racks must be hiding somewhere else. Possible destinations include training, reinforcement learning, internal inference, cloud providers' long-term contracts, enterprises building their own infrastructure, and inventory of GPUs that were bought but are not heavily utilized. Brandon Carl put it more bluntly in a reply: 'We innovated ourselves into oversupply.' That judgment may not be right, but it is closer to the tension exposed by this table than 'demand is infinite.'

One detail worth remembering is the price. The DeepSeek uncensored variant handled 11 million requests a day for less than $20,000 per day. The inference business for open-source models is very thin—so thin that profit has to be squeezed out through scale, caching, and scheduling. Where the GPUs went may not be answered in OpenRouter's bill. A public routing platform is only a sample; the training and internal R&D that really consume GPUs will not post their itemized details.

Original tweet: https://x.com/shoucccc/status/2115253960907133280

Chon Tang's tweet: https://x.com/chontang/status/2115201354293801111

The platform mentioned by Chaofan Shou: https://platform.yeta.ai