The middlemen selling tokens are turning GPU rentals into a fast-moving business

A group of inference providers is using rented GPUs to run open-source models and sell tokens, reaching $1 million in monthly revenue within months.

Deedy broke down the economics in a tweet. Inference providers do not train models; they rent compute such as B300, GB300, and Vera Rubin, deploy open-source models such as GLM 5.3, Kimi K3, and DeepSeek 4.1 Flash/Pro, and charge by token. Even at $8/GPU-hr to rent B300, at 100 tok/s throughput some open-source models can generate $10 to $20/GPU-hr in revenue. DeepSeek's own DeepInfer can hit $38/GPU-hr during off-peak hours. Throughput and interaction curves differ across chips and models, and there is still arbitrage between high rent and token prices.

The chart lays out the problem. The x-axis is utilization, and the y-axis is gross margin. A footnote says rental prices come from InferenceX: Vera Rubin at $8.50/GPU-hr and GB300 at $5.00/GPU-hr. There is a reference line for “Fireworks reported ~50%.” One reply pointed out that GLM 5.3 running on GB300 at 200 tok/s needs about 90% utilization to reach the Fireworks line. Utilization is a hard constraint. An eight-GPU node costs $64 per hour, and you pay whether requests come in or not.

On the demand side, gateways such as OpenRouter and Vercel help aggregate demand. Inference providers are also willing to offer discounts in exchange for long-term commitments. Deedy also mentions a finer detail: some Chinese open-source model labs ask for revenue sharing once a provider reaches a certain scale, especially for day-one support of pre-release models. The barrier to entry is also falling. Small teams can use a vibe-coding approach to get open-source models running on rented clusters. The result is dozens of new providers popping up every week, revenue ramping fast, gross margins staying thin, and then using that revenue to raise funding.

It looks like a cloud resale business: buy GPU-hours, sell tokens. Neoclouds want to move into inference, while inference providers want to become Neoclouds and offer higher-margin reserved instances and post-training. One reply joked that the most profitable job is still being the middleman. Another reply said that once everyone has read this tweet, the good times may not last.

Ibrahim Ahmed, co-founder and CTO of inference_net, added two points: low-cost compute and the engineering ability to push token efficiency extremely high. He acknowledged that the long-term landscape is unclear, but that there is room to compete in the short term. The risks are also on the table: whether open-source models will remain competitive, whether model owners will take a larger share of revenue, whether changes in model form factor will rewrite compute demand, how chip changes will affect the economics, and who will get compute when supply is tight.

Many inference providers are making money right now because the pie is growing fast. But revenue is not a moat. Rent is a fixed expense, price wars will push token prices down, and the larger you get, the more utilization and the engineering stack matter. How fast the middlemen can run depends on how steadily the x-axis moves to the right.