Who Is Your AI Actually Saving Money For?

Mozilla CTO Raffi Krikorian does not want to see seven AGIs. He wants seven billion of them.

The phrase comes from his conversation with Tim O'Reilly about open-source AI. In the conversation, there is a more concrete question: who does your AI work for—you, or the business model behind it?

Browsers were called user agents in the early days for a reason. The browser stands on the user's side, rendering pages, managing privacy and cookies, and trying as much as possible to block people who want to take advantage of the user. Raffi believes AI needs the same loyalty.

First, an experiment

Researchers tested whether AI assistants can be trained to favor a company's commercial interests. In simulated booking requests, 18 of 23 models recommended the more expensive, sponsored flight more than half the time, passing over cheaper options. The paper is here, and the original conversation is at O'Reilly. The experiment itself points to an awkward fact: an agent's loyalty can be priced.

The article compresses this distinction into one line: privacy protects secrets, ownership protects judgment. Data can remain private while being used to steer you.

The harness is the entry point

Raffi's argument is bigger than open weights. An agent also needs memory, tools, a runtime, and rules for dealing with other agents. Together, these things are the harness. It determines what the AI can see, what it remembers, and what it executes.

Owning the weights is not enough. If someone else controls the harness, they can still control the critical layer outside the model. Open memory protocols are therefore important. Memory must be portable, inspectable, and erasable. You should be able to move your history to another model, inspect what the agent knows, and then delete it. When the connection is severed, the door is closed; the notes should not stay in the room.

One commenter in the replies broke the problem down more directly: if 18/23 models chose sponsored flights, the loyalty problem lies in the harness, not the weights. Ethan's response was that both matter: the harness sets incentives, and the weights shape responses. Another commenter added that an agent working for the user must be able to say no to sponsors.

Open models won usage, not revenue

Mozilla's 2026 open-source AI report says open models are about 3% behind the frontier, but deployment is still harder. Hosting, guardrails, and memory all have to be handled yourself. Open source is easy to release and hard to run. You don't need a Ferrari to manage a calendar; what's missing is an installer.

Usage and revenue are also asymmetric. Open models support about one-third of usage but capture only 4% of model-layer revenue. Tim O'Reilly explained this with Steve Baer's Clothesline Paradox: sunlight dries clothes without entering the energy ledger. Open source often wins by becoming infrastructure; invoices are not the main way it does so.

This judgment is crucial. What is really worth watching is the layer of control outside the model; model leaderboards can explain only part of it.

Treat AI as infrastructure

Raffi says treating AI as a product is a category error. When an AI company fails, the product disappears. When a company changes its business model, features you depend on can be pulled away. Roads outlive contractors. Nation-states are legislating where data is processed and how it can enter and leave, and the monetization models of large labs may not be compatible. What they need is infrastructure.

The article mentions that in June, Anthropic temporarily cut off access to the Fable model to comply with export controls. Leading open models mostly come from China, and leading closed models mostly come from the United States. Both sides can be cut off by executive order. Diversity is resilience. Hospitals should download local models as backups; critical systems will not rely on only one intelligent vendor.

Open and closed can coexist. Closed products are simple; open systems provide sovereignty and room for experimentation. Raffi says a well-tuned harness can improve performance by 20% on some coding tasks. You can make a difference outside the model without training a frontier model.

Owners and renters

Being an owner is not just downloading weights. It also means controlling the things around the model: memory, permissions, the ability to leave, and the incentives that shape every answer. A personal agent should not only understand you; it must also be on your side.

The original conversation is at O'Reilly, and the discussion thread is on X.