AI ROI Can't Be Calculated Yet—First Separate Learning Costs from Savings

Alex Lieberman says in an X post that he has spoken with more than 100 executives about AI ROI, and no one can give a reliable method. Companies follow a similar path: first open general LLM accounts, sign minimum-token agreements, run training, then give product and engineering coding agents; the token bill spikes, and ROI discussions begin. Then they discover that data migration, cleaning, and an agent gateway all need to be added, spending grows, and the question becomes how to calculate ROI on the data foundation. Months pass, money is spent, bets are made, and the ROI method still has not been written.

ROI has never been an exact science. It is easier to calculate for sales roles and harder for creative roles; software and services are the same. AI makes this more glaring because spending can scale up quickly and impact arrives quickly too. The CFO sees an ever-lengthening bill; the team sees a pile of experiments that have not yet grown into systems. Neither side is wrong.

Alex offers a framework called 1-3-1. One foundation, three budgets, one rule.

One foundation is shared infrastructure. Every AI project runs on it; it is the operating cost of deploying AI safely and reliably. It needs to be funded upfront. In regulated industries such as insurance, the foundation budget can account for 8% to 12% of the AI budget.

The three budgets are experimentation, efficiency, and innovation. Experimentation is like R&D: set an amount you can afford to lose entirely, based on how many experiments you want to run. Efficiency and innovation must earn their own funding; projects trade evidence for money.

One rule: do not treat projected value as realized returns. A project saying it can save $1 million does not mean it has already saved $1 million.

For the company-wide AI spending cap, Alex recommends that early-stage companies start at 0.5% to 2% of annual revenue. A BCG survey in September 2026 of 1,330 executives found that enterprises on average invest 3.3% of revenue in AI, up from 1.7% a year earlier. This recommended cap is below the market average; the intent is to make projects compete for budget with evidence.

The article uses a fictional insurance company, Meridian, as an example. It has annual revenue of $500 million and an AI spending cap of $5 million, or 1% of revenue. Of that, $400,000 goes to the foundation, or 8% of the AI budget; $700,000 goes to experimentation; up to $3.9 million is left for efficiency and innovation, and projects must prove themselves.

How are experiments judged? Meridian's claims team first tries document summarization. One claims adjuster finds that for an 80-page claims file, AI can summarize it in 20 minutes, while doing it manually takes 90 minutes. One file does not prove anything, but it can be written down as a hypothesis. The team spends $25,000 to run a pilot on 500 claims cases. The pass criteria are fixed in advance: at least 90% of summaries pass quality checks, and after adjuster review, processing time still drops by about 40%. The result: 92% of simple files pass, but only 78% of complex files pass. Routine files make up about 70%; moving to a full business case would release about $1.1 million in claims capacity per year, with $300,000 to deploy and $180,000 in annual running costs. Complex files go back to experimentation, with a different retrieval approach or a new model to try again. Every pilot must end with scale, change, or stop; it cannot stay in pilot purgatory forever.

How is efficiency judged? Alex quotes Tenex co-founder Arman: gross productivity gain is not realized savings. Saved time does not automatically turn into money. There are four ways to convert it: avoid hiring, increase throughput, shift people to more productive work, or directly reduce costs. Meridian commits to absorbing the six claims-adjuster roles it would otherwise have hired, with about $540,000 in annual hiring cost. Two quarters after deployment, the workload is handled and the hiring plan is not executed. After subtracting $300,000 to build the agent and $180,000 in annual running costs, first-year net savings are $60,000, and net savings are $360,000 per year thereafter. The released time then goes to complex claims; additional upside must be proven by outcomes such as faster closing and fewer excess payouts, and cannot be double-counted.

How is innovation judged? Meridian's underwriting team uses AI to cut quote time from three days to four hours. The article assumes the bind rate rises from 22% to 26%. With 50,000 quotes and an average annual premium of $4,000, that looks like about $8 million in additional premium. But faster quotes do not necessarily produce more profitable business. Meridian gives the tool to half the underwriting team and uses the other half as a control, comparing bind rates between the two groups. Revenue-type projects take two to four quarters to show results; start with quote turnaround and early bind rate as leading indicators.

How is the foundation judged? The foundation does not directly claim the returns of any specific agent. It claims enablement: are new use cases launched faster, are costs lower, are security reviews faster, is the team actually using it. Meridian's first agent deployment took nine months; the second took six weeks, build cost was about one-third, and security review shrank from three months to two weeks. The article notes that if an agent saves $1 million, that $1 million is credited to the agent; the foundation is credited with making subsequent deployments faster and cheaper. Early on, about 20% of realized gains can be reinvested in the foundation.

In another article cited in the discussion, ::_rajeev proposes two ROIs. The first stage is called Re-learning or Re-building on Investment; only the second stage is Return on Investment. When early teams try AI in real workflows, the most valuable output is organizational learning: what is learned technically, what is learned about the business, how the way of working changes, and which ideas should be quickly discarded. This stage has costs—tokens, time, attention, failed experiments—most of which are not in the spreadsheet.

::_rajeev draws an AI Valley of Death. In the exploration phase, token spending rises while value lags; at the bottom, spending may be cut out of fear and projects die; projects that survive move into build and launch, token usage stabilizes, and value compounds with adoption and reuse. Early token spending is more like a learning cost; do not rush to treat it as waste, but of course do not turn usage of cat-meme calls into a leaderboard either. Teams should kill projects themselves—stop them quickly, honestly, and with a record—and that is itself the return of the first stage. Traditional ROI only deserves to be applied after systems stabilize; first treat it as something to be earned. The article quotes Soichiro Honda saying success is 99% failure and interprets it as a discipline of learning.

The comments are more down-to-earth than the framework. One commenter says much AI ROI is actually calculated as reduced full-time equivalents (FTEs), but no one publicly writes that goal down. Someone says revenue attribution is the key and that you must separate the increment brought by AI. Someone used Claude solo for nine months and reached $25,000 per month without hiring; small teams are easy to calculate, large organizations are hard. Someone asks whether the problem is that no one measures the before. Someone sees ROI as time arbitrage: use the saved hours for higher-leverage work and subtract AI costs from the difference. Someone only trusts the hours saved in one specific workflow; rolling Claude out to everyone actually yields no data. Someone says budgets are based on faith; if the model is not tied to saved costs or signed deals, the discussion will always feel like a debate. Someone reminds us that cost is the only number that can be measured precisely: aggregate it by vendor invoice and tag it by budget. Someone says no one actually books fewer hours for hours saved, so the payback period stays blank. Someone says data cleaning is often left out and ends up being the biggest item. In government contexts, someone mentions cost per claim, cost per FOIA request, and cost per unit of work. Someone else records minutes saved per task multiplied by weekly run count, estimates first, runs for a week, and then reconciles costs.

These disagreements point to the same operational step: company-level AI ROI can be answered later, but specific workflows cannot stay blank forever. First write down clearly what is being funded—foundation, experimentation, efficiency, or innovation; then define the business case and the KPIs the function already has; when approving money, write down the goal, and do not add it after results come in; review once a quarter which items get more money, which get tested again, and which stop. A $20,000 tool and a $2 million rollout should not have the same evidence threshold. A successful pilot can get more funding before achieving returns, but that depends on risk appetite, opportunity size, and the cost of waiting.

If you have to submit an AI ROI spreadsheet tomorrow, do not start with a company-wide master sheet. Pick one workflow, write down the before, aggregate costs by invoice, and set a scale/change/stop decision for one quarter later. It is not shameful to be unable to calculate the whole company; it is shameful to keep filling budgets with faith.