91% vs 45%: The gap between enterprise data Agents is hidden in notebook comments

Databricks CEO Ali Ghodsi said in a tweet that he is 10 times better than a year ago, and the whole company runs on Genie. The reason comes down to Agentic Ontology: it continuously organizes and evolves context for the company, making agents higher quality and significantly reducing costs.

This sentence could easily be dismissed as CEO platitudes. The benchmark Databricks subsequently released looks more like evidence. Blog authors Jeff Fletcher and Rishabh Singh wrote a new enterprise data Agent evaluation, in collaboration with Turing, comparing Genie and two frontier Agents in the same live Databricks workspace.

The evaluation used a fictional company, Calderwick Energy, a UK electricity and gas supplier. It has hundreds of tables, the largest with about 120 million rows of half-hourly meter data. On top of the tables are notebooks, scheduled jobs, dashboards, saved SQL, code, metrics, documentation, and Genie Agents left behind by different teams. There is also the mess found in real environments: duplicated notebooks, abandoned analyses, old documentation, development copies, frozen reports, and dashboards untouched for months.

Genie Ontology is the enterprise context layer. It automatically extracts business knowledge from these assets and organizes it into a shared understanding that every agent can use. You can see a demo of this Ontology manager:

The results are straightforward.

Genie had a 91% pass rate, Agent A 45%, and Agent B 58%. Genie used less than half their tokens and was 3 to 4 times faster.

The blog gives a very specific example. The task is to find Calderwick Energy's historical Commercial planning rate. The bill cost stack table has a field cfd_rate_p, and the FY24/25 row says 0.31 pence per kWh. It looks like you can answer just by querying the table.

But the notebook that built this table contains another piece of information: Commercial sets the planning rate every year, and this mart uses FY25/26 assumptions, including applying those assumptions to earlier billing periods. Another section, Planning rates by year, preserves the history: FY24/25 is 0.52 p/kWh, FY25/26 is 0.31 p/kWh, and the FY26/27 draft is 0.45 p/kWh.

The question asks for the planning rate set by the Commercial team in FY24/25. The 0.31 in the table has already been overwritten by the FY25/26 assumption. The table contains only the overwritten number; the real answer is in the comments of the notebook that built the table and in the historical table.

Genie Ontology found the snippet containing the historical rate and linked it to the source notebook 45_mart_bill_cost_stack. After checking the notebook, Genie returned 0.52 p/kWh on its first answer. Across 5 runs, the first answer was always 0.52. The other agents did not get this historical rate: one consistently returned 0.31, and the other returned 0.31 or questioned whether there was any planning rate recorded by year.

The value of this example is not in the number itself. It shows that enterprise data questions often get stuck on business definitions: how revenue is calculated, what counts as an active customer, in which year a given rate was set, and which assumption has overwritten a number in a report. No matter how strong the model is, if it does not read that sentence in the notebook, it will answer incorrectly.

A commenter, John Rood, pointed out that the halved-token number is even more interesting. Ontology is shared meaning compiled once; agents do not have to re-derive what revenue or active customer means in this company on every run. The ambiguity-resolution loop disappears, and cost drops because the work has already happened before the request.

Another commenter, 0xlukewae, said that ontology is the moat, while the model is rented. That judgment is a bit harsh, but the direction is clear: frontier model capabilities are commoditizing, every new model release chases the leaderboard, and the differentiation of enterprise agents increasingly comes down to the harness and the context it can retrieve.

Governance questions follow as well. A commenter asked: who clicked Promote on these snippets, a human or Genie itself? Another commenter, Max Petrusenko, said that the quality of an ontology depends on its deletion queue, and stale entries cost more than missing entries. Neither of these problems can be solved by model parameters.

The blog also notes limitations: this benchmark tests an environment that has already accumulated a large amount of workspace knowledge in a Databricks deployment. In new deployments that have not yet built up knowledge, or when questions can be answered directly from tables and schemas, the gap between agents will be smaller. This also explains why some data agent benchmarks that look only at tables show little difference.

Most of Genie Ontology's work happens before the question is asked.