Wink Pings

Among Five Ontology Projects, Only One Actually Ran Ablation Experiments

Jiuyuanke listed five Data Agent ontology projects; EvoOntology is the only one that did Bench and ablation comparisons. The comments focus more on who maintains the evaluation set and what problem ontology self-evolution actually solves.

Data Agent in China is often called intelligent data querying or data agent. Jiuyuanke listed five ontology-related projects on X and gave an implementation sequence: first build an evaluation set, then automatically construct the ontology structure, manually review and optimize it, and finally automatically evolve based on bad cases.

image

Image source: Jiuyuanke's X post

According to Jiuyuanke's summary, the five projects are:

  • microsoft/Ontology-Playground: A purely static visual ontology exploration, design, and teaching sandbox for Microsoft Fabric IQ.
  • fuyuxiang/ontology, Yuanshu Ontology: An enterprise-grade, runnable full-stack ontology platform modeled on Palantir Foundry/AIP concepts.
  • NVIDIA/auto-ontology: Open-sourced by NVIDIA, #1 on the BEAVER leaderboard and #2 on BIRD-Interact, an enterprise-grade automated data ontology compilation and high-reliability Text-to-SQL platform.
  • gogoymh/auto-ontology-mcp: Derived from NVIDIA/auto-ontology, a lightweight MCP semantic layer and Obsidian structured knowledge base server with no model hosting (BYO-Client-LLM).
  • ruc-datalab/EvoOntology: Open-sourced by Renmin University of China Datalab, accompanied by the paper EvoOntology-2609.15779.pdf, a trajectory-driven self-evolving ontology layer for heterogeneous data Agents.

Jiuyuanke's assessment of EvoOntology is direct: it is the only project that has done Bench and ablation comparisons and implemented ontology evolution, using an Agent to read traces and iteratively update the ontology structure. If you want to publish a paper, experiments are unavoidable. In the comments, KAI XU also mentioned that among the five, only it did ablation comparisons, so when choosing, at least you know which part actually works.

Ontology here can be understood as a business semantic layer that tells the Agent how tables, metrics, and definitions connect. Jiuyuanke's implementation sequence is: build an evaluation set, automatically construct the ontology structure, manually review and optimize, and automatically evolve based on bad cases.

The comments really converged on two questions. Shengxiao bai said that the biggest fear for a Data Agent is that it can do everything in a demo, but after launch no one can clearly explain where it went wrong; having an evaluation set first is the right path. Bai Jiahei added that if an evaluation set is not built, then no matter how the ontology is changed, you won't know whether it is getting better or worse. Chen Xiaoyi and Huishu both asked how to start the evaluation set and who should maintain it.

Another question came from Youfu Mao Dada: the goal of Ontology is to integrate with the target system so the Agent knows what it can do at a given position. If the backend does not change, ontology self-evolution seems meaningless. This question is not trivial. In a reply, Jiuyuanke distinguished it from GraphRAG: it is different from GraphRAG; ontology here only provides semantic-layer prompt injection, and ultimately it still generates SQL.

Bots mixed into the comments, but several follow-up questions were real. When selecting, first look at the evaluation set, then ablation, and only then discuss self-evolution.

发布时间: 2026-11-17 22:32