One Table, Three Parse Results: DPP Wrote 140 Rules to Judge Them

The same sales table has a two-year header and quarterly subheaders. Three parsers output three different Markdown versions. Ask an agent, “What was system same-store sales in Q2 2024?” A stuffs the quarter into the body text, and the 4.1% has no year above it; C drops the year, and Q2 maps to two columns; B gets the result right but invents a “2024 Q2” label that is not printed on the page. LlamaIndex judges all three as wrong. The reason is simple: if the structure is wrong, correct numbers are useless.

On October 28, LlamaIndex’s Jerry Liu released Document Parsing Protocol working draft 0.1. It contains 140 rules covering text, tables, forms, charts, diagrams, and even chemical formulas and musical scores. Each rule comes with an example page and expected output. It comes from the rule set used when annotating ParseBench. DPP has not yet become a full evaluation benchmark; it is currently an evaluation guide for judging the quality of LLM document parsing, and it also invites discussion about what counts as the “best” parsing approach.

DPP’s answers are very specific. A Markdown table cannot express spanning headers, so DPP uses HTML tables to preserve colspan: under 2024, Q1, Q2, and Q3, with 4.1% under Q2. Empty form fields are written as [blank], and checkboxes as [x] and [ ]. A form region is a key-value list; it must not become a two-column table just because labels and values line up. Only a boxed grid printed on the page counts as a table. This point is critical: many parsers turn form fields into tables, inventing structures the page does not have. Charts become numeric tables, one row per category and one column per series, with printed values copied as-is. Diagrams are converted to Mermaid when possible; otherwise they are described with all labels preserved. Hidden text in a PDF is dropped, while visible but unreadable text is written as [illegible]. Position information is optional; a sidecar gives each element’s position on the page for traceability.

Under the tweet, someone asked who arbitrates when two of the 140 rules conflict; someone asked how to deterministically score a parser; someone suggested annotating in the output which rule was used, because downstream agents most fear two parsers that both comply but produce different structures. Another comment noted that second-row headers often hold units, and after flattening an agent gets a column of numbers without knowing whether they are euros or hours. DPP does not fully answer these questions. Working draft 0.1 is open on GitHub for people to file issues and PRs. DPP also defines acceptable differences, such as whether thead and tbody are wrapped, as long as they are consistent throughout. But “who arbitrates” remains a governance problem, and writing 140 rules does not automatically solve it.

parseprotocol.com has guides and exercises. For Claude, download the skill zip and upload it to Customize → Skills; code execution is required. For ChatGPT, add the reference file. Attach the page image and paste the site prompt to have it parse according to DPP. The Apple FY2024 sample page lets you check whether both 6485s are preserved and whether the parentheses around (565) remain. This sample includes a walkthrough of eight parsing decisions.

The most valuable part of DPP is not the 140 rules themselves. It turns “parsing correctly” from each vendor’s private taste into publicly debatable text. Before, two teams could take the same output and give it different scores; now they can at least argue while pointing to the same rule. For people building agents, the more practical next question is: when two rules both comply but the output structures differ, how does the downstream system know which one to trust? Tagging outputs with rule IDs may be one approach; someone in the comments has already raised it. Whether DPP becomes a standard depends on whether it can absorb these conflicts. The number of rules is only surface-level.