H-Net Wants to Ditch the Tokenizer: The Compute Savings Are Real, and So Is the Controversy

Cartesia Research has released a set of H-Net results with a very direct goal: remove the last hand-designed component in language models, the tokenizer. In a tweet, Albert Gu framed this as a step toward "learning from raw data," citing Sutton's bitter lesson. The claim is grand, but the data is fairly detailed.

Background first. Today's mainstream LLMs do not read bytes directly; BPE first compresses text into tokens, with a fixed vocabulary, and pre-tokenization still relies on regex. BPE plus pre-tokenization effectively hard-codes judgments such as "what counts as a word, how numbers are split, and how punctuation is bound." Cartesia's H-Net does dynamic chunking: the model learns boundaries during training and compresses bytes into semantic chunks level by level. A 1-stage H-Net can be understood as replacing the tokenizer with a learnable encoder and decoder; the 2-stage version adds one more layer.

They first ran controlled comparisons. Only the tokenization part was replaced, the backbone and hyperparameters were aligned, and compression was also aligned to BPE's 4.57 bytes/token. On validation, 1-stage H-Net used 1.28x less compute to reach the same loss; Paloma averaged 1.83x, and OLMES averaged 2.06x. Code is even more striking: a median of 2.04x across 15 in-training-set languages, 2.40x on Go; a median of 3.10x across 10 out-of-training-set languages, and up to 5.37x on Prolog. This direction is noteworthy: the harder the distribution is to tokenize and the less it follows fixed patterns, the bigger H-Net's advantage.

The larger experiment used a 2-stage H-Net with 3.67B parameters, aligned with SmolLM3-3B in FLOPs per byte. The blog puts it clearly: H-Net reaches, with about 1T tokens, the final average score that SmolLM3 reaches after about 11T tokens. The tweet summarizes this as roughly 10x data and compute efficiency. Keep one caveat in mind: this is a pretraining-evaluation result, not "10x across all tasks." But the accounting is already concrete enough.

The second blog post is more interesting. It splits the scaling law into two parts: compute-to-loss and loss-to-loss. The former looks at whether pretraining saves compute; the latter looks at whether pretraining loss transfers to downstream tasks. The conclusion is somewhat counterintuitive: BPE is strong on compute-to-loss, but its loss-to-loss is worse than that of pure byte models. BPE-NP, which removes pre-tokenization, has a higher compression rate—6.03x, exceeding BPE's 4.57x—but transfers worse downstream and even requires boundary correction for prompt boundaries. Adding pre-tokenization back improves BPE on both fronts, showing that the often-overlooked regex step itself carries a strong prior.

H-Net gets the best of both: it preserves compression efficiency on compute-to-loss and matches byte-level models on loss-to-loss. According to the blog's table, H-Net relative to BPE is 1.28x from compute to validation, 1.61x from validation to OLMES, and 2.06x from compute to OLMES. BLT-like approaches use an entropy proxy to decide chunking; despite the same architecture and compression rate, they still underperform H-Net, which learns boundaries end to end. Cartesia's conclusion is that both boundary quality and end-to-end training matter. For architecture researchers, this conclusion is more worth watching than "several times less compute."

The behavior section also has details. H-Net exhibits in-context tokenization: words it has not seen are first broken apart, then merged after repeated appearances; variable names and repeated formats in code get compressed into larger chunks. In DNA experiments, H-Net shows 5.95x to 7.06x compute gains across three sets and can even hit codon boundaries in the human genome. In vision, chunks learned by 2D H-Net in a pixel diffusion model do not need to be spatially contiguous; it merges background regions occluded by foreground into one chunk. These results show better than the language numbers that "dynamic chunking" is not merely a tokenizer swap.

Community reactions split into two camps. Some shout "free the token"; others ask what actually happened in the paper abstract. hailports's reply is worth quoting: language has already been preprocessed by humans; the real test is data that no one has bothered to name. This challenge is spot-on. Language text is full of spaces, punctuation, and code indentation; these structures themselves are helping the model. In domains such as DNA, audio, and vision, there are no ready-made "words," so H-Net's value has only just begun to be tested.

Jonathan Chang asked in a reply, "what happened here?" The screenshot is the H-Net paper abstract.

In the short term, the tokenizer will not disappear overnight. Existing inference stacks, vocabularies, caches, and toolchains are all built around tokens. What H-Net truly loosens is a default assumption: that the tokenizer must be hand-designed, fixed offline, and shared across the whole model. It turns chunking into a learnable module inside the model and provides reproducible compute accounting on code, OOD code, and DNA. The next thing to watch is post-training and multimodality—that is where "learning from raw data" becomes harder.