DNA Gets Stuffed Into the Chat Box: A 14B Model Cuts Per-Question Cost to About 1/1800

Hugging Face Bio has released the FineDNA dataset and the Carbon-X model. The dataset's ambition is stated in the tweet: teach LLMs to understand life. The model's goal is more specific: let language models read DNA directly, then answer in English whether a variant is harmful, what a sequence regulates, and what the next base is.

FineDNA has three stages. Mid-training uses 300B+ tokens, mixing DNA sequences, papers, and web pages so that the abstract DNA representations of genomic models connect with natural-language semantics. SFT has 200,000 examples covering DNA computation, sequence analysis, and variant effect prediction, using real DNA sequences and deterministic verifiers to prevent hallucinations. RLVR has 100,000 calibration samples, specifically targeting hard-to-learn abilities such as locating a DNA segment within a long sequence. Lewis Tunstall says models trained with FineDNA can chat with DNA and come close to frontier models on DNA understanding tasks.

Carbon-X is the product of this dataset. The model card lists 14.8B parameters, starting from Qwen3-14B; DNA and text share a single token stream, with no separate DNA encoder. The tokenizer is hybrid: text uses BPE, DNA is split into 6-mers, with one token covering 6 bases, saving about 3x tokens versus BPE. Context is 40,960 tokens. Training first freezes the main body to learn new DNA embeddings, then mid-trains on 302B tokens, followed by SFT and RL. RL uses asynchronous GRPO, and rewards come from task-specific code-checking verifiers.

Average across five benchmarks is 66.7%; GPT-6.1 Sol is 67.5%; per-question inference cost is about 1/1800. Kimi K3 improves by 11.1 points after consulting Carbon-X. The model card gives a more concrete accounting: to analyze 2.4 million variants of uncertain significance in ClinVar, Carbon-X costs about $300, while frontier models cost about $32,000 to $4 million. The weights can run in bf16 on a single 80GB GPU, under Apache 2.0, and GGUF is also available for local llama.cpp.

The limitations are stated plainly: there is no clinical validation, and it cannot be used for medical decisions. Precise sequence operations miscount bases, get coordinates wrong, and make editing errors; localization within long sequences is especially weak. When context is missing, it fabricates sequence identity or function. On GenomeQA, splice sites are 29.8% and histone marks 34.6%. Quality degrades for long DNA generation; the model is trained to read and interpret existing sequences, not to design new ones. Thinking mode is supported, but it is turned off in both post-training and evaluation.

There is a key question in the replies. Hanmin asked how much of the gain comes from 300B-token mid-training versus RLVR, and whether there were ablations. No answer is visible in the public replies. Matt asked whether the team had a preferred open-source model for this kind of data and ML training pipeline; Lewis recommended gpt-6.1-sol, on the grounds that it is least likely to refuse legitimate requests. Asking about open-source models and getting a closed-source model is a somewhat funny detail, but it also shows that general assistants' safety guardrails can interrupt workflows in real work. Aleksandar Janca's comment is closer to a product judgment: small expert models being called by large models is what most business tasks should grow into. basedcapital also mentioned that putting DNA and text in the same loss is what lets Kimi use it as a sub-agent rather than merely as a lookup table.

DNA is one of the few domains where rewards are genuinely checkable. Whether a variant is harmful, whether a base is predicted correctly, and whether a position in a sequence is found correctly can all be verified with code. RLVR is more substantive here than in open-domain chat. Another signal is the cost curve. On the leaderboard, 66.7 and 67.5 differ by only 0.8 points, while cost differs by three orders of magnitude; it goes without saying which way engineering choices will lean.

Dataset: https://huggingface.co/datasets/HuggingFaceBio/FineDNA
Code: https://github.com/huggingface/carbon
Model: https://huggingface.co/HuggingFaceBio/Carbon-X