One forward pass, zero output tokens: this 31B decision model tops the Decision Index
Karpo Decision 31B, an open-source decision model based on Gemma 4, ranks first on the Decision Index. It does not generate text; instead, it directly returns the probability of each option, with a median latency of 94 ms on a single GPU.
Karpo Decision 31B has been released. 31B parameters, based on Gemma 4, ready out of the box. It scored 73.9 on the Decision Index, ranking first, well ahead of the second-place Jev (60.2 points). It is twice as fast and 20% cheaper.
This model does something different from ordinary LLMs. Given a state (webpage, document, conversation) and a few multiple-choice questions, it directly returns the probability of each option. It does not generate tokens or decode text; it produces results in a single forward pass.

As shown, on the Decision Index 0.4.1 leaderboard dated 2026-11-12T00:00:00.000Z, karpo-decision-31B ranks first with 73.9 points.
What is a decision model
Traditional LLMs are generative: you ask them a question, they produce one word at a time, and then you parse the answer from the text. Decision models do not do that. Each option is encoded as a code, and the model reads a linear-head score at a specified position; after softmax, it directly gives probabilities.
The benefits are:
- The answer is always one of the valid options, so there are no format errors
- Every answer comes with a confidence score, so you can set thresholds for automatic handling
- It is fast, because there is no autoregressive decoding
For example, if you ask "Can this order be returned?", the model returns P(yes)=0.82, P(no)=0.18. You can automatically approve anything above 0.8 and send the rest to a human.
Performance data
Karpo Decision 31B performs on several benchmarks:
| Metric | Value |
|---|---|
| Decision Index Full | 73.9 (#1) |
| JevBench Capability | 87.2 (#3) |
| Median latency (single GPU) | 94ms |
| Calibration error | 4.8 points |
It supports three types of questions: true/false questions (noul), multiple-choice questions (choice, up to 255 options), and scoring questions (score). Input supports text and images, with a 32k-token context.
Video demo:
Model family
In addition to 31B, there are five sizes: 0.8B, 2B, 4B, 9B, and 12B, all open source. Small models are suitable for resource-constrained scenarios, while 31B is suitable for high-precision scenarios.

How to use
Installation is simple:
pip install "karpo-decision @ git+https://github.com/machinepulse-ai/karpo-decision"
Load the model:
from huggingface_hub import snapshot_download
from karpo_decision.decider import Decider
path = snapshot_download("MachinePulseAI/karpo-decision-31B")
decider = Decider.from_pretrained(path, device="cuda")
response = decider.decide({
"state": "Order 1182: the customer asks to return a blender bought 45 days ago. Policy: returns within 30 days.",
"questions": {
"eligible": {"type": "noul", "instructions": "Is the return eligible under the policy?"},
"action": {"type": "choice", "instructions": "What should the agent do?",
"criteria": {"refund": "Issue a full refund", "decline": "Politely decline the return",
"escalate": "Escalate to a supervisor"}},
},
})
print(response["answers"])
The weights are on Hugging Face under the Apache 2.0 license, and you can host them yourself. A single RTX PRO 6000 can run it, with peak memory around 60GB.
What it's good for
The officially recommended scenarios: search, recommendation, tool selection, model routing, and browser operations. These tasks are essentially "pick one from a set of options"; a decision model directly gives probabilities, which is much faster than asking an LLM to write a JSON snippet.
One commenter said model routing is the first scenario to try: "A single forward pass gives the probability of each option, and the cost of routing is lower than the routed call itself." Another said that many simple classification tasks previously done with an LLM plus structured output can be migrated to this model.
There are also doubts: the leaderboard was run by the model authors themselves and needs third-party reproduction. That is normal; wait and see.
Training details are also worth mentioning. They used an RSI (recursive self-improvement) loop, training multiple candidates each round, scoring them, and merging weights, replacing the model only when the weighted score was higher. The final released version is the average of 5 checkpoints; the public score did not change, but hidden-domain accuracy increased by 1.6 points.
In a nutshell
If you are building an AI agent and need fast, reliable judgments with confidence scores, this model is worth trying.
Links:
References: Santiago's tweet, LeahW's tweet, Karpo official blog.
发布时间: 2026-11-21 04:47