Wink Pings

One forward pass, zero output tokens: this 31B decision model tops the Decision Index

Karpo Decision 31B, an open-source decision model based on Gemma 4, ranks first on the Decision Index. It does not generate text; instead, it directly returns the probability of each option, with a median latency of 94 ms on a single GPU.

Karpo Decision 31B has been released. 31B parameters, based on Gemma 4, ready out of the box. It scored 73.9 on the Decision Index, ranking first, well ahead of the second-place Jev (60.2 points). It is twice as fast and 20% cheaper.

This model does something different from ordinary LLMs. Given a state (webpage, document, conversation) and a few multiple-choice questions, it directly returns the probability of each option. It does not generate tokens or decode text; it produces results in a single forward pass.

leaderboard

As shown, on the Decision Index 0.4.1 leaderboard dated 2026-11-12T00:00:00.000Z, karpo-decision-31B ranks first with 73.9 points.

What is a decision model

Traditional LLMs are generative: you ask them a question, they produce one word at a time, and then you parse the answer from the text. Decision models do not do that. Each option is encoded as a code, and the model reads a linear-head score at a specified position; after softmax, it directly gives probabilities.

The benefits are:

  • The answer is always one of the valid options, so there are no format errors
  • Every answer comes with a confidence score, so you can set thresholds for automatic handling
  • It is fast, because there is no autoregressive decoding

For example, if you ask "Can this order be returned?", the model returns P(yes)=0.82, P(no)=0.18. You can automatically approve anything above 0.8 and send the rest to a human.

Performance data

Karpo Decision 31B performs on several benchmarks:

MetricValue
Decision Index Full73.9 (#1)
JevBench Capability87.2 (#3)
Median latency (single GPU)94ms
Calibration error4.8 points

It supports three types of questions: true/false questions (noul), multiple-choice questions (choice, up to 255 options), and scoring questions (score). Input supports text and images, with a 32k-token context.

Video demo:

Model family

In addition to 31B, there are five sizes: 0.8B, 2B, 4B, 9B, and 12B, all open source. Small models are suitable for resource-constrained scenarios, while 31B is suitable for high-precision scenarios.

family chart

How to use

Installation is simple:

pip install "karpo-decision @ git+https://github.com/machinepulse-ai/karpo-decision"

Load the model:

from huggingface_hub import snapshot_download
from karpo_decision.decider import Decider

path = snapshot_download("MachinePulseAI/karpo-decision-31B")
decider = Decider.from_pretrained(path, device="cuda")

response = decider.decide({
    "state": "Order 1182: the customer asks to return a blender bought 45 days ago. Policy: returns within 30 days.",
    "questions": {
        "eligible": {"type": "noul", "instructions": "Is the return eligible under the policy?"},
        "action": {"type": "choice", "instructions": "What should the agent do?",
                   "criteria": {"refund": "Issue a full refund", "decline": "Politely decline the return",
                                "escalate": "Escalate to a supervisor"}},
    },
})
print(response["answers"])

The weights are on Hugging Face under the Apache 2.0 license, and you can host them yourself. A single RTX PRO 6000 can run it, with peak memory around 60GB.

What it's good for

The officially recommended scenarios: search, recommendation, tool selection, model routing, and browser operations. These tasks are essentially "pick one from a set of options"; a decision model directly gives probabilities, which is much faster than asking an LLM to write a JSON snippet.

One commenter said model routing is the first scenario to try: "A single forward pass gives the probability of each option, and the cost of routing is lower than the routed call itself." Another said that many simple classification tasks previously done with an LLM plus structured output can be migrated to this model.

There are also doubts: the leaderboard was run by the model authors themselves and needs third-party reproduction. That is normal; wait and see.

Training details are also worth mentioning. They used an RSI (recursive self-improvement) loop, training multiple candidates each round, scoring them, and merging weights, replacing the model only when the weighted score was higher. The final released version is the average of 5 checkpoints; the public score did not change, but hidden-domain accuracy increased by 1.6 points.

In a nutshell

If you are building an AI agent and need fast, reliable judgments with confidence scores, this model is worth trying.

Links:


References: Santiago's tweet, LeahW's tweet, Karpo official blog.

发布时间: 2026-11-21 04:47