27B Distilled into 5B: Perplexity Decision Model Now Runs for Free on Mac

Perplexity has open-sourced a new decision model, pplx-decider-flash-v1.2. It has 5B parameters, can be downloaded for free, runs locally, requires no API key, and has no bill. Shubham Saboo demonstrated running this model on a Mac on X, and someone in the comments remarked, “I just got schooled by a MacBook.”

This model is the small-size version of Perplexity’s decision model series. The original pplx-decider-v1.2 has 27B parameters, while the flash version is built on Qwen3.5-4B and extracts knowledge from the 27B model through model distillation. It is about one-seventh the size of the original, with 5B parameters, under the Apache 2.0 license. Distillation not only preserves most of the capability; on some held-out tasks in the model card’s tests, it is even more stable than the 27B.

What can it do? Simply put, given a piece of context, the model can answer a set of structured questions. For example, from a problem description, it can determine which team it should be assigned to, whether the customer is blocked, their emotional state, and so on. It supports text and image inputs, and the request format is concise. The model card provides example code:

import sys
from pathlib import Path
from huggingface_hub import snapshot_download

checkpoint = Path(snapshot_download("perplexity-ai/pplx-decider-flash-v1.2"))
sys.path.insert(0, str(checkpoint / "source" / "src"))
from pplx_decider import Decider

decider = Decider.from_pretrained(checkpoint, device="cuda")
response = decider.decide({
    "state": "My integration keeps failing with a 401 since we rotated our API key. Please help.",
    "questions": {
        "team": {
            "type": "choice",
            "instructions": "Which team should handle this request?",
            "criteria": {
                "billing": "Charges and refunds",
                "support": "Technical integration errors",
                "sales": "Questions about buying a product",
            },
        },
        "urgent": {"type": "noul", "instructions": "Is the customer blocked right now?"},
        "frustration": {"type": "score", "criteria": ["calm", "mildly annoyed", "frustrated", "angry"]},
    },
})
print(response["answers"]["team"]["choice"], response["answers"]["urgent"]["noul"])

In terms of performance, the model card provides evaluation data: on the Decision Index 0.3 public dataset, the flash version’s single-question mode scores 60.05 overall, while the 27B version’s joint request scores 68.09; the visual decision metric is 72.53 vs. 77.43. Although overall it is somewhat lower, what it gains is being seven times smaller, making it suitable for local execution. The model card specifically notes that for the flash version, on most multi-question tasks, “one question per request” is more stable; grouping is only helpful in specific scenarios (such as “select all”–type tasks).

The community’s reaction is also interesting. Some say “The cloud bill was evicted, and AI stayed,” some say “No subscription, no meter—kind of liberating,” and others point out “The bill just changed from API quota to RAM.” Some also ask “What do you use it for?” and “Compared with a base model with a system prompt, what exactly does it win at?” This is indeed the key question for decision models: if adding a prompt can achieve a similar effect, then it is not worth deploying separately. However, according to Perplexity’s evaluation, the distilled model still clearly outperforms ordinary base models on decision tasks.

The model is now available for free download on Hugging Face, with about 1,929 downloads in the past month. For developers who want to move decision/classification logic from the cloud to local, this size and license are quite friendly. The only thing to note is that the model card states that a CUDA GPU is required, but it has already been run successfully on a developer’s Mac. If you try it too, you can share your actual results.