Handing Decision Models a Multiple-Choice Exam: Switch Aims to Sidestep the Tool-Calling Pitfall

Switch is a harness framework for decision models. The author, Onur Solmaz, released the repository together with an explainer video, with a clear goal: to let models like Jev, which only output probabilistic answers, also enter the agent loop.

Decision models are different from chat models. They answer questions such as bool, choice, and score, give probabilities, and do not write natural language. Coding agents rely on models to generate tool calls, which Jev cannot do. Switch moves this step into code: the domain defines states, the question function derives finite options from the state, the model chooses one, and the transition function takes the answer to generate the next state.

This loop compresses open-ended generation into discrete routing. The options are code, not prose.

Among the examples provided by the author, the grading judge is the most practical. The traditional approach extracts PDF, docx, and md into text and stuffs it into the context; when visual formatting is also a scoring item, each page also needs to be converted into an image. For a 200-page document, the questions only involve a few pages, so most of the context is wasted. Switch's rubric judge lets the model read files step by step, look at page images, and search keywords until it can score item by item.

A netizen said that letting the judge flip through 200 pages itself, instead of pouring every page image into the context, convinced him. Evaluation costs are rising fast, and this way of saving is straightforward.

The repository also includes chess, WikiRace, and ticket triage examples. In the triage example, the model each time chooses which log to read, chooses stop after reading, and finally chooses a team. The domain is responsible for writing the state, question, and transition clearly; the model only makes choices.

Installation requires Node.js 22.18 or newer:

npm install @osolmaz/switch

Switch is not tied to Jev. Any decision model that pi-ai lists as a classifier can be connected. After a run, a trace is left; replay does not call the model, and you can also fork the first n steps to let the model continue.

The limitations are also clear. The author himself says Jev is very weak at chess, and as a judge it still cannot replace GPT 6.1 Sol. A netizen posted numbers from the TypeSafe dashboard: Jev's agreement rate with the reference is 67.8%, while Opus 5 in workflow mode is 74.1%. This number is suitable as a bootstrap, not for directly replacing production judges.

What is more worth looking at is option generation. A netizen asked which collapses first: a large option set or a messy state; others pointed out that the generated menu still misses legal moves. Open-ended generation becomes finite selection, making verification easier, but where the options come from and how to keep them small is still domain engineering.

If the domain is narrow enough, this approach has a cost advantage. First use a strong model to generate data, then fine-tune an open-source decision model in a harness like Switch; for a single task, it may be much cheaper than a general model. The premise is that the state, question, and transition are all written correctly. What Switch standardizes is exactly this layer of dirty work.

Repository: https://github.com/osolmaz/switch

Pi: https://pi.dev

Jev and System One model documentation: https://docs.typesafe.ai/concepts/system-one