Wink Pings

Moving AI Decisions from the Cloud Back to Local: 8.6ms per Item, Weavetab Gate Goes Open Source

Weavetab Gate launches: decision models run on local CPUs at 8.6ms per item, and a single call can process 10,000 inputs. Open-sourced under Apache-2.0 and released simultaneously on npm, PyPI, and crates.io. The benchmark against gemini-decision-1 is now available. Developer discussion points in one direction: local and cloud, each handling its own part.

Weavetab Gate has launched. It runs decision tasks such as Choice, Yes/No, and Rubric on local CPUs—choosing one of several options, making binary classifications, and scoring against criteria. These are lightweight tasks, so there is no need to pay API fees per token.

Official figures: 8.6 milliseconds per item, with a single call processing 10,000 inputs. Apache-2.0, with code simultaneously published to npm, PyPI, and crates.io.

Benchmark comparison with gemini-decision-1 has also been released.

Weavetab Gate benchmark comparison chart

A developer commented that a cloud version is needed so the benchmark test can be fair. Weavetab replied with one word: bet.

In a reply, Igor Ganapolsky of ThumbGate proposed: don't blindly trust the model's directly provided best answer (argmax); using confidence for gating is more valuable—add a sub-millisecond local check (PreToolUse) before tool execution, intercept obvious inputs, and route ambiguous calls to gemini-decision-1.

Weavetab completely agreed, and put it more plainly: edge models should not replace the cloud router; they should protect the cloud router from wasting latency on obvious inputs. The local fast path handles inputs whose confidence exceeds the threshold, and the ambiguous tail is escalated to Gemini. Best of both worlds.

This is probably the right way to put it into practice: routine inputs are absorbed by the local CPU, and only the ambiguous tail needs a large model as a fallback. Latency goes down, and the bill goes down too.

发布时间: 2026-11-22 10:36