8.8B model reaches 90% of frontier scores, while knowledge questions expose its limits

webAI has released the TwIL-8B weights on Hugging Face. With 8.79B parameters, it reaches near 90% of frontier models on several benchmarks for code, instruction following, and competition math. That number is worth looking at, and so are the limitations that come with it.

Andrej Karpathy gave five judgments in a tweet. One line from it is very direct: a 1B model may be more effective than most current LLMs; 90% of models spend a huge amount of capacity and money memorizing unimportant things. The five points can be condensed as follows.

1. Intelligence has never been just knowledge. Strip away the memorized internet and leave reasoning, and the remaining cognitive core is very small.
2. That core might fit into about 1B parameters, or even fewer. In the past it was like paying for a library, when what was actually needed was a brain.
3. Parameter scale is no longer the bottleneck. Whether an agent can work depends on what tools it can call and how tight its feedback loop is.
4. Intelligence lives in the system, not in the weights. A bare model is like a CPU without an operating system; a small core plus a good harness may outperform a large model on its own.
5. You can own the model yourself. With a model running on your own hardware, no one can throttle it, raise its price, or take it down.

TwIL-8B looks like a downloadable example of this set of judgments. It is based on ibm-granite/granite-4.2-8b, goes through full-parameter SFT, then GRPO/DAPO reinforcement learning, and finally adds a round of length-aware RL, producing the default balanced version and a verbose version that is more willing to reason at length. The context is 131,072 tokens; bf16 weights are 16.38 GiB, Q4_K_M GGUF is about 4.98 GiB, and it can be tried on an ordinary CPU or a GPU with modest VRAM.

On benchmarks, TwIL-8B scores 98.2 on MATH-500, 88.3 on AIME 2025, 81.6 on LiveCodeBench, and 74.4 on IFBench. According to the comparison provided by the publisher, it reaches 93% of frontier models’ LiveCodeBench score, 92% of IFBench, 90% of HMMT Nov 2025, and 88% of AIME 2026 and HMMT Feb 2025. On HMMT Feb 2026 it drops to 69%.

The “90%+” claim corresponds to shares on different benchmarks, not a single overall score. Knowledge questions are the obvious weak spot. GPQA Diamond is 70.2, 11.5 points lower than Qwen3.5-9B and 14.1 points lower than Gemma 4 31B; HLE is 12.6. The model card also states clearly that it is not a knowledge model and has no additional safety, preference, or tool-use training.

The evaluation methodology also needs to be laid out. Scores marked with † were measured on the verbose version and have not yet been run on balanced. Competitor scores come from public model cards or third-party evaluations and were not rerun. An AIME or HMMT set has only 30 to 33 questions, with each question worth about 3 points, so a few points of difference may be within noise. Token cost is also not low: AIME 2026 uses about 19.5k output tokens per question, 38% fewer than Qwen3.5-9B, but still about four times the roughly 5k tokens of GPT-5.5.

In the comments, someone asked how much accuracy Q4_K_M loses compared with bf16, and David Stout replied that the loss with their AIR method is very small; others asked outright what the point is of a small model doing code and math, saying they would rather see browser and computer operation. These reactions are closer to real usage scenarios than the launch copy.

The value of a small model is not in taking over every task a large model does. It is more like a deployable reasoning core that, with tools, memory, and a feedback loop, can handle high-frequency, fixed, cost-sensitive steps. TwIL-8B’s code and instruction-following data give this route a ready-made example, while its knowledge shortfall and non-commercial license also remind us not to treat it as an all-purpose assistant. Parameter worship can cool down, and there is no need to rush to heat up benchmark worship either.

Original link: https://x.com/0xCodez/status/2118424826968748242

Model page: https://huggingface.co/webAI-Official/TwIL-8B