webAI reports 98.2 on MATH-500 for TwIL-8B, and that number will travel faster than the footnotes. It should not. The score is self-reported, from one competition-math exam, on an 8.79-billion-parameter model released under a non-commercial license. It is not an independent result. It makes a poor buying guide.
The useful question is narrower. Hand this model a pricing check or a formula when the facts are already in the prompt. Do not hand it a brief that depends on the account. Route the quantitative step. Keep a model built to know things on everything else.
What actually shipped
TwIL-8B is not a new foundation model. webAI post-trained IBM’s Granite 4.2 8B, an Apache 2.0 base, into an English, text-only reasoning model aimed at contest math, code, and instruction following. The exact count is 8.79 billion parameters. The “8.8B” label is rounding.
It is a dense model. A mixture-of-experts model uses only part of its weights per token. TwIL uses all of them. Context length is 131, 072 tokens, inherited from Granite. Two checkpoints share that design. Balanced is the default, and it is the one trained to keep reasoning shorter. Verbose reasons longer and is aimed at hard contest-style problems. Both write the reasoning out before the answer. You get the scratch work, then the conclusion.
The model card describes three post-training stages. Full-parameter supervised fine-tuning came first, on reasoning traces for math, STEM, and instruction following. Every weight was updated, not a small adapter clipped on the side. A reinforcement-learning stage, labeled GRPO/DAPO on the card, produced the verbose checkpoint. A further length-aware stage, which rewards shorter traces against per-domain length references, produced the balanced one. The reward formula is not disclosed.
The weights are marked webAI Non-Commercial License ver. 1.0. That is a separate grant from IBM’s Apache 2.0 terms on Granite. The license text is not in the score write-up. Exact permissions, and whether a commercial license is offered, are not established here. Read that document before you plan anything you would ship.
How to read the table
Every TwIL figure below is self-reported, in the setting the card calls high-effort pass@1, with thinking turned on. Pass@1 means one attempt is scored, not the best of several. Figures are for the balanced checkpoint unless the card marks them as verbose-only. Competitor numbers are published results. webAI did not re-run them in the same harness.
Qwen3.5-9B, the near peer on the card, is listed at 9.7 billion parameters, about a tenth larger than TwIL. Gemma 4 31B is more than three times the size. “Size class” is a loose label. These models were not lined up for a matched race.
Three contrasts that matter
MATH-500 is a 500-problem competition-math set. webAI reports 98.2. Qwen3.5-9B is listed at 96.6, from a third-party leaderboard. Gemma 4 31B is listed at 87.9, from a Red Hat lighteval run, and the card flags that figure as unusually low, possibly an answer-extraction artifact. Highest number in the printed comparison is a fair description. A clean win over a much larger model is not.
Contest math against Qwen is a split. Skip the coronation. On the four contests both publish, the card says the averages land within 0.4 points. That is the mean. The contests swing several points either way.
TwIL leads HMMT November 2025 by 3.8 points, 86.7 to 82.9, and HMMT February 2025 by 1.8, 85.0 to 83.2. It trails AIME 2026 by 4.2, 88.3 to 92.5, and HMMT February 2026 by 3.0, 68.2 to 71.2. AIME is the American Invitational Mathematics Examination. HMMT is the Harvard-MIT Math Tournament. Both are contest sets. Neither is a business task.
One oddity should stay visible. The card prints 88.3 for AIME 2025 and 88.3 again for AIME 2026. On the 2025 row there is no Qwen figure, and Gemma 4 31B is listed at 87.1, also from Red Hat. The identical TwIL scores are what the card shows. The card does not explain them.
Knowledge is the clear loss. The card’s own line is that this is “not a knowledge model.” GPQA Diamond, a set of graduate-level science questions, is 70.2. That is 11.5 points behind Qwen3.5-9B at 81.7, behind Gemma 4 12B at 78.8, and 14.1 points behind Gemma 4 31B at 84.3. MMLU-Pro, a harder broad-knowledge exam, was measured only on the verbose checkpoint. The score is 76.5 against Qwen’s 82.5 and Gemma 4 31B’s 85.2.
Code is the other bright spot, and it is not on the checkpoint you get by default. LiveCodeBench is a coding benchmark that is refreshed over time, so one number is not a permanent exam. On the verbose checkpoint, webAI reports 81.6, against 65.6 for Qwen3.5-9B, 72.0 for Gemma 4 12B, and 80.0 for Gemma 4 31B. A real gap versus the near peer. A narrow edge over the much larger Gemma. Not shown for balanced.
The paperwork under those rows is messier than a single leaderboard. TwIL’s scores and the peer scores were not produced in one harness. The Gemma 4 31B math figures above come from Red Hat. Some other Gemma contest figures on the card were measured by the Qwen team and reported on the Qwen3.6-27B model card, not by Google. TwIL is not compared with Qwen3.6-27B. A printed lead can go stale as soon as the next card lands.
Fewer tokens is not a price
webAI’s pitch is cost per task. The card measures output length. It does not measure dollars, latency, or hardware throughput.
On AIME 2026, TwIL averages about 19, 500 output tokens per problem. Qwen3.5-9B uses 31, 200. That is 38 percent fewer tokens, and a 4.2-point accuracy concession, 88.3 against 92.5. On HMMT February 2026 the gap is smaller: 32, 900 versus 40, 600, 19 percent fewer tokens, with accuracy at 68.2 against 71.2. Shorter than a close peer. Still a long monologue. No price attached.
The local-size claim is more concrete, and easy to mix up with the table. A quantized build is 4.98 GiB, small enough to try on a CPU or a GPU with modest video memory. Full-precision weights are 16.38 GiB per variant. The published scores are for the full checkpoint, not the 4.98 GiB file. Quantized quality has not been independently measured. And because the model thinks before it answers, a short generation cap cuts the reasoning off. What remains is a different system from the one behind the table.
Where it belongs
Small models keep getting described as if the expensive one could simply be swapped out. TwIL-8B is a useful counterexample. Close to a 9.7-billion-parameter peer on contest math. Ahead of that peer on one coding exam, on the longer checkpoint only. Clearly behind on knowledge. That is a specialist: a sharp analyst who can work a proof and still miss the context of the account.
In an automation flow, routing beats replacement. A quantitative check, with the inputs already assembled, is a fair job. A brief that has to know the customer is a bad one. Contest math will not stand in for retrieval. GPQA and MMLU-Pro are the rows that should kill that idea.
An agent setup can still put a specialist on the hard quantitative step and leave policy, history, and anything that has to be known to another model. Nothing published here shows how TwIL behaves with tools, as an AI agent, or on ordinary work outside the exam suite. One checkpoint trying to be both is how you get a confident wrong answer with excellent formatting.
Two limits belong in the pilot memo. The card states there is no safety or preference tuning beyond whatever Granite 4.2 already carries. And the model is English only. A customer-facing flow that assumes either extra alignment work or another language is assuming things this release does not claim.
Questions worth asking before anyone builds on it
- Does 98.2 on MATH-500 mean TwIL-8B leads its size class?
No. It is the top printed figure in webAI’s own comparison, and the Gemma 4 31B number under it may be depressed by answer extraction. Against Qwen3.5-9B, the four-contest average lands within 0.4 points. Individual contests swing by 1.8 to 4.2 points either way.
- Are the headline code and knowledge scores on the model you download by default?
No. LiveCodeBench and MMLU-Pro were measured on the verbose checkpoint and, per the card, have not been run on balanced. The default is the shorter variant. Assume less until someone shows otherwise.
- Does fewer tokens mean a lower cost per task?
Only as output length. The card shows about 19, 500 tokens versus 31, 200 for Qwen3.5-9B on AIME 2026, 38 percent fewer, at a 4.2-point accuracy cost. It does not publish dollar, latency, or hardware benchmarks. The 4.98 GiB file is a deployment fact. That file did not produce the table.
- Can a company ship this in a product?
Not established. The weights are marked webAI Non-Commercial License ver. 1.0, and IBM’s Apache 2.0 grant covers the Granite base, not this post-training. The license text is not in the score write-up, so exact permissions, and whether a commercial grant exists, have to be read there rather than inferred.
- Is the knowledge gap a reason to walk away?
For a general assistant, and for any brief that needs account context, yes. GPQA Diamond at 70.2 is 11.5 points behind a near peer. For a bounded math or code step where you already hold the facts, the gap matters less, because you are not asking it to know the client.
- What about safety?
The card states there is no safety or preference tuning beyond whatever Granite 4.2 already carries. The model is English only. A pilot that assumes extra alignment work, or coverage in another language, is assuming something this release does not claim.
- Should you trust the table as a buying guide?
Read it as a vendor’s map. TwIL’s scores are self-reported. Peer scores come from other groups and were not re-run in the same harness. Independent replication is still missing.
Read the license before the leaderboard.