A local agent file under 8 GB that still calls tools is a real deployment question. Underdog’s answer is Saluki 27B: an IQ2-mix build of Qwen3.8-27B, listed at 7.89 GB against 54 GB for the full weights, in a standard GGUF that stock llama.cpp can serve. The card says the skill it protected is tool calling. The same page shows what that protection did not cover: competition math, a slice of software-engineering tasks, and the formatting of parallel calls.
That is a deployment pitch. It does not claim general parity. The IQ2-mix name sits at the aggressive end of local quantization, the neighborhood of two bits per weight. Memory and bandwidth are why a team tries it. Accuracy is why they should hesitate. The card is at huggingface.co/ConwayResearch/Underdog-Saluki-27B-1.0. It lists the file as Apache 2.0. The repo is under Conway Research, and the card says the model was built by Underdog and does not explain the relationship.
The useful reading is the maker’s tool-call table, the losses on the same page, why none of this is a Berkeley leaderboard rank, whether the file is just a relabeled quant, and what the smaller footprint actually buys on long agent traces.
What the card actually measured
On Underdog Bench (120 Berkeley Function Calling Leaderboard v4 tasks, frozen before any model was tested, thinking off, temperature 0), the card credits Saluki with 88 of 120 and the full Qwen3.8-27B with 84 of 120. That is about 73 percent versus 70 percent, a gap of four tasks. A separate set of 100 parallel tasks, scored with the official checker and again with thinking off, lands at 42 of 100 against 35 of 100.
Four tasks is too small a difference to act on, even if you take the set at face value. Underdog says 120 tasks is a modest test, and that a gap of a few tasks is run-to-run variation. That caveat is theirs. The result is also not independent. The same organization built the model, froze the tasks, and published the score. The parallel gap is a bit wider, seven tasks in 100, and it is still the maker’s harness.
The card’s other small model on that bench is Bonsai 2, at 5.95 GB and 70 of 120. Shrinking alone does not produce Saluki’s tool-call number. A still-smaller file on the same bench did clearly worse. Size is the deployment fact. It does not explain the number.
The card credits Saluki as built on Qwen3.8-27B and on ISTA-DASLab’s GSQ-RCO GGUF, and states both as Apache 2.0. The public card does not publish a score for that ISTA parent file.
The losses on the same page
The rest of the table does not support a general-quality claim. On a 50-issue SWE-bench Verified slice the maker ran, Saluki resolved 30 of the issues and the full model resolved 33. Those are issue counts. Do not read them as percentages.
Comparisons the card marks as public go the wrong way on work that shows up once an agent leaves the function-call sandbox. MBPP+, a coding set, is 78.0 versus 83.9. MuSR, a multi-step reasoning set, is 67.5 versus 79.6. AIME 2025, average of four, is 79.2 versus 96.7. AIME 2026, same scoring, is 80.0 versus 94.6. The card’s own reading of that math gap is about 82 to 85 percent of the full model’s competition-math score.
There are small wins, and they are easy to overread. On IFEval prompt-loose, a lenient score of instruction following, Saluki is at 93.5 against a public full-model 91.5. IFBench prompt-loose is 72.7 against 71.0. Keep the warning on IFBench. Qwen’s own card lists IFBench at 79.5, so Underdog’s 71.0 and Qwen’s 79.5 are different reported figures. They do not form one ranking. A 50-issue SWE-bench slice is a different test from SWE-bench Pro, where Qwen reports 61.7 for the full model under a different setup.
The card’s line about 96 percent average retention across nine benchmarks is a vendor summary. The nine inputs are the 120-task bench, the 100-task parallel bench, the 50-issue SWE-bench slice, IFEval, IFBench, MBPP+, MuSR, AIME 2025, and AIME 2026, each taken as Saluki divided by the listed full-model number. That mean hides a tool-call bump, double-digit drops on math and MuSR, and a smaller drop on MBPP+. Some of those full-model numbers come from another harness, which the card discloses. The average looks calm. A workflow that depends on the weak rows will not.
The limitation that should worry anyone shipping AI automation is quieter than a math contest. Underdog warns that about a fifth of parallel-call replies have small formatting slips. That warning is separate from the 42-of-100 result. One number is a success rate on a 100-task sample. The other is a format-error rate. The model can hold the right arguments and still fail the parser your server actually uses.
A sales workflow that must check inventory, update the opportunity, and pull a pricing rule in one turn lives in that failure mode. Strict scoring keeps only the calls llama.cpp parses, then hands them to the Berkeley checker. BFCL does not include a lenient reader that scavenges raw function text. A formatting-sensitive set can move with the parser even when the weights do not. A Marktechpost walkthrough describes a harness built to run that comparison, one file at a time, on one GPU, and the published write-up does not report the measured scores, speed, or memory. Until those numbers are public, the vendor table is what you have, and it already flagged parallel calls.
Three different things called BFCL
None of this is a Berkeley leaderboard rank. BFCL v4, as Berkeley describes it, is a broader agentic evaluation, including web search, scored as an unweighted average of sub-categories. The leaderboard page at gorilla.cs.berkeley.edu/leaderboard.html ties its results to Gorilla commit f7cf735, also packaged as bfcl-eval 2025.12.17. That package version is the date stamp on “current.” The label is not fixed.
Underdog’s number is a frozen slice: 120 tasks, plus a separate 100-task parallel set. A verification notebook drew yet another sample (five kinds, 24 tasks each, seed 2026, 120 tasks total) and pinned Gorilla commit 6ea57973, not f7cf735. Those five kinds are not the full v4 leaderboard. All three descriptions can be narrowly true. None of them lets you say Saluki ranks on BFCL.
Qwen’s own agent numbers should stay in their own column. On its card, the full model is positioned for coding, professional work, research, and long-horizon agents, with thinking on by default. Reported figures such as SWE-bench Pro belong to that model, often under other harnesses, sometimes corrected and re-run. Do not read them as Saluki scores. A managed Qwen3.8-27B endpoint with a million-token context and built-in tools was described as coming soon, with no date. Waiting for that endpoint and running a 7.89 GB file next to your data are different bets.
Not a new sticker on ISTA’s file
A buyer skimming metadata would assume Saluki is ISTA’s IQ2_XS file with a new name. The GGUF header still names ISTA-DASLab as the quantizer. The card says Saluki was built on that file, at huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF. It does not say the weights were left untouched.
A walkthrough pinned both files and compared headers and bytes. Saluki stores 146 tensors at fewer bits, and every type change cuts bits. Those cuts did not land on the attention projections. The weights are not byte-identical.
Saluki is a lightly changed model, not a re-encoding of ISTA’s file.
That is one inspection’s conclusion. There is no second lab behind it. It is still the right procurement question. The file you deploy is a lightly edited checkpoint that sits on that quant. A known ISTA artifact would be a different file.
The pinned copies print as 7.90 GB for Saluki and 8.42 GB for ISTA’s IQ2_XS. Against the card’s 7.89 GB, that difference is rounding. It does not mark a different product. The same walkthrough embeds a vendor line scoring the ISTA file at 76 on a different unpublished set of 120. That 76 is not on the public card. Do not treat it as card-backed. Bonsai 2’s 70 of 120 is the third tool-call number the card actually publishes.
What the memory cut actually buys
Qwen3.8-27B does not use 64 ordinary attention blocks. The layout is 16 repeats of three linear-attention layers and then one full attention layer. Most layers keep a fixed memory state that does not grow with the prompt. Only 16 of the 64 layers grow a key-value cache as context gets longer. Long agent traces are cheaper here than in a fully attentional 27B of the same width. That is the business point.
From the published head counts, if cache elements are two bytes, those 16 layers cost 64 KiB per token: 16 layers, times key and value, times 4 key-value heads, times a head dimension of 256, times 2 bytes. The same cache shape on all 64 layers would be 256 KiB per token. That 64 KiB figure is arithmetic from the spec. Qwen does not print it.
Do not inherit the million-token marketing by accident. Qwen’s native context is 262, 144 tokens, extensible to 1, 000, 000. Underdog’s quickstart sets the server context to 32, 768. The verification bench used 16, 384. The card does not say the IQ2 GGUF supports a million-token window. For AI for business, the honest window is the one you set and test.
Serving it is ordinary local inference. Match temperature 0 and thinking off, or the run will not reproduce the tool-call setup. Thinking is on by default and can be switched per request. llama-server‘s own defaults, as the verification notes point out, are temperature 0.8 and top-k 40. The main file is text only. Vision is a separate download, 928 MB at full precision or 629 MB in a smaller build, in line with the card’s 0.6 to 0.9 GB note. Underdog reports a smoke test of chat, thinking, and tool calling on Ollama 0.40.4 on Apple silicon. A Mac can confirm the file loads. A pilot still wants your real tool schemas.
Questions worth settling before a pilot
-
Is 7.89 GB small enough to change where an agent runs?
As a deployment fact, yes. The card puts Saluki at 7.89 GB against 54 GB, in a GGUF that stock llama.cpp can serve. That is a one-GPU conversation, sometimes a workstation conversation. Do not take it as a claim that the small file thinks like the large one.
-
Does it beat the full model at tool calling?
Only on the maker’s own sets, which are not an independent evaluation: 88 of 120 versus 84 of 120, and 42 of 100 versus 35 of 100 on parallel tasks, both with thinking off. Four tasks in 120 is too small to act on. On the card’s math, MuSR, MBPP+, and 50-issue SWE-bench numbers, Saluki is worse.
-
Is that a Berkeley Function Calling Leaderboard score?
No. It is a frozen slice of BFCL v4 tasks. The public leaderboard is a broader evaluation pinned to Gorilla commit f7cf735, packaged as bfcl-eval 2025.12.17. A seeded 120-task notebook sample is a third thing again.
-
Is the file just ISTA’s quant with a new name?
The header still names ISTA-DASLab, and the card says Saluki was built on that GGUF. A byte-level inspection found lightly changed weights and 146 tensors stored at fewer bits. It did not find a pure re-encode. That inspection is one source. The card neither confirms the diff nor claims the weights were untouched.
-
What should a workflow expect to lose?
Competition math, down to about 82 to 85 percent of the full model by the card’s own reading, plus formatting. About a fifth of parallel-call replies have small formatting slips, which is separate from the 42-of-100 success rate. A correct argument can still fail the parser. Those slips matter more to AI for sales than a screenshot of 88 of 120.
-
Should this run a customer-facing agent tomorrow?
Not on the vendor table alone. The card’s quickstart does run in stock llama.cpp. Match temperature 0 and thinking off if you want anything like the published tool-call setup, and treat vision as a separate download. The published independent harness does not yet show its scores or speeds. Freeze your own traces, score parsed calls, and treat parallel tool use and any arithmetic as the tests most likely to fail.
Test the calls you actually make
For AI agents in a real stack, ask whether the calls your automation actually makes survive the shrink, on your parser, against your tools. Parity between a roughly two-bit Qwen and full Qwen is a separate question.
A scheduling agent with one function and short arguments is a different risk from an agent that fans out parallel calls and then does arithmetic on the results. Saluki’s own table says the first family is where the small file looks close. The second family (parallel formatting, math, multi-step coding) is where the 54 GB model still has the advantage.
Use the vendor numbers as a hypothesis. Pull 50 or 100 traces from production, including the ugly parallel ones, and score the parsed calls rather than the prose around them. Their four-task gap is too small to act on, and your traces will have a different count and a different mix, so measure the variance on your own set. The file is small enough to run that test this week. The cutover is not.