The small model that follows the rules, then drops the errand
A customer-service agent that knows when to stay quiet is useful. One that cannot finish the job is a liability. NVIDIA’s Nemotron 3.5 Bolt E4B 4GB currently looks like both.
Bolt is a small reasoning model NVIDIA post-trained from Google’s Gemma 4 E4B instruction checkpoint, gemma-4-E4B-it. The jobs NVIDIA names are the ones that hate a cloud round trip: on-device voice assistants, game characters, and the speech-and-tool layer of an embedded or industrial robot. The “4GB” in the name is a memory budget, meant to leave room on the same board for a speech model and a vision model. It is not the size of the file you download.
The buying decision is a split. On scores NVIDIA has published, Bolt is stricter when instructions collide, and more cautious about calling a tool it should not call. It is weaker when the job is a multi-turn customer errand. It is also a different product on a desktop graphics card than it is on a small robot board.
Fast on a desk, a pause you can feel on a robot
NVIDIA’s hardware list runs from GeForce RTX to Jetson and Spark-class boards. The figures a buyer can use come from NVIDIA’s tests of a Q4_K_M build, a 4-bit local quantization, in llama.cpp, with a batch size of 1, about 3, 000 input tokens, 50 output tokens, and no cached context.
On an RTX 5070 with 12GB, NVIDIA reports 3.5GB of memory in use, 548 milliseconds to the first token, and 133.4 tokens per second. RTX Spark is listed at 3.1GB, 930 milliseconds, and 67.0 tokens per second. Jetson AGX Orin is 4.2GB, 3, 687 milliseconds, and 24.0 tokens per second. The Jetson Orin Nano, an 8GB board, is 4.2GB, 5, 785 milliseconds, and 15.5 tokens per second.
Read that memory line before the slogan. The published quantized file is listed at 6.18GB, and NVIDIA’s broader RTX range runs from about 3.1 to 4.3GB, so even the desktop band is not a clean sweep of the 4GB target. Both Jetson measurements, at 4.2GB, miss it. Fitting an 8GB Orin Nano is a different claim from staying inside 4GB. The model can sit on that small board. It does not meet the memory name there.
Nearly six seconds before speech starts is not a thoughtful pause. It is dead air. On the RTX 5070, the same model looks like a live conversation. On the Orin Nano, it is a pause you can feel. Those are different products.
Do not paste a faster number next to that table. NVIDIA has reported Bolt with a DFlash drafter at 272.6 tokens per second on an RTX 5070, against 225.9 for Gemma 4 E4B with its own drafter, about 1.21 times faster in that comparison. The drafter card lists a different setup: llama.cpp, an RTX 5090, and Linux. The helper proposes up to seven candidate tokens in a step, and Bolt verifies them. It has no embeddings and no output head, so it cannot run alone. It has to be loaded with the Bolt model file. The same card lists a maximum context of 131, 072 tokens and says the drafter attends only to the most recent 1, 024. That is a helper’s card, not a production prompt budget to copy onto every checkpoint.
Whose instruction wins, and who finishes the errand
Higher is better on the scores below, and every one of them is NVIDIA’s own measurement, not an outside audit. Vendor numbers move from harness to harness. Use them as a map of this release.
The clearest lead is instruction hierarchy. On IHEval, which NVIDIA scores as conflict accuracy (whose instruction wins when they collide), Bolt is at 84.8. Gemma 4 E4B is at 67.0. Qwen3.5-4B is at 60.6. The prior Nemotron 3 Nano 4B is at 51.3. If you are putting AI agents in front of customers or staff, that is the number to stare at. System policy has to outrank a clever user prompt.
InverseIFEval, NVIDIA’s score for counterintuitive instructions, points the same way: Bolt at 46.8, Gemma at 42.0, Qwen at 31.0, Nano at 35.2. Structured and ordinary instruction following sit in the same neighborhood. On StructEval-T, a structured-output score in NVIDIA’s table, Bolt is at 80.4 against Gemma’s 79.3 and Qwen’s 76.0. On IFBench, an instruction-following score, the gap over the base model is wider: 63.8 for Bolt, 42.8 for Gemma, 58.9 for Qwen.
Think of Bolt as the receptionist who will not let a visitor rewrite the building rules. That is a feature. It is not the whole job.
Tau2-Bench is NVIDIA’s multi-turn customer-task score. Bolt is at 55.1. Qwen3.5-4B is at 75.1. Gemma is at 38.5 and Nano at 29.2, so Bolt is not last. Against the small model most buyers would actually compare it with, it is twenty points behind.
A local agent that holds a policy still has to rebook, check a balance, and call the next system without dropping a step. Tool use is a split, not a single loss. Qwen also leads BFCL-v3, a function-calling benchmark, 70.9 to Bolt’s 68.5, and ToolSandbox, a test of tools when the world’s state changes, 69.5 to 68.8. Those gaps are narrow. The counterweight is When2Call-MT, which asks when to call a tool, clarify, or abstain. Bolt scores 63.4. Qwen scores 37.8. Gemma is close behind Bolt at 62.9. Knowing when not to fire a tool is a real skill. Finishing the task once the tools are in play is a different one.
Complex agent work is a lead with weak absolute scores. On AgentIF, NVIDIA’s test of complex agentic instructions, Bolt scores 25.8. The other three models land between 20.1 and 21.9. Best of a weak group is not ready for open-ended agent work.
Characters, not just policies
For game characters, the more relevant published number is RoleBench, a custom win rate on speaking style against a reference. Bolt is at 67.0 percent. Qwen is at 53.7. Gemma is at 49.9. Nano is far behind at 12.0. Custom yardsticks deserve a raised eyebrow (NVIDIA designed the metric), but the margin fits how NVIDIA is pitching Bolt as the third-generation small model for ACE, its system for game characters.
RoleMRC, a role-identity score NVIDIA labels as excluding multi-turn, is 80.6 for Bolt, 74.6 for Gemma, and 65.8 for Qwen. Do not read that as proof a character stays in role across a long scene. MultiChallenge, a conversation score, is a slim lead rather than a blowout: 53.1 for Bolt, 51.8 for Qwen, 42.1 for Gemma, 35.3 for Nano.
The trade was the training decision
NVIDIA’s account of the post-training is the explanation for that split. The company says it tried supervised fine-tuning, found that the stage improved tool calling, and also found that it hurt the conversational quality and the verbosity inherited from Gemma. It rejected that stage. The shipped recipe, in NVIDIA’s telling, used reinforcement learning only, through NeMo-RL and NeMo-Gym, NVIDIA’s training and evaluation tooling.
NVIDIA also says every training sample ran in both reasoning and non-reasoning modes, with batches held to a two-to-one ratio of non-reasoning to reasoning, and that length penalties kept answers from swelling. If that account holds, the trade was deliberate: keep Gemma’s conversational habits, and add instruction following, hierarchy, and tool judgment without a conventional fine-tune. The Tau2 result says the tool-completion half of that trade is unfinished.
Two licenses, then the test you still have to run
In the robot demo NVIDIA has described, a Reachy Mini on a Jetson Orin Nano 8GB uses Cosmos 3 Edge for vision and Bolt for speech replies and tool calls. In that setup, Bolt is not the eyes. A small language model that talks and calls tools, beside specialist models that see and hear, is a sensible way to spend one memory budget. That describes the demo. It is not a measured limit on what Bolt itself can see.
Local inference earns its place when latency, privacy, an offline site, or a hard device limit makes a cloud model a bad fit. A kiosk, a game client, a robot that cannot wait on a network: those are coherent uses. A sales desk that lives or dies on multi-turn tool use is a weaker one, on the evidence NVIDIA itself published.
NVIDIA describes a conversational mode for short, speech-ready replies, a thinking budget meant to cap reasoning latency, and asynchronous tool calls for multi-turn work. Those are the knobs a voice product would want. They do not cover the failure cases: a contradicted instruction, a tool the agent should refuse, a customer who changes the request on turn four.
The license is a shipping risk, not a footnote. The Gemma 4 E4B base remains under the Apache License 2.0. The DFlash card says use of that release is governed by the OpenMDW License Agreement, version 1.1, and describes the artifact as ready for commercial or non-commercial use. That sentence is not a reading of the license, and it does not by itself settle datasets, recipes, or every related checkpoint. Two licenses on one stack is how a product team gets surprised. Have counsel read OpenMDW 1.1, and confirm what it covers, before a customer hears the voice. Weights are on Hugging Face. NVIDIA NIM is another route named in the launch materials, and pricing and usage limits are not in what has been published.
The published deployment language is the right limit. Use-case testing is required before anyone calls a deployment safe. A strong IHEval score is a reason to test Bolt where policy has to beat a user prompt. It is not that test.
Questions worth asking before you pilot it
-
Does “4GB” mean the model is a 4GB download?
No. It is a VRAM target, not the file size. The published quantized file is listed at 6.18GB. NVIDIA’s RTX runs used about 3.1 to 4.3GB, and the Jetson runs used 4.2GB, enough to fit an 8GB board and still over the named envelope.
-
Did Bolt win the small-model comparison outright?
No. NVIDIA’s published scores show a lead on instruction priority, structured outputs, role style, and knowing when not to call a tool. Qwen3.5-4B leads the tool-completion tests, and the multi-turn customer-task gap is the one that should slow a purchase.
-
Is this the default for sales or support automation?
Not as a default. Tau2-Bench, the multi-turn customer-task score, has Bolt at 55.1 and Qwen3.5-4B at 75.1. Bolt is more interesting where policy must beat a user prompt, or where refusing a tool call matters more than chaining many of them.
-
Will it feel fast enough for a live voice agent?
On NVIDIA’s llama.cpp figures, an RTX 5070 looks conversational, with 548 milliseconds to the first token and 133.4 tokens per second, while a Jetson Orin Nano waits 5, 785 milliseconds. A separate DFlash figure of 272.6 tokens per second is a different desktop test, and the drafter card lists an RTX 5090, so do not quote it as the Jetson result.
-
Can one model see, talk, and call tools?
Not in the demo NVIDIA has described. Cosmos 3 Edge handles vision there, and Bolt handles speech replies and tool calls. That is how the demo is built, not proof of what Bolt can or cannot see on its own.
-
Can you ship it commercially?
The Gemma base is Apache 2.0, and the DFlash card points to OpenMDW 1.1 while saying that artifact is ready for commercial or non-commercial use. Read OpenMDW 1.1 before you treat product use, data, and recipes as settled, and test the failure modes before a customer hears the voice.