NVIDIA releases Nemotron 3.5 Bolt E4B, a 4GB model for local voice agents

NVIDIA ships the small Bolt

NVIDIA has released Nemotron 3.5 Bolt E4B 4GB, the smallest model in the Nemotron 3.5 family it announced eight days ago. The weights went up on Hugging Face on 23 October 2026, with a hosted version on build.nvidia.com, a technical report and a research page the same day. When NVIDIA launched Nemotron 3.5 Super VL and Bolt 74B on 15 October, it promised this smaller sibling for 23 October. It arrived on schedule.

The name describes the pitch. E4B means about 4 billion effective parameters, and 4GB is the rough memory budget of the quantized build. NVIDIA wants it to run next to a game, a voice assistant or a robot on a consumer graphics card or a Jetson module, and answer fast enough for a spoken conversation. It is not trying to beat frontier cloud models at hard reasoning. It is trying to be a good talker that follows instructions, calls tools and stays in character within a small memory budget.

What Bolt E4B is

Bolt E4B is not trained from scratch. It is a post-trained version of Google's Gemma 4 E4B instruction model, which has about 8 billion total parameters and 4 billion effective ones. It keeps Gemma's text, image and audio inputs. The model card lists a 64K-token context for both input and output, and NVIDIA's research page says it supports 35 languages, on top of a base model pre-trained on more than 140.

The main release is a Q4_K_M GGUF file for llama.cpp, quantized with an importance matrix calibrated on NVIDIA's own data so the 4-bit build keeps more of the full model's quality. A full-precision BF16 version and a separate speculative-decoding drafter are also published. Developers get two switches in the chat template. enable_thinking turns step-by-step reasoning on or off, and a runtime thinking budget caps how many tokens it can spend on it. conversational_mode tunes replies for spoken, back-and-forth use rather than long written answers.

The licence is NVIDIA's OpenMDW 1.1, which permits commercial use. Because the model is built on Gemma 4, use of the underlying Gemma weights is governed by Apache 2.0, which the model card notes. That is a looser setup than many open models with custom terms, and similar to the Apache 2.0 licence KRAFTON chose for its Raon-1 mini preview this week.

Reinforcement learning only, no fine-tuning pass

The most unusual part of the technical report is the training recipe. NVIDIA skipped supervised fine-tuning, the usual first step in which a model is shown thousands of example answers. Instead it used reinforcement learning only: rewards from verifiable checks, such as whether a tool call is correct, plus reward models that judge conversational quality. The training ran on NVIDIA's NeMo-RL and NeMo-Gym libraries using the GRPO algorithm.

The report says the team did try supervised fine-tuning first. It made the model worse at conversation, producing longer and more verbose replies, so they dropped it. They also fed the model twice as many non-reasoning examples as reasoning ones, because most real conversations do not need a chain of thought, and added length penalties to discourage padding. The result, NVIDIA says, is a model that uses up to about 80% fewer output tokens than peers in reasoning-enabled comparisons. Fewer tokens means less decoding work, which matters more than raw quality when the model shares a GPU with a game.

The benchmark claims

NVIDIA's research page says Bolt E4B scores about 40% higher on average than its previous small model, Nemotron 3 Nano 4B, and about 10% higher than other peers it tested. Its headline chart puts Bolt E4B at an average of 70.2 across its chosen benchmarks, against 63.2 for the original Gemma 4 E4B, 59.1 for Qwen 3.5 4B and 48.8 for Nemotron 3 Nano 4B. Larger local models score a little higher, Gemma 4 26B-A4B reaches 72.8, but need far more memory and take about twice as long to start answering.

The biggest gains are on instruction following. On IHEval, which tests whether a model respects the order of system, user and tool instructions, Bolt E4B scores 84.79 with full precision against 67.04 for Gemma 4 E4B and 60.57 for Qwen 3.5 4B. It also leads on MultiChallenge, a multi-turn conversation test, with 53.10 against 51.76 for Qwen, and on RoleBench, a role-play test, with 67.05. On RelyToolBench, which checks whether a model invents tools that do not exist, it scores 63.16 against 61.60 for Gemma.

These are NVIDIA's numbers, measured in its own setup, and several rely on other models as judges, including GPT-4o, GPT-4.1, GPT-5.2 and GPT-5.5. Two of the benchmarks, Brevity Bench and When2Call-MT, are NVIDIA's own and have not been published yet. The report also notes it re-measured Gemma 4 E4B itself and found differences from the scores in Google's model card.

Where it is weaker

The report is open about the trade-offs. Qwen 3.5 4B still beats Bolt E4B on several agent and long-context tests: 75.06 against 55.10 on Tau2-Bench, a customer-service agent benchmark, 70.95 against 68.51 on the BFCL v3 function-calling test, and 95.15 against 86.99 on RULER at 64K tokens. Gemma 4 E4B keeps a small edge on EQ-Bench, an emotional-intelligence test, at 41.17 against 39.40.

Vision got worse. Because training focused on text and speech conversation, Bolt E4B loses ground to the Gemma model it started from on image tests: about 5.6 points lower on MMMU-Pro Vision with reasoning on, and lower on MathVision, MedXpertQA and document parsing. NVIDIA says it tried extra training on the vision encoder alone without improvement, and lists restoring vision as future work. Audio held up better, with gains on VoiceBench and Audio MultiChallenge, although speech input is limited to 30 seconds per turn.

Speed, memory and the DFlash drafter

NVIDIA measured the 4-bit build at batch size 1 with llama.cpp. On an RTX 5070 with 12GB, a 3,000-token prompt used 3.5GB of graphics memory, took 548 milliseconds to the first token and then generated 133 tokens per second. With a 32,000-token cached context, memory rose to 4.3GB. On an RTX Spark laptop the same short test used 3.1GB with a 930-millisecond first token and 67 tokens per second. On a Jetson Orin Nano 8GB, the small robotics board, it used 4.2GB but needed almost 5.8 seconds to the first token and produced 15.5 tokens per second, usable for a robot but slow for live speech.

To speed up generation, NVIDIA released a DFlash drafter, a 360-million-parameter helper model with five layers that guesses seven tokens at a time for the main model to check in parallel. With reasoning on, NVIDIA reports a 2.74 times speed-up on an RTX PRO 6000, from 124.7 to 341.2 tokens per second, and 3.56 times on DGX Spark. Against Gemma's own multi-token prediction, the gain is smaller: 1.21 times on an RTX 5070 and 1.05 times on RTX Spark. For production servers, NVIDIA offers a NIM container running on vLLM, validated on DGX Spark, RTX PRO cards and H100.

Games, robots and assistants

NVIDIA's RTX blog calls Bolt E4B the third-generation small model for ACE, its toolkit for AI-driven game characters. The first generation powered AI teammates in a PUBG public beta earlier this year, and the second appeared in a preview of the game Aniimo at Gamescom. From 23 October, players can try Bolt E4B inside SkyrimNET, a mod by MinLL that gives every non-player character in The Elder Scrolls V: Skyrim memory, a voice and its own goals. It supports local inference on RTX Spark laptops as well as cloud models.

The research page also shows the model running on a Reachy Mini desktop robot driven by a Jetson Orin Nano, alongside NVIDIA's Cosmos 3 Edge model, plus a blacksmith character demo. Small robots are a growing target for this class of model, as the preorders for Hugging Face's Microduck robot show. NVIDIA also released Dory1.0 4B last week, so it now has two quite different 4B-class open models: one tuned for loops of reasoning, the other for fast, short conversation.

What to watch

The first test is independent benchmarking. Most of NVIDIA's comparisons rely on its own setup and AI judges, and two benchmarks are unpublished. Community results on llama.cpp and leaderboards will show whether the instruction-following lead over Gemma 4 E4B and Qwen 3.5 4B holds, and how much the vision regression matters for multimodal assistants.

The second is adoption in games. SkyrimNET is a mod, not a commercial release, and earlier ACE models have mostly appeared in previews and betas. A shipped game that runs Bolt E4B locally on players' own graphics cards would be the clearest sign that small on-device models are ready for real-time characters. Until then, Bolt E4B is a well-documented open model that developers can run on a mid-range GPU today.