NVIDIA Nemotron 3.5 Bolt E4B: 8B-parameter Gemma 4 post-train with 64K context for local assistants

NVIDIA released NVIDIA-Nemotron-3.5-Bolt-E4B-4GB on 2026-10-23, a small language model post-trained from Gemma-4-E4B-it with 8B total parameters (4B effective) and up to 64K tokens of input and 64K tokens of output. A GGUF build is now listed in Unsloth's Hugging Face repository, which also points to a guide on running Nemotron 3.5 Bolt and to its Unsloth Dynamic 3.0 GGUF quantization benchmarks.

Model details

NVIDIA positions Nemotron-3.5-Bolt-E4B for fast local voice and device assistants, interactive personas, and embedded or edge interfaces. According to NVIDIA, it keeps Gemma's conversational strengths while improving multi-turn tool use, instruction following, and input grounding. It is designed for low-memory, low-TTFT (time to first token) operation, so agents can respond quickly and leave headroom for applications or adjacent model pipelines.

Spec	Value
Total parameters	8B (4B effective)
Base model	Gemma-4-E4B-it
Precision (this build)	Q4_K_M
Context length	64K input, 64K output
Modality	Text in, text out
License	OpenMDW-1.1
Post-training data cutoff	May 2026

The model has a reasoning mode toggled by enable_thinking=True/False in the chat template, plus a runtime thinking budget. A separate conversational_mode flag is also exposed. NVIDIA recommends temperature 1.0 and top_p 0.95 in both thinking modes. A full-precision BF16 version and a companion DFlash drafter for speculative decoding are published separately. The sample llama.cpp command uses --spec-draft-n-max 7. NVIDIA lists llama.cpp deployment as "recipe pending."

Training

NVIDIA trained the model with multi-environment reinforcement learning using GRPO (Group Relative Policy Optimization). The environments covered instruction following, multi-step tool use, multi-turn conversations, and structured output. The run used an asynchronous RL architecture that decouples training from inference, built with NeMo RL and NeMo Gym. Model dates run from December 2025 to May 2026.

Benchmarks

NVIDIA compared the model against Gemma4 E4B-it, Qwen3.5 4B, and Nemotron-3 Nano 4B at both BF16 and Q4_K_M. Scores were measured by NVIDIA under its NeMo Gym harness, and NVIDIA notes they may differ from vendors' self-reported numbers. Most evaluations ran in reasoning mode, except RoleBench, EQ-Bench v3, and RULER.

Q4_K_M results for Nemotron-3.5 Bolt-E4B (BF16 in parentheses):

• MultiChallenge: 55.60 (53.10)
• RoleBench Custom: 71.21 (67.05)
• IFBench: 64.00 (63.79)
• IHEval: 82.40 (84.79)
• BFCL-v3: 67.31 (68.51)
• Tau2-Bench: 52.32 (55.10)
• RULER (64K): 85.76 (86.99)

For comparison at Q4_K_M, Gemma4 E4B-it scores 39.36 on MultiChallenge, 60.51 on RoleBench Custom, and 42.67 on IFBench. Qwen3.5 4B leads on some rows. At BF16 it scores 75.06 on Tau2-Bench against 55.10 for Nemotron, and 95.15 on RULER (64K) against 86.99.

Availability and requirements

NVIDIA published the model on Hugging Face and Build.NVIDIA.com on 2026-10-23. Supported hardware listed is NVIDIA Ampere (A100), Hopper (H100-80GB), and Blackwell, on Linux, with PyTorch as the listed runtime engine. Supported languages include English, Spanish, French, German, Italian, and Japanese, among others. The model is cleared for commercial and non-commercial use. NVIDIA notes it was post-trained on Gemma 4, whose use is governed by Apache License 2.0. No API pricing is listed.

What this means

The benchmark table shows where NVIDIA's post-training changed the base model: against Gemma4 E4B-it, Nemotron-3.5 Bolt-E4B posts higher scores on multi-turn instruction following, role and persona tasks, and tool-grounding tests at both precisions. The Q4_K_M build scores close to BF16 on most rows, and higher on a few (MultiChallenge, RoleBench Custom, ToolSandbox), which matters for memory-constrained deployments. Long-context retrieval and Tau2-Bench are weaker spots relative to Qwen3.5 4B in NVIDIA's own results. Developers building on-device assistants should test against their own workloads, since the numbers come from NVIDIA's harness.