Introduction

NVIDIA just released Nemotron 3.5 Bolt E4B 4GB, a small open language model it built for in-game NPCs, companions, and other interactive characters that need to talk and act, running locally in about 4GB of VRAM. The weights went up on Hugging Face on October 23 as a Q4_K_M GGUF under the OpenMDW-1.1 license (commercial use allowed). NVIDIA's

and technical blog list Hugging Face, Ollama, LM Studio, Unsloth, Jetson AI Lab, and the NVIDIA In-Game Inferencing SDK as places to run it today.

This one belongs on a game-dev desk because of the demo NVIDIA picked: a blacksmith trader NPC. The agent checks that the player paid the right amount, then calls tools on its own to look up the recipe, collect the ingredients, craft the item, and hand it over. That's an NPC brain that handles game logic, not a chatbot bolted onto a character.

What shipped

  • Model: NVIDIA-Nemotron-3.5-Bolt-E4B-4GB
  • Vendor: NVIDIA (post-trained from Google's Gemma-4-E4B-it)
  • Modality: text in, text out; conversational agent with multi-turn tool calling
  • Size: 8B total parameters (4B effective), shipped as Q4_K_M GGUF
  • Context: up to 64K tokens
  • Access: open weights, training data, and recipes; OpenMDW-1.1 license (Gemma 4 base under Apache 2.0)
  • Where to run it: llama.cpp via the model card's Docker recipe, plus Ollama, LM Studio, Unsloth, Jetson AI Lab, and the NVIDIA In-Game Inferencing SDK per NVIDIA's blog
  • Extras: a DFlash speculative drafter (NVIDIA says up to 21% higher throughput than Gemma-4-E4B's original MTP drafter) and a BF16 checkpoint

Numbers that matter for NPCs

NVIDIA's own tables, all at batch size 1 on llama.cpp (CUDA), Q4_K_M:

HardwareWorkload (cache / input / output)VRAMTTFTTokens/s

RTX 5070 (12GB)

0 / 3K / 50

3.5 GB

548 ms

133.4

RTX 5070 (12GB)

32K / 2K / 256

4.3 GB

1,238 ms

106.2

Jetson Orin Nano (8GB)

0 / 3K / 50

4.2 GB

5,785 ms

15.5

On the persona side, NVIDIA reports (BF16) RoleBench 67.0 win rate for character speaking style against 53.7 for Qwen 3.5 4B and 49.9 for Gemma 4 E4B, and RoleMRC 80.6 for role identity adherence. NVIDIA says it averages about 40% higher than its previous Nemotron 3 Nano 4B across its benchmark set and uses up to 80% fewer output tokens in reasoning-enabled comparisons. It doesn't win everything: Qwen 3.5 4B still leads on Tau2-Bench, BFCL-v3, and long-context RULER.

The 3.5GB footprint is what matters for games. NVIDIA says the small footprint is meant to leave headroom for the application and for other models in the same pipeline. On a 12GB card, that means the game itself. In NVIDIA's Reachy Mini demo, it's a Cosmos 3 Edge vision model feeding Bolt what the robot sees.

Run it locally

Straight from the model card. Pull the GGUF and the drafter, then serve with llama.cpp:

export MODEL_REPO=nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB
export MODEL_FILE=Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf
export DRAFTER_REPO=nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB-DFlash
export DRAFTER_FILE=Nemotron-3.5-Bolt-E4B-DFlash-BF16.gguf
export MODEL_DIR="$PWD/bolt-models"

hf download "$MODEL_REPO" "$MODEL_FILE" --local-dir "$MODEL_DIR"
hf download "$DRAFTER_REPO" "$DRAFTER_FILE" --local-dir "$MODEL_DIR"

docker run --rm --gpus all -p 8000:8000 -v "$MODEL_DIR:/models:ro" \
  ghcr.io/ggml-org/llama.cpp:server-cuda13-b11459 \
  -m "/models/$MODEL_FILE" \
  --spec-draft-model "/models/$DRAFTER_FILE" --spec-type draft-dflash \
  --spec-draft-n-max 7 --spec-draft-ngl 999 \
  --alias nemotron-bolt --host 0.0.0.0 --port 8000 -c 65536 -ngl 999 --jinja

That gives you an OpenAI-compatible endpoint on port 8000. The chat template has a conversational_mode toggle for short, speech-ready replies and an enable_thinking switch with a runtime thinking budget, so you can turn reasoning off for quick barks and back on for quest logic.

Writing the NPC prompt

NVIDIA's prompting guide is clearly written with game characters in mind. The parts worth stealing:

  • Split the system prompt into labeled sections: ROLE, IDENTITY, PERSONALITY, TASK RULES, SPEECH EXAMPLES.
  • Write rules as condition → action: "After crafting succeeds, call the delivery tool."
  • Label ownership explicitly with blocks like YOUR INVENTORY, PLAYER VIEW, and SHARED CHEST. A nearby chest doesn't mean the NPC owns what's in it, and an inventory the NPC can't see is unknown, not empty.
  • Send world state as a separate # CURRENT STATE message before the player's line, and tell the model that state is context, not something to read out loud.
  • Use hard limits like "limit responses to 1-2 sentences" instead of "be brief."

If you have tools whose results arrive late (a crafting timer, say), the card also documents an async pattern: acknowledge the tool call with a pending result, keep talking, then feed the finished result back later as a TOOL RESULT: user message.

Conclusion

Bolt E4B is NVIDIA betting that NPCs need a small, fast model that keeps character and calls tools correctly, not a frontier model on the other end of an API. About 3.5GB of VRAM, a 548 ms first token on an RTX 5070 for a 3K-token prompt, open weights you can use commercially, and a trader-NPC demo make it a serious option for local NPCs. Pull the GGUF, wire your inventory and crafting functions in as tools, and see if your shopkeeper can keep track of whose sword is whose.

—Titus