Introduction

NPCTalk is a new MIT-licensed toolkit from the Universal Modder project, and it makes NPCs in a single-player game talk back. You speak into your mic, the NPC answers in character in a cloned voice, and it can act on what you said. Voices come from Fish Audio Drama 3 (or S2 Pro) running on a GPU you host yourself. The repo went public on Oct 15 (UTC) and ships as a plugin for Claude Code, Codex, Gemini CLI, Copilot, OpenCode and Cursor. The first full example is GTA V: walk up to anyone in Los Santos and start talking.

The parent project, Universal Modder, has been one of the week's breakout repos at nearly 8,000 GitHub stars. It gives coding agents skills for modding almost any PC game you own: engine recon, decompiling, fal-generated art, 3D and audio, and in-game testing. NPCTalk adds the conversation layer on top.

What shipped

  • Name and vendor: NPCTalk, a Universal Modder project, repo owned by Christos Antonopoulos
  • Modality: real-time NPC dialogue in-game. That means speech-to-text, an in-character LLM reply, streamed voice-cloned speech, and an optional physical action
  • Open vs API: the code is MIT. The voices use Fish Audio's open weights, self-hosted on Modal or fal. Speech-to-text (NVIDIA Nemotron ASR on fal), the LLM (Cerebras, swappable for any OpenAI-compatible endpoint) and actions (TypeSafe Jev, optional) are paid APIs
  • Where to run today: install it into your coding agent and point it at your game. The GTA V mod needs Windows, the .NET SDK 8+, Script Hook V and a copy of the game
  • What's inside: 8 agent skills, the npctalk CLI, a voice server (server/drama3.py, server/s2pro.py, keep-warm, fine-tune and scorer), and the full GTA V mod with 19 characters

The latency budget

The build notes set one target: the reply should start within about two seconds, so it feels like talking. Here's what the author logged in-game:

StepEngineTime

End of your sentence

open mic, 700 ms of silence

0.7 s

Speech-to-text

fal nvidia/nemotron-asr-multilingual/asr

0.38–0.80 s

In-character reply

Cerebras qwen-3.8-27b, reasoning off

0.28–1.0 s (spikes to 5–8 s)

Action pick

TypeSafe Jev, in parallel

0.07–0.18 s

First voice audio

Fish Audio on a Modal H100, streamed

0.35–0.55 s

Sentence end to voice

push-to-talk

1.15–2.2 s

The open mic adds its 0.7-second wait on top of that. Two choices kept it inside budget: a fast LLM with reasoning turned off, and a voice server that streams, so the NPC starts talking before the rest of the line is generated.

Install

Pick your agent:

# Claude Code
/plugin marketplace add christos-antono/NPCTalk
/plugin install npctalk@npctalk

# Codex
codex plugin marketplace add christos-antono/NPCTalk
codex plugin add npctalk@npctalk

# Gemini CLI
gemini extensions install https://github.com/christos-antono/NPCTalk

# Skills only, any agent
npx skills add https://github.com/christos-antono/NPCTalk

Keys go in environment variables: FAL_KEY, CEREBRAS_API_KEY, TYPESAFE_API_KEY (optional), plus modal setup for the voice GPU. You also need Git, Python 3.10+, uv and ffmpeg. Then start with a prompt like:

Make the villagers in my game answer when I talk to them, in their own voices.

The make-npcs-talk skill takes the agent through the same build order every time. It recons the game, stands up the voice server and tests it with npctalk say, then gets one NPC answering typed text through the four hooks a game has to provide: who you're talking to, scene facts, voice playback at the NPC, and actions. After that it adds voice input, personas, actions, verification in the real game with every step timed, and finally a field note for the next agent. The skill includes per-engine notes for GTA V, Unity, Unreal, Bethesda games, Minecraft and custom engines.

How good are the voices?

The author measured one character, GTA V's Lamar, on ten test lines. "Voice match" is speaker similarity against 60 held-out real clips:

SetupVoice matchWord error

ElevenLabs instant clone

0.612

10.5%

Fish Audio Drama 3, defaults

0.643

7.1%

Drama 3, best of 8 takes, checked

0.704

8.6%

Fish Audio S2 Pro, best reference clip

0.707

5.6%

S2 Pro fine-tuned on the character

0.785

7.5%

Without fine-tuning, Drama 3 and S2 Pro tie on likeness. Drama 3 acts out tags like [crying] and [laughing] better and starts talking faster, with first audio at about 0.33 s from the PC. The write-up also admits that one early "win" didn't hold up on a second run, which is the kind of honesty you want from a benchmark table.

Costs and caveats

  • GPU bill: an H100 costs about $4 an hour while it's up. It scales to zero 5 minutes after the last request, and a cold start takes 1–3 minutes, so the mod pings it every 90 seconds while you play and falls back to MiniMax voices on fal while it wakes. MiniMax costs $1.50 per voice clone and takes 2.5–4.5 s per line
  • Single-player only: the rules forbid injecting into online games with anti-cheat. For GTA V, never launch GTA Online with a script hook installed
  • Voice rights: the GTA build cloned voices from character voice-line compilations for private use. None of that audio ships, and npctalk check blocks committing audio, transcripts, keys or server URLs. Use recordings you have the rights to
  • Small and new: the repo is a day old with a handful of stars. Expect rough edges outside the GTA V path

Original Source

https://github.com/christos-antono/NPCTalk

https://github.com/christos-antono/NPCTalk/blob/main/docs/how-it-was-made.md

https://github.com/rehan-remade/universal-modder

Conclusion

NPCTalk is the clearest recipe yet for voice-driven NPCs you can run on your own stack. It comes with a logged latency budget, a scored voice comparison, and an agent skill that builds it in a fixed order. If you mod single-player games or prototype NPC dialogue in Unity or Unreal, the voice server and the four-hook pattern are worth stealing even if you never touch Los Santos.

—Titus