Introduction

inclusionAI just put the weights for Duo v1 on Hugging Face, and it's the closest open thing yet to an NPC you talk to face to face. Duo is a "reciprocal world model": it keeps taking in your audio, video and text while it generates a character's video and voice back, so the character can respond, get interrupted and pick the conversation back up. The code is on GitHub under Apache 2.0, the paper is on arXiv, and four checkpoints went public on November 13.

What shipped

  • Model: Duo v1 (Duo-1.0 causal, distilled and FP8-quantized, plus a Duo-1.2 causal base)
  • Vendor: inclusionAI (Ant Research)
  • Modality: real-time audio-visual interaction, with video and speech out and your video and speech in
  • Open vs API: open weights, released under the MiniMax H3 Community License. The code is Apache 2.0
  • Where to run it today: locally from the Hugging Face collection with the repo's offline demo, or the vLLM-Omni and SGLang-Omni streaming branches
  • Concrete fact: the distilled Realization model samples in 4 Euler steps (flow shift 5.0) instead of 50, at a default 768×576 output

How it works

Duo splits the character into two halves that share one timeline:

  • Intent is autoregressive. It reads what you're saying and doing and decides when the character should speak, listen or act.
  • Realization is the video and audio generator. It was initialized from MiniMax H3 and draws the character's response chunk by chunk.

Attention masks keep both halves from peeking at future input, which is what lets the model handle interruptions instead of waiting for you to finish. The paper says users can also show it real objects, reshape a virtual scene or play simple audio-visual games without a mouse or keyboard. For game designers that's the interesting part. It isn't a talking-head lip-sync. It's a character model that decides when to take its turn.

Which checkpoint to grab

  • Duo-1.2-causal-base: the full bundle with quality improvements, and the repo's default. 50 sampling steps.
  • Duo-1.0-causal-base: the full 1.0 bundle with Realization, Intent, codecs, speech recognizer and forced aligner. 50 steps.
  • Duo-1.0-distill: a 4-step Realization overlay on the 1.0 base.
  • Duo-1.0-distill-quant: the same overlay in ModelOpt FP8.

The distilled and quantized repos are overlays, so you download the 1.0 base first and drop them into the same folder:

git clone https://github.com/inclusionAI/Duo.git && cd Duo
hf download inclusionAI/Duo-1.0-causal-base --local-dir weights
hf download inclusionAI/Duo-1.0-distill-quant --local-dir weights
cp configs/inference.example.json configs/inference.json
.venv-model/bin/python run_demo.py --config configs/inference.json --port 4173

Set up the two Python environments the README describes first (one for the model, one for Qwen3-ASR). Then open http://127.0.0.1:4173/video/index.html, record or upload a clip, and Duo plays back its audio-visual reply.

The catch

  • Hardware: the reference setup wants two CUDA GPUs, one for Intent and the codecs and one for Realization. The FP8 Realization overlay alone is roughly 47 GB, and the Duo-1.2 repo is about 153 GB on Hugging Face.
  • Offline first: the main branch processes a finished recording. Live full-duplex streaming lives in the vLLM-Omni (recommended) and SGLang-Omni branches, and the project page says its real-time results are still to come.
  • It's a preview: v1 covers basic audio-visual conversation. The team says Duo v2, which will condition the video directly on your camera feed, is coming soon.
  • License territory: because the weights come from MiniMax H3, they carry its community license, and that license excludes the EU, the UK, South Korea and the United States. If your studio is in one of those places, the Apache 2.0 code doesn't help you. Read the LICENSE file before you build anything you plan to ship.

If you've been faking reactive NPCs with an LLM, TTS and a lip-sync pass, Duo is the first open model that does all three in one loop and can be interrupted. Running it takes a lot of hardware, but you can test it today.

—Titus