Introduction

Streaming Code World Model shipped on October 12: open training/sampling code on GitHub plus two LoRA checkpoints on Hugging Face. Tingting Liao and Guangyi Liu (both list MBZUAI on their GitHub profiles) took the released Code World Model (CWM) from Westlake University's AGI Lab and NTU, and made it causal. Video now comes out chunk by chunk from a live code proxy, and no chunk can see future frames. Modality: proxy-conditioned video world model, built as LoRAs on MiniMax-H3. Where to run it today: self-hosted on one NVIDIA H200.

What shipped

  • Code: the rtcwm package covers CARLA data generation, pre-encoding, streaming diffusion-forcing training, 4-step DMD distillation and eval scripts. The GitHub "Initial release" commit landed 2026-10-12 at 12:04 UTC.
  • Checkpoints: streaming_df.safetensors (35-step) and streaming_dmd_4step.safetensors (4-step). Both are LoRAs for the released MiniMax-H3 and load on top of the CWM release.
  • Announcement:
    , "Code + checkpoints are open."

How it works

CWM's idea is that a coding agent keeps the world state as an executable proxy, and a video model renders it in any style. The original CWM renders against the whole proxy at once. The streaming version gives up that view of the future:

  • Chunks: each chunk is one 17-frame VAE clip (5 latents), about 0.71 s of video.
  • Causal conditioning: chunk c sees the caption, the start image, proxy chunks up to c, and earlier video chunks in its span. Nothing after that.
  • Sliding span: attention covers 6 chunks (the current one plus 5 before it). The KV cache moves by RoPE rotation, so positions don't grow over a long stream.
  • Diffusion forcing with a noisy history: each chunk gets its own noise level, so the model learns not to trust its own past output blindly. The authors say this slows long-horizon drift.
  • 4-step DMD self forcing: the student streams 8–14 chunks on its own output. A frozen teacher scores the last span.

This matters for game pipelines because a real-time renderer can't know what the player does next. If the proxy is a sim or game loop, the renderer has to commit one chunk at a time, and that's the setup this project trains for.

The numbers

These are the authors' figures: four held-out CARLA clips, 24 chunks (17 s) each, pixel MSE to ground truth, one seed, at 1344×768.

ModelStepsMean MSE ↓s / chunk (H200)

Streaming diffusion forcing

35

2231

47.9

+ DMD, released CWM as teacher

4

1741

5.2

+ DMD, streaming model as teacher

4

1660

5.1

The 4-step model is about 9× faster than 35-step sampling and also scores better. It still isn't real time, and the README says so: 5.1 s of compute for each 0.71 s chunk.

Run it

git clone https://github.com/TingtingLiao/streaming-code-world-model.git
cd streaming-code-world-model
hf download Luffuly/streaming-code-world-model --local-dir ckpt

# held-out CARLA clips, 4-step sampler
EVAL_ARGS="--steps 4 --rule x0 --shift 5 --kv-reuse" \
    sbatch scripts/eval/eval.sbatch ckpt/streaming_dmd_4step.safetensors out/ clip_0048 clip_0244

What you need first:

  • GPU: one H200 (141 GB). Peak memory is 123 GB in bf16 and 108 GB with --fp8. 80 GB cards aren't supported yet.
  • Stack: PyTorch ≥ 2.9 (CUDA 12.8), FlashAttention-3, the CWM repo's patched musubi-tuner, cwm_h3_inference, and the released CWM models.
  • Data: a separate CARLA 0.9.15 env to render clips, plus Qwen3-VL-32B-Instruct for start-frame captions. The Slurm scripts assume 8-GPU nodes. The authors list training at about 7 h on 8 nodes (diffusion forcing) and about 5 h on 4 nodes (DMD).

Caveats

  • Narrow training data: 342 CARLA clips (3.7 h), all of one pedestrian followed by a third-person camera. Results outside that setup are unproven.
  • Open-world path depends on code that isn't public yet: the README's 60-second open-world script expects scenes from "open-code-world-model". Its linked repo (guangyliu/cwm-proxy) returned 404 at time of writing.
  • No license file: neither the GitHub repo nor the HF card declares one. The upstream CWM repo is Apache-2.0. Check with the authors before any commercial use.

Where it sits

This is the second "engine owns the state, diffusion draws the frame" release in two days. NVIDIA's StreamRender-H3 drives a MiniMax-H3 renderer from a Three.js engine. Streaming CWM keeps a coding agent's proxy as the source of state and focuses on causal, drift-resistant streaming. Both point the same way: game logic stays in code, and the video model becomes a swappable renderer you can restyle.

Conclusion

Streaming Code World Model is a research release, not a product. It's narrow on data and heavy on hardware, but the code and LoRAs are real and runnable today. If you're building proxy-driven or sim-driven neural rendering for games, the causal masks, RoPE-rotated KV cache and DMD self-forcing recipe are the parts worth reading. Watch for the open-world proxy repo going public and a sub-80 GB path.

—Titus