Introduction

PlayGen2 is a new open playable world model from researchers at CUHK, Tencent and NTU (arXiv 2611.07867, listed Nov 6, 2026). Code and pretrained weights are up under Apache-2.0 at MajorDavidZhang/PlayGen-2 (release v0.1.0, 264 MB weights zip). You get a real-time, keyboard-playable Doom level where the learned model runs the world and the renderer. The original game engine isn't used during rollout. It runs locally on CUDA, Apple MPS or CPU.

What's different: most "neural Doom" demos (GameNGen, the first PlayGen) use generated frames as their only memory. PlayGen2 splits the job the way a game engine does. It keeps an explicit 3D state and then draws it.

What shipped

  • World-state model (41.2M params): a persistent world memory. Static geometry lives in a volumetric occupancy grid, and the player plus every entity get identity-tracked 3D states. Inside a local region it predicts action-conditioned movement, fills in missing geometry and discovers new entities.
  • Neural renderer (29.1M params + fixed 55.3M VAE): draws first-person RGB from depth and semantic projections of that state. Generated frames are never fed back into state prediction, so the visuals can't wear down the map memory over time.
  • Modality: keyboard actions (7 action IDs) → explicit 3D state + generated 160×120 view
  • Open vs API: fully open, Apache-2.0 code and weights. No hosted API or cloud account.
  • Training scale: about 500 hours / 500 GB of bot gameplay on Freedoom2 MAP01. Trained from scratch in about 288 A100 GPU-hours (4×A100, ~3 days).

Concrete fact: on a MacBook Pro M4 Pro the full simulate-and-render loop runs at about 160 ms backend latency (paper headline). Serial mode measures 4.98 FPS, and overlapping world updates with rendering reaches 7.16 FPS.

Run it locally

git clone https://github.com/MajorDavidZhang/PlayGen-2.git
cd PlayGen-2
python -m venv .venv && source .venv/bin/activate
pip install -e '.[fast]'
curl -fL --retry 3 -o playgen2-weights.zip \
  https://github.com/MajorDavidZhang/PlayGen-2/releases/download/v0.1.0/playgen2-weights.zip
unzip -o playgen2-weights.zip
python -m playgen2.server --device auto

Open http://127.0.0.1:8090, pick a spawn and an init mode, then hit Reset → Run. W/S moves, A/D turns, Space attacks, E uses. On NVIDIA use --device cuda --amp bf16. Keep FP32 on MPS/CPU. --sampling-steps 4 halves renderer calls, and --layout-only skips RGB entirely so you can watch the state model by itself.

Two init modes are worth trying back to back:

  • Global: the full static map is loaded and locked. Only the player and NPCs are model predictions.
  • Local: only the starting neighborhood is seeded. New terrain and enemies come from the model as you walk.

Why game devs should care

  1. State you can script. The paper shows programmable gameplay: rule-based edits to the explicit state change what happens next. Pixel-only world models don't give you that hook.
  2. Multiplayer by composition. Multiplayer is built by composing single-player models. No retraining.
  3. Zero-shot maps. It was evaluated on seven unseen community Doom II levels (Scythe), not just the training map.
  4. Small and cheap. About 70M learned params plus a VAE, trained on a 4-GPU budget. That's indie-lab scale compared with the frontier world models we've covered (Frameboy/Atlas, PrimeSim).

Honest limits: it has only been tested on Doom, at 160×120, with static terrain (doors are left permanently open). The demo server is single-session and bound to loopback. Traction so far is tiny (single-digit GitHub stars). Treat it as a research blueprint for "engine-style" neural games, not a shipping toolkit.

Where to run it today

Same-day takeaway

PlayGen2 is the clearest open example yet of a neural game that keeps a real world state: voxels plus tracked NPCs underneath, with a renderer on top. If you're prototyping AI-native gameplay, clone it, play MAP01 on a laptop, then flip --layout-only to see the "engine" the model learned.

—Titus