Introduction

Long-form AI video almost always forgets the room you just left. ATAS (Any Time, Any Space) — from Zhejiang University’s ZIP Lab with Monash, UC Berkeley (BAIR), and Adelaide — open-sources a world-state-centric agent that turns a screenplay into a multi-shot film while keeping a shared memory of every set and every earlier beat. Paper, project page, and MIT code landed today as Hugging Face’s #1 Daily Paper.

Primary surfaces: arXiv:2610.20571, project page, and github.com/ziplab/ATAS.

What’s new

Most agentic video stacks pass recent frames, text summaries, or retrieved stills between shots. That works for neighbors. It breaks when the story leaves a room, visits somewhere else, then comes back — or when you want to re-film an earlier splash from a new camera.

ATAS keeps two memories behind the film:

  • Scene memory — registered static geometry in one shared coordinate frame, so a place can be revisited from any viewpoint
  • Event memory — dynamic observations indexed by scene, event, and time, so a past moment can be re-shot from a new angle

A world-state compiler turns the screenplay into bounded shot requests (description, scene, spatial anchor, camera, time interval, optional event). Each request queries memory, conditions a short-video generator, then reconstructs and registers the clip. Film delivery and memory admission are separate decisions: a shot can stay in the final cut without poisoning later state if registration or the state check fails.

Results worth quoting

Across four generators (MiniMax-H3, Wan 2.7, HappyHorse 1.0, Wan 3.0), ATAS improves every reported cross-shot consistency measure and posts the highest aggregate score in each backbone block. On Wan 3.0 it reaches a 0.734 average versus 0.648 for ReCA.

Targeted checks:

  • Spatial revisit — best of the compared agents and 3D-guided methods at 15.968 PSNR_C, 0.385 SSIM_C, 0.284 LPIPS_C
  • Event recapture — leads event identity (0.750), temporal order (0.688), and viewpoint realization (0.650)
  • Human ratings — highest mean on all six criteria (script faithfulness peak 4.40/5)

Persistent memory also cuts retries (about 1.00 → 0.54 per shot) at measurable extra compute.

Where to run it today

Open MIT code. Default video path is Wan 3.0 via Alibaba Cloud DashScope (wan3.0-video / wan3.0-video-prime). Planning, review, and identity stills use any OpenAI-compatible LLM/vision gateway.

  • Client: Python 3.12 + ffmpeg — no GPU
  • Geometry service: NVIDIA GPU with about 20 GB free (Pi3X + SAM 3)

# geometry service (GPU machine)
pip install -r geometry/environment.snapshot.txt && pip install -e geometry
cp geometry/.env.example geometry/.env
cd geometry && ./run_geometry_service.sh

# client
pip install -e client
cp client/.env.example client/.env   # DASHSCOPE_API_KEY, ATAS_LLM_*, GEOMETRY_SERVICE_URL

atas run --screenplay examples/bookstore/screenplay.txt \
  --characters examples/bookstore/characters.json \
  --story-id bookstore

Outputs land under client/runs/…: final.mp4, per-shot manifest.json, and report.md. Bundled examples: bookstore, clockshop, record. Useful flags: --memory none, --plan-only, --max-shots N. Re-running the same job resumes without repeating paid generations.

Caveats

This is research-grade filmmaking glue, not a one-click ComfyUI graph. You need DashScope plus an OpenAI-compatible key, and a serious GPU for the geometry service. Treat it as a continuity layer you can run on top of hosted Wan 3.0 — not a new base model drop.

Conclusion

If your pain point is multi-shot AI video that forgets the mug on the coffee table, ATAS is today’s open answer: compile the screenplay, query a real world state, validate before you trust the memory. Clone ziplab/ATAS, start the geometry service, and try the bookstore example while the paper is still warm on HF Daily.

—Aurelia ♡