Introduction
One phone clip. No multi-cam rig. No calibration chart. No tripod prayer circle.
That is the dream behind 4DAnyone, a SIGGRAPH Asia 2026 open project from Zhejiang University, Robbyant, Ant Group, HKUST, and CUHK. Feed it a casual monocular human video and it synthesizes dozens of synchronized, view-consistent videos — the kind of multiview footage you normally need a calibrated camera array to capture — so you can push straight into downstream 4D Gaussian Splatting.
If you have ever wanted a free-viewpoint “anyone” from a single take, this is the week the toolkit got a lot more inviting.
Why this drop feels fresh
The research story is cool. The why now is practical:
- 4DAnyone-Turbo (released 2026-09-02) is the default distilled path, claiming about a 5.58× denoising speedup over 4DAnyone-Base while keeping generation quality in the same neighborhood.
- Peak CUDA memory is tuned for consumer cards — about 22 GB, comfortably under 24 GB after the 2026-09-05 update — so an RTX 4090-class GPU is in play.
- Timing on that class of hardware: roughly 27 seconds per 121-frame video on a single RTX 4090.
- A Gradio + Rerun GUI landed 2026-09-16 for interactive inference and visualization, so you are not stuck staring at CLI logs alone.
In other words: open weights, consumer VRAM, Turbo by default, and a viewer you can actually poke.
What it does (and what it does not ask of you)
4DAnyone’s bet is simple and a little delightful. Instead of reconstructing a photoreal 4D human only from a synchronized multi-camera dome, it generates the videos that dome would have recorded from one casual monocular clip.
Crucially:
- No input camera parameters are required.
- A static camera is not required (mild motion is fine).
- You do not need a pro capture stage to start experimenting.
The hard part at reconstruction scale is consistency across many views when compute forces generation into groups. The paper’s answer pairs two ideas:
- Reference Context Packing (RCP) — compresses a linearly growing reference context into a fixed budget.
- Target Context Routing (TCR) — routes context across disjoint generation groups so structure does not drift between batches.
Geometry conditioning leans on a 3D skeleton rather than fragile metric depth or noisy camera params — sparse, accurate cues that help the model generalize to in-the-wild footage.
Under the hood, the Hugging Face pack sits on Wan2.2 VAE / TI2V-related stack pieces (plus motion/pose helpers such as GVHMR assets), with both 4DAnyone-Base and the distilled Turbo LoRA available.
Input guidance that actually matters
The repo’s custom-data checklist is refreshingly honest. Aim for:
- a single person, full-body or upper-body
- mild camera motion, clear footage
- 1080p+, preferably 9:16 portrait
- at least 121 frames
Start with a compact 6-view orbit if you just want a smoke test; denser 24- or 48-view layouts (including multi-pitch layers) are there when you are serious about 4DGS coverage.
From generated views to Gaussian magic
Generated multiview videos are the bridge, not the finish line. For 3DGS, there is a nerfstudio export/train guide in the docs. Full open 4DGS (the team points toward FreeTimeGS-style reconstruction) is still on the roadmap — stay tuned — but the multiview synth step is already the part that used to be the capture bottleneck.
Weights live on Hugging Face at AntResearch/4DAnyone; code and news land on the GitHub repo (~1.3k stars and climbing).
Original Source
Project page and paper links live here — start with the interactive demos, then clone if you have a 4090-shaped GPU free: 4DAnyone
Conclusion
4DAnyone is one of those releases that makes the “multi-cam or nothing” era feel suddenly optional. Turbo + sub-24GB peaks + a fresh GUI will not magically invent perfect geometry from a blurry selfie video — but for creators and researchers who already shoot clean single-person clips, it turns monocular footage into a multiview playground worth exploring.
—Aurelia ♡