Introduction

Sony and KAIST just open-sourced FAR (Future-Aware Recall) — a trainable episodic-memory layer for video world models that learns which past frames to pull and which cues to trust before the future is known. The primary maker surface is the sony/far repo (code push 2026-10-01) with an H100 quickstart, plus checkpoints and LoopNav / SoundSpaces / AI2-THOR latents on Hugging Face. Paper: arXiv:2609.34677.

For game-gen teams, the headline corpus is LoopNav — a static outdoor Minecraft navigation benchmark — with additional interactive AI2-THOR household tours and audio-visual SoundSpaces indoor runs. This is not a playable browser toy; it is open research code + weights that teach a world model to remember the right history when it revisits a place.

What shipped

FAR sits outside the generator as an external retriever. During training it uses the realized future to score which candidate memories actually lowered diffusion prediction loss, then trains a future-blind scorer that fuses time, pose, vision, and (where available) audio with a learned per-query gate. At inference the future is unavailable — the retriever only sees cue embeddings and Top-K recall.

Runnable today from the README:

git clone https://github.com/sony/far.git far && cd far
bash bash_scripts/setup_env.sh
conda activate far
bash bash_scripts/quickstart.sh

The quickstart downloads FAR + WorldMem for AI2-THOR plus demo clips (~2.4 GB), rolls one revisit episode, and writes a side-by-side verdict video under results/quickstart/. Plan ~8 minutes and ~14 GB VRAM on one H100. Full checkpoints are ~13 GB via bash bash_scripts/download.sh checkpoints. License is CC BY-NC 4.0.

Why game makers should care

Hand-designed recall (recent frames, FOV overlap, visual similarity) fails in the places game worlds hurt: corridors with ambiguous geometry, containers whose contents changed off-screen, and NPCs that keep moving while the camera looks away.

Concrete LoopNav numbers from the paper / project page: at loop closure, FAR’s DreamSim is 0.082 versus 0.130 for WorldMem and 0.133 for LongLive-RAG — about 37% lower perceptual error than the best baseline. On AI2-THOR container reveals, FAR hits 75.7% state-correct renders versus 12.5% for WorldMem. The off-scene corridor probe (second agent patrolling while you are inside) jumps from ~42% temporal / ~32% WorldMem to 94.6% with FAR’s multi-cue agent signal.

That is the game-world model problem in one chart: revisit consistency and state-aware NPC/object memory, not prettier first-frame video.

Where it fits the stack

  • Open code + inference checkpoints — not an API-only drop.
  • Minecraft LoopNav redistributed latents (cite LoopNav) so you can reproduce the navigation memory results without regenerating the world.
  • AI2-THOR arms show interaction-dependent memory (put tomato in fridge → open later → tomato still there).
  • Baselines shipped in-repo: Temporal (NWM-style), WorldMem, LongLive-RAG — so you can A/B the retriever without rebuilding the DiT.

If you are wiring long-horizon memory into a discrete or continuous game world model (MaskGIT / diffusion / AR), FAR is a drop-in recipe for learned episodic access instead of another recency heuristic.

Limits

Quickstart wants a large GPU; this is research reproduction, not a WebGPU toy. SoundSpaces data is Matterport3D-gated. CC BY-NC blocks commercial redistribution of the released weights. The work studies recall, not memory writing / forgetting / compression — you still own the memory bank design.

Original Source

Conclusion

FAR is a clock-hot open release for anyone building persistent game worlds: Minecraft LoopNav + AI2-THOR proofs that learned multi-cue recall beats FOV heuristics on revisit consistency. Clone sony/far, run the quickstart, then steal the retriever ideas for your own engine loop.

—Titus