ORIGINAL · NARRATED · CAPTIONED · DETERMINISTIC SVG
Deep visual explainer
The course preview places the live HTML video above this page. It uses semantic SVG paths, exact narration ownership, captions, and seek-safe motion.
What the explainer teaches
- There is no single world-model score: A world model can be visually accurate yet uncontrollable, useful for short planning yet unstable over long horizons, or competent on average while dangerously miscalibrated in rare states. Evaluation must separate prediction, control, uncertainty, persistence, transfer, efficiency, and downstream decision quality. The metric suite should reflect how the model will actually be queried.
- Abstraction must change with time scale: Fast control needs geometry, contact, and velocity. Long plans need objects, relations, subgoals, and causal structure. Carrying every pixel forever is impossible and unnecessary. The open problem is hierarchical prediction: compress low-level dynamics into stable abstractions without discarding the detail that a later decision suddenly requires.
- Prediction does not choose purpose: Most systems receive a reward, instruction, demonstration, or goal image from outside. They can optimize or follow that objective without generating it. Autonomous goal formation asks how intentions are created, prioritized, revised, and constrained. A better predictor does not by itself answer what the agent should want.
- A testable hybrid, not a magic monolith: A plausible system combines broad implicit representations, action-conditioned latent dynamics, multi-scale planning, fast reactive control, and explicit safety constraints. Each interface should be testable. The honest frontier is not claiming a complete internal universe. It is building predictive contracts that remain calibrated, controllable, and useful under real feedback.
CONCEPT MAP
The mechanism at a glance
OBJECTIVE 1
Design an evaluation matrix covering prediction, control, uncertainty, hierarchy, and transfer
Demonstrate this capability in the guided or transfer task; reading alone marks it only as introduced.
OBJECTIVE 2
Explain why long-horizon abstraction and goal formation remain open problems
Demonstrate this capability in the guided or transfer task; reading alone marks it only as introduced.
OBJECTIVE 3
Propose a hybrid world-model system while clearly marking assumptions and safety limits
Demonstrate this capability in the guided or transfer task; reading alone marks it only as introduced.
REFERENCE CHAPTER
Prediction quality is multidimensional
Evaluate one-step accuracy, long-horizon consistency, action controllability, state persistence, calibration, counterfactual validity, transfer, data efficiency, computational cost, and downstream return or safety. A single perceptual metric cannot establish planning utility. A single control benchmark cannot establish broad world knowledge.E1, E3, E7–E8, E12
The hierarchy problem
Low-level prediction cares about contact, texture, velocity, and timing. Long-horizon reasoning cares about objects, relations, subgoals, and causal structure. Keeping every low-level detail across a long horizon is wasteful; discarding the wrong detail is fatal. Hierarchical world models need abstractions that change with the planning time scale while remaining mutually consistent.
The goal problem
Most current systems optimize predefined reward, imitate demonstrations, follow instructions, or plan toward supplied goal images. Autonomous goal formation asks a different question: how are objectives generated, prioritized, revised, and constrained? Prediction does not answer what should be wanted.
Unified explicit and implicit systems
A plausible architecture combines broad predictive representations from passive multimodal data, an action-conditioned latent dynamics module grounded by interaction, a planner that reasons over multiple time scales, a fast reactive controller, and a safety layer with explicit constraints. This is a research direction, not a solved blueprint.
Evaluation must follow the query distribution
World models are not used uniformly. Planners concentrate queries around high-value or adversarial branches; policies shift the data distribution as they improve. Evaluation should record where the system queries the model, how uncertainty changes, and whether errors correlate with decisions. Static held-out prediction is necessary but insufficient.
Capstone
Design a world-model agent for one domain. Specify its state, action, dynamics, uncertainty, objective, planner, feedback loop, safety constraints, source data, and evaluation. Mark every place where you rely on an assumption rather than evidence. The finish line is not a grand claim; it is an architecture whose predictive contract can be tested.
CURATED COMPANION VIEWING
Watch with a question
These external videos extend the chapter. Their demonstrations and arguments are contextual material; the primary evidence ledger below remains authoritative for technical claims.
EXTERNAL LECTURE · Google DeepMind
Genie 3: Creating dynamic worlds that you can navigate in real-time
Official demonstration for discussing real-time controllability, persistence, and evaluation limits.
Open on YouTube ↗GUIDED + TRANSFER
Use the model
Guided practice
Create an evaluation table with columns for metric, stress test, failure threshold, and evidence source; include at least prediction, action control, calibration, transfer, and safety.
Transfer
Draft the capstone architecture for a household robot, autonomous browser, or driving assistant. Mark explicit, implicit, reactive, and safety components.
Feedback rule: Name the state, action, prediction, objective, uncertainty, and real-world correction signal. If one is missing, the explanation is incomplete.
DIAGNOSTIC CHECKS
Can you reason with it?
CHECK 1 · discrimination
Why is next-frame accuracy insufficient as a world-model metric?
CHECK 2 · application
What makes hierarchical prediction difficult?
CHECK 3 · transfer
Which claim is best supported by current evidence?
EVIDENCE LEDGER
Sources and limits
“World model” is used inconsistently across the literature. This course uses the anchor paper’s explicit–implicit distinction as an organizing device, while treating hybrids and task-specific operational tests as first-class.
- ev-world-model-frontiers A Tutorial on World Models and Physical AI · Il-Seok Oh / arXiv
The tutorial identifies unified explicit–implicit modeling, hierarchical prediction and planning across time scales, and autonomous intention or goal formation as major unresolved directions. These are research hypotheses and gaps, not demonstrated AGI capabilities. - ev-world-model-definition A Tutorial on World Models and Physical AI · Il-Seok Oh / arXiv
World models are organized around predictive structure: explicit models expose learned dynamics for rollouts and planning, while implicit models encode predictive regularities inside learned representations. Both aim to support reasoning beyond reactive control. - ev-world-model-design A Tutorial on World Models and Physical AI · Il-Seok Oh / arXiv
Important design axes include state representation, deterministic versus stochastic dynamics, how prediction is used, and the training objective. The explicit–implicit distinction is useful but is not a claim that every system fits a perfectly clean binary. - ev-vjepa2 V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning · Assran et al. / arXiv
V-JEPA 2 pretrains an action-free joint-embedding predictive model on large-scale video, then post-trains an action-conditioned latent world model with robot video. The paper demonstrates image-goal planning on robot arms without task-specific reward training in the deployment environments. - ev-genie Genie: Generative Interactive Environments · Bruce et al. / arXiv
Genie learns controllable generative environments from unlabelled Internet video by combining a video tokenizer, latent action model, and dynamics model. It illustrates a generative route to action-conditioned world simulation.