World Models // Predictive Observatory
0% completeCourse map

CHAPTER 02 / 07 · I · Foundations

Inside a latent world model

Understand the encoder, memory, dynamics, decoder, uncertainty, and the two clocks of training and imagination.

Core thesis
The architecture separates two questions: what state best explains the observation, and what state should follow when the observation is unavailable?
Target time
5 min explainer + 35–55 min study
Evidence
3 grounded records
Predict before the reveal: If a model can predict task-relevant futures but cannot reconstruct every pixel, is it still a world model? Write your criterion before reading.

ORIGINAL · NARRATED · CAPTIONED · DETERMINISTIC SVG

Deep visual explainer

Inside a latent world model

The course preview places the live HTML video above this page. It uses semantic SVG paths, exact narration ownership, captions, and seek-safe motion.

Open player
What the explainer teaches
  1. Compress what matters: High-dimensional observations contain both predictive signal and distracting detail. The encoder maps pixels, audio, proprioception, or text into a compact latent state. Compression is successful only if the state preserves what future decisions need. A visually beautiful reconstruction can still omit contact, velocity, or causal structure that control depends on.
  2. History carries hidden state: A single frame rarely reveals the complete state. The recurrent update combines the previous memory, previous latent state, and previous action into a context for the present. That context can remember motion, object permanence, commands, and occluded events. It is a learned belief state: fallible, compressed, and evaluated by whether it improves prediction.
  3. Posterior versus prior: During training, the posterior uses the observation to infer which latent state explains what actually happened. It is the model's evidence-conditioned belief. During imagination, the observation is absent. The prior predicts a distribution over next latent states from history. Training aligns prior predictions with posterior states without forcing every future to be deterministic.
  4. Reconstruct only with a reason: A decoder can reconstruct images or other targets, grounding latent states and making predictions inspectable. But reconstruction also spends capacity on details that planning may not need. The design test is downstream sufficiency. If reward, value, policy, or compatibility can be predicted reliably, a latent model may support decisions without rendering the future back into pixels.

CONCEPT MAP

The mechanism at a glance

observation xₜencoder E beliefhₜ , zₜ prior Fpolicy πdecoder D(optional)

OBJECTIVE 1

Map the roles of E, H, F, and D in a recurrent latent state-space model

Demonstrate this capability in the guided or transfer task; reading alone marks it only as introduced.

OBJECTIVE 2

Distinguish posterior state inference from prior state prediction

Demonstrate this capability in the guided or transfer task; reading alone marks it only as introduced.

OBJECTIVE 3

Explain why stochastic latent variables and reconstruction are design choices

Demonstrate this capability in the guided or transfer task; reading alone marks it only as introduced.

REFERENCE CHAPTER

Four functions, two modes

The encoder E turns the current observation and context into a latent state. The history update H carries information forward. The dynamics model F predicts the next latent state when no future observation is available. The decoder D optionally reconstructs observations or other targets. During training, observations anchor inference; during imagination, the model must run on its own predictions.E2, E5

Posterior and prior

The posterior asks, “given context and the observation I actually saw, which latent state explains it?” The prior asks, “given context alone, which latent state did the dynamics expect?” Training makes the prior predictive of the posterior. At deployment, imagined rollouts use the prior because future observations do not yet exist.

Why memory matters

A recurrent state can carry information across frames: the direction an object was moving, which drawer the key entered, or the command issued earlier. This is a learned belief-state approximation. If history is too short or the update forgets the wrong detail, planning becomes internally inconsistent.

Why stochasticity matters

Many futures are genuinely ambiguous. A pedestrian may stop or cross; a grasp may slip or hold. A stochastic latent variable can represent multiple plausible outcomes. Deterministic prediction often averages incompatible futures, producing a latent state that corresponds to none of them.

Do you need a decoder?

Reconstruction helps ground representations in sensory detail and makes predictions inspectable. But MuZero shows that a model can support planning without reconstructing raw observations. The right question is not “can it draw the future?” but “does its state preserve the quantities the downstream decision requires?”

The core failure seam

Representation collapse preserves too little. Over-complete reconstruction preserves distracting detail. Model bias predicts the wrong transition. Posterior–prior mismatch makes training look good while imagination drifts. A robust design audits all four seams separately.

GUIDED + TRANSFER

Use the model

Guided practice

Sketch E, H, F, and D for a robot pushing a block. Annotate exactly which inputs exist during training and which remain available during an imagined rollout.

Transfer

Design a latent state for an autonomous browser agent. Name one hidden variable history must preserve and one visual detail the model should discard.

Feedback rule: Name the state, action, prediction, objective, uncertainty, and real-world correction signal. If one is missing, the explanation is incomplete.

DIAGNOSTIC CHECKS

Can you reason with it?

CHECK 1 · recall

Which component must operate when the future observation is not yet available?

CHECK 2 · discrimination

Why can a decoder be optional?

CHECK 3 · transfer

A pedestrian can either cross or stop. A deterministic pixel predictor produces a blurred average. Which change addresses the core issue?

EVIDENCE LEDGER

Sources and limits

“World model” is used inconsistently across the literature. This course uses the anchor paper’s explicit–implicit distinction as an organizing device, while treating hybrids and task-specific operational tests as first-class.

  1. ev-world-model-operation A Tutorial on World Models and Physical AI · Il-Seok Oh / arXiv
    A generic latent world model can be decomposed into an encoder E, history update H, latent dynamics F, and optional decoder D. Imagined rollouts alternate policy actions with learned latent transitions instead of repeatedly querying the real environment.
  2. ev-world-model-design A Tutorial on World Models and Physical AI · Il-Seok Oh / arXiv
    Important design axes include state representation, deterministic versus stochastic dynamics, how prediction is used, and the training objective. The explicit–implicit distinction is useful but is not a claim that every system fits a perfectly clean binary.
  3. ev-dreamer Mastering diverse control tasks through world models · Hafner et al. / Nature
    DreamerV3 uses a recurrent state-space model to predict latent states, rewards, and continuation, while actor and critic networks learn from imagined latent trajectories. The published system reports one configuration across more than 150 tasks.