A 12-lesson path from policies and imitation learning to a working ANIMA-Kiwi creature—grounded in ETH Zürich’s 2026 course, its guest spotlights, and a practical LeKiwi build sequence.
Take the lessons in order on your first pass. Each one adds only the machinery needed for the next. Watch the selected lecture slice, read the three-idea model, do one trace, then complete the lab or transfer prompt.
Camera frames and joint readings are what the policy receives. They may be incomplete or noisy; do not casually call them the full state.
02
A policy closes the loop
A policy π(a|o) chooses an action from the current observation. Running it creates the next observation, so errors feed back into future decisions.
03
An MDP is a modeling tool
Use (S, A, P, R, γ) when rewards and consequences matter. For imitation learning, demonstrations may be the more useful starting point.
Depth, when you want it
Markov means the chosen state contains everything needed to predict the next-state distribution given the action. It does not mean the world is deterministic.
Worked trace
Follow one loop
1
Task: place a mug on a coaster.
2
Observe: wrist image + joint angles.
3
Decide: a short action chunk.
4
Act: move joints and gripper.
5
Learn: compare the outcome with the demonstration or reward.
Common trap
“The camera image is the state.”
It is an observation. Hidden friction, object mass, and occluded geometry can still affect what happens next.
Practice
Paper-policy warm-up
20–40 min
Before moving on: changed-surface prompt
For a mobile robot asked to “follow me,” name one hidden state variable and one sensor you could add to make it less hidden.
Start with the simplest useful robot-learning recipe, then repair its two characteristic failures.
60 min guided pathGentle + one equation3 outcomes
Keep this loop in view
observe
decide
act
learn
Why this matters
Demonstrations are often the fastest route to a working robot. The trap is assuming supervised-learning accuracy is the same as closed-loop reliability.
By the end
Explain behavior cloning as supervised learning
Predict compounding error under covariate shift
Choose between better data, DAgger, and a multimodal head
Curated lecture slice
Watch the idea form
16:36 core clip
Lecture 3 introduces behavioral cloning, distribution shift, the compounding-error argument, DAgger, causal confusion, and multimodality.
Collect pairs (observation, expert action), minimize a prediction loss, and deploy the learned policy in closed loop.
02
Small errors change the data
At deployment, the policy visits states the expert did not demonstrate. One error can create unfamiliar states and more errors.
03
Two failures need two fixes
DAgger adds labels on learner-visited states. Expressive heads—mixtures, diffusion, or flow—avoid averaging distinct valid actions.
Depth, when you want it
If a one-step error rate is ε over horizon T, a loose behavioral-cloning bound can scale like O(εT²) because early mistakes alter later inputs. DAgger can improve the dependence toward O(εT) under its assumptions.
Worked trace
Follow one loop
1
Expert demonstrations stay on the safe side of an obstacle.
2
A cloned policy drifts a few centimeters.
3
The new view was absent from training.
4
DAgger asks the expert what to do there.
5
The recovery becomes training data.
Common trap
“Low validation MSE means the robot is ready.”
Validation from expert data tests prediction in familiar states. A rollout tests whether the policy can survive its own state distribution.
Practice
Feel covariate shift on a 2D toy
20–40 min
Before moving on: changed-surface prompt
You have budget for 20 new demonstrations. Which failure states should you deliberately include, and why?
Vπ(s) predicts discounted return from a state under policy π. Qπ(s,a) asks the same question after taking action a.
02
Bellman turns a long future into one step + a forecast
Immediate reward plus discounted value of the next state is the reusable recursive idea behind many RL algorithms.
03
Two broad routes
Value-based methods improve actions through Q estimates. Policy-gradient methods adjust the policy directly; actor–critic methods learn both policy and value.
Depth, when you want it
Bellman expectation: Vπ(s)=E[r+γVπ(s′)]. Policy gradient: ∇J(θ)=E[∇log πθ(a|s) A(s,a)]. The baseline inside advantage reduces variance without changing the expected gradient.
Worked trace
Follow one loop
1
The robot attempts a grasp.
2
Reward says whether the grasp succeeded.
3
The critic estimates which earlier decisions helped.
4
The actor shifts probability toward better decisions.
5
New experience checks whether the estimate was right.
Common trap
“RL means trial-and-error with no prior data.”
Modern systems often start from imitation or pretrained representations, then use offline or online experience to improve.
Practice
Three rungs, not one giant leap
20–40 min
Before moving on: changed-surface prompt
For a fragile real robot, propose one way to reduce unsafe online exploration while still benefiting from rewards.
Understand why robot actions are often distributions—and choose an action head by quality and latency.
65 min guided pathModerate, visual first3 outcomes
Keep this loop in view
observe
decide
act
learn
Why this matters
A single observation can have several correct futures. Generative action heads preserve those choices instead of predicting an unsafe average.
By the end
Explain diffusion and flow matching as noise-to-action transport
Compare ACT, diffusion, and flow on multimodality and latency
Select an action head for a constrained robot
Curated lecture slice
Watch the idea form
24:50 core clip
Lecture 6 builds from latent-variable models to diffusion policies and flow matching; the ANIMA action-model guide turns the tradeoff into a LeKiwi bake-off.
A conditional VAE produces a short sequence of actions at once. It is fast and a strong first baseline.
02
Diffusion iteratively denoises
Start from noise and repeatedly move toward a plausible action chunk. This captures multiple modes but can need many network evaluations.
03
Flow matching learns a velocity field
A deterministic path carries noise toward actions with fewer integration steps, which is attractive when the robot must keep moving.
Depth, when you want it
Diffusion learns a reverse stochastic process; flow matching regresses a conditional velocity field along a chosen probability path. Both can model multimodality, but their numerical sampling costs differ.
Worked trace
Follow one loop
1
Observation: object can be reached from left or right.
2
Noise represents an undecided action chunk.
3
The model conditions the path on the current scene and instruction.
4
One coherent route emerges.
5
The runner executes part of the chunk, then replans.
Common trap
“The most expressive head is always best.”
A head that misses the control deadline is not usable. Quality, latency, chunk horizon, and hardware are a coupled choice.
Practice
Action-head bake-off
20–40 min
Before moving on: changed-surface prompt
Make a three-column decision record for ACT, diffusion, and flow: success, latency, and operational risk.
Each token forms a query and retrieves useful information from keys and values. In robotics, tokens can represent language, vision, proprioception, or action.
02
Chunks make short-horizon commitments
Predicting several actions together reduces jitter and preserves coherent motion, but a long chunk reacts more slowly to surprises.
03
Compression is practical, not cosmetic
Robot trajectories are high-rate and redundant. Tokenizers such as FAST reduce sequence length so models can scale.
Depth, when you want it
Self-attention uses softmax(QKᵀ/√d)V. The equation matters less than the routing interpretation: a token can dynamically gather context from any earlier or current token allowed by the mask.
Worked trace
Follow one loop
1
Instruction tokens specify “pick the red cup.”
2
Vision tokens locate objects.
3
Proprioceptive tokens locate the arm.
4
Attention fuses the relevant context.
5
The decoder emits a short action chunk.
Common trap
“Action tokens make robotics identical to language modeling.”
The physical loop still has deadlines, continuous dynamics, safety constraints, and embodiment-specific action spaces.
Practice
Choose a chunk horizon
20–40 min
Before moving on: changed-surface prompt
At 20 Hz, what does a 50-action chunk imply? Would you execute all 2.5 seconds open-loop?
One-minute check
What is the central chunking tradeoff?
Correct. Longer chunks can be smoother and cheaper, while shorter chunks can react sooner.
Sources and optional depth
Build a useful mental model of world models: compress, predict, and improve behavior in imagination.
65 min guided pathModerate3 outcomes
Keep this loop in view
observe
decide
act
learn
Why this matters
Real interactions are expensive. A world model lets an agent reuse experience by practicing futures in a learned latent space.
By the end
Describe the representation, dynamics, and reward parts of a world model
Explain RSSM and imagined rollouts
Identify model-bias failure modes
Curated lecture slice
Watch the idea form
8:47 core clip
Lecture 8 develops recurrent state-space models and Dreamer; the ANIMA world-model page adds a small runnable lab and a fast/slow control interpretation.
An encoder turns high-dimensional pixels into a latent state that keeps information useful for predicting and controlling.
02
Predict through actions
A dynamics model estimates the next latent state; reward and continuation heads forecast consequences.
03
Learn in imagination, verify in reality
The actor and critic train on predicted trajectories, but real data must keep correcting model bias.
Depth, when you want it
An RSSM combines a deterministic recurrent path for memory with stochastic latent variables for uncertainty. Dreamer trains behavior on imagined latent rollouts rather than rendering every predicted frame.
Worked trace
Follow one loop
1
Encode the current camera frame.
2
Try several candidate action sequences in latent space.
3
Predict outcomes and values.
4
Choose a promising action.
5
Observe reality and update the model.
Common trap
“A good-looking video prediction is automatically a good control model.”
A control model must preserve action-conditioned, reward-relevant details—even if its decoded pixels are not photorealistic.
Practice
Dream a tiny world
20–40 min
Before moving on: changed-surface prompt
Name one real-world signal that a Kiwi world model must preserve even if the pixels can be compressed aggressively.
Understand the modern VLA recipe and the systems decisions hidden behind the model diagram.
70 min guided pathModerate3 outcomes
Keep this loop in view
observe
decide
act
learn
Why this matters
VLAs borrow broad visual and language knowledge, then add a robot-specific action interface. This is the bridge from foundation models to embodied behavior.
By the end
Sketch a VLA from inputs to action chunks
Compare discrete tokens and continuous action experts
Explain cross-embodiment and data-mixture challenges
Curated lecture slice
Watch the idea form
19:27 core clip
Lecture 9 covers generalist-policy ingredients and modern VLA architectures; the ANIMA VLA guide dissects SmolVLA and its continuous flow-matching expert.
A pretrained vision-language backbone supplies perception and instruction understanding; an action expert maps those features to motor commands.
02
Action interfaces differ
Some VLAs emit discrete action tokens. Others use a continuous diffusion or flow expert for precise action chunks.
03
Generalist does not mean embodiment-free
Different robots have different cameras, joints, delays, and action spaces. Co-training needs normalization, adapters, or embodiment-specific heads.
Depth, when you want it
The π0/SmolVLA family conditions a flow-matching action expert on backbone features. The expert predicts a velocity field that transports action noise toward data-consistent chunks.
Worked trace
Follow one loop
1
Language: “bring me the yellow block.”
2
Vision-language backbone grounds yellow and block.
3
Robot state supplies current joint and base context.
4
Flow expert generates a continuous action chunk.
5
Async execution keeps the body fed while the next chunk is predicted.
Common trap
“A large VLM plus a linear action layer is automatically a robust VLA.”
Action representation, temporal chunking, data balance, calibration, and deployment latency are first-class parts of the system.
Practice
Dissect SmolVLA before running it
20–40 min
Before moving on: changed-surface prompt
If you replace the arm but keep the cameras, which pieces of the VLA pipeline likely need new data or adaptation?
A slow process can decompose goals, inspect visual evidence, or compare candidate plans before issuing a skill-level command.
02
Control protects timing
A fast policy should continue safe, responsive behavior while slower reasoning runs asynchronously.
03
Evidence beats demo charisma
Ask about task distribution, number of trials, baselines, interventions, failure modes, and whether the robot had hidden assistance.
Depth, when you want it
Test-time scaling spends additional inference compute—sampling, search, critique, verification, or learned reasoning—to improve a decision. Robotics adds hard constraints: the world changes while compute is spent.
Worked trace
Follow one loop
1
User asks for a multi-step task.
2
Slow loop decomposes it into skills.
3
Fast policy executes the current skill.
4
Perception reports progress or failure.
5
Reasoner revises the plan without freezing control.
Common trap
“More test-time thinking always improves a robot.”
Reasoning can add latency, hallucinated subgoals, and extra failure surfaces. It must earn its place through measured task success.
Practice
Audit one frontier claim
20–40 min
Before moving on: changed-surface prompt
Turn “the robot understands cleanup” into a measurable claim with a test distribution and success criterion.
One-minute check
A planner takes five seconds while the robot is moving near a person. What architecture is safest?
Right. Slow deliberation should not own the hard real-time safety loop.
Sources and optional depth
Use one stable robot ID, record ports and package versions, and run the installed CLI help before copying commands.
02
Calibrate before collecting
Bad motor limits and mismatched IDs contaminate every episode. Treat calibration artifacts as part of the dataset provenance.
03
Replay before training
A dataset that cannot reproduce the demonstrated drive-and-grasp sequence should be repaired before any GPU time is spent.
Depth, when you want it
For Kiwi, record approach and grasp as one coherent episode when you expect one policy to perform both. Splitting them silently changes the task and data distribution.
Worked trace
Follow one loop
1
GPU host prepares training and serving.
2
Control host reads the leader arm.
3
Raspberry Pi runs the follower body.
4
Teleoperation produces synchronized observations and actions.
5
Replay verifies the stored episode on the real robot.
Common trap
“Training can smooth over a few broken episodes.”
Missing frames, drift, and inconsistent base motion teach the wrong dynamics. Repair the pipeline or recollect.
Build gate
Data gate
hands-on
Before moving on: changed-surface prompt
Define a rejection checklist for one episode: camera, joints, base, timing, task completion, and safety.
ACT is a fast way to prove the dataset, action shape, chunking, and evaluation harness before adding a VLA backbone.
02
Smoke-test before marathon training
Run a tiny job, load the checkpoint, and execute a short dry evaluation before paying for a multi-hour run.
03
Async serving needs a buffer invariant
Prediction must replenish chunks faster than execution consumes them; log buffer depth and underruns.
Depth, when you want it
Keep n_action_steps aligned with the checkpoint chunk length. Re-querying every control step can discard the coherence the action model learned and increase jitter.
Worked trace
Follow one loop
1
Train ACT and record baseline success.
2
Fine-tune SmolVLA from the pretrained base.
3
Load the latest checkpoint in a smoke test.
4
Serve chunks from the GPU host.
5
Measure task success, latency, and underruns over fixed trials.
Common trap
“Lower training loss selects the shipping model.”
Shipping depends on real success, recovery, latency, and operational stability—not only the optimization curve.
Build gate
Train with gates
hands-on
Before moving on: changed-surface prompt
Write a go/no-go gate for ACT versus SmolVLA using success rate, p95 latency, and underrun count.
Voice, basic safety, and small reactive motions continue at tight cadence. They do not wait for a long plan.
02
Slow loop adds meaning
The orchestrator, world model, and policy server reason about goals and longer outcomes asynchronously.
03
Events join the loops
Small messages—intent, phase, success, affect—coordinate subsystems without shipping every raw tensor through one process.
Depth, when you want it
Measure the VAD→STT→LLM→TTS cascade first. A speech-native duplex path removes the turn gate and text bottleneck, while an intent watcher stays a passive tap rather than a blocking stage.
Worked trace
Follow one loop
1
Voice stream hears “bring the yellow block.”
2
A passive watcher emits a structured intent.
3
Face acknowledges immediately.
4
Slow loop sends the language goal to the body policy.
5
Phase events update gaze, speech, and affect through completion.
Common trap
“A single synchronous pipeline is simpler.”
It is locally simpler but makes end-to-end latency additive and creates visible freezes when one stage slows down.
Build gate
Make responsiveness measurable
hands-on
Before moving on: changed-surface prompt
Design three events—acknowledged, grasping, and succeeded—and show how voice, face, and body react to each.
The orchestrator owns task phase; the body owns motor safety; the voice loop owns conversational continuity.
02
State machines make recovery explicit
Listening, acknowledging, acting, reporting, and failed states have defined entries, exits, and timeouts.
03
Demo quality is an evaluation protocol
Fix object placements, trial counts, timing logs, reset steps, and failure labels before recording the hero run.
Depth, when you want it
Treat slow components as workers behind queues. Bound message age, make commands idempotent where possible, and define what the body does if the network or policy server disappears.
Worked trace
Follow one loop
1
Listening emits a structured request.
2
Acknowledging gives immediate voice and face feedback.
3
Acting starts the async policy episode.
4
Body events update the creature’s expression.
5
Reporting states success or a bounded failure, then resets safely.
Common trap
“Once each track works alone, integration is just wiring.”
Shared clocks, stale messages, retries, ownership, network loss, and physical reset behavior are new system-level failure modes.
Build gate
Capstone runbook
hands-on
Before moving on: changed-surface prompt
Write the exact timeout and recovery behavior for one dropped policy response and one misunderstood spoken command.
Built as a private, source-grounded learning path. ETH Zürich materials and guest recordings remain the authoritative lecture sources. The 2026 slide PDFs link directly to the official ETH course files. Hardware work requires a clear workspace, reachable emergency stop, conservative speed limits, and supervision appropriate to your setup.