THE MECHANISM
Values, Bellman backups, and policy gradients
03Value is a forecast
→04Bellman turns a long future into one step + a forecast
→05Two broad routes
FAILURE → REPAIR
Common trap
“RL means trial-and-error with no prior data.”
Better model
Modern systems often start from imitation or pretrained representations, then use offline or online experience to improve.
YOUR NEXT GATE
Three rungs, not one giant leap
- 1Run value iteration on the ETH grid world.
- 2Train DQN on CartPole and inspect the replay buffer role.
- 3Run PPO or SAC only after the first two feel legible.