Reinforcement learning (for robots)

Reinforcement learning is how you teach a robot a skill by letting it try millions of times, rewarding it when it gets closer to the goal and punishing it when it gets worse. It's how Unitree's H1 learned to run. It's how Boston Dynamics tuned Atlas's parkour. It's how DeepMind taught a humanoid to play football.

The recipe

You need three things to do reinforcement learning (RL):

1. An environment — usually a physics simulator (NVIDIA Isaac Sim, MuJoCo, Gazebo). Real robots are too expensive to break millions of times.
2. A reward function — a number that says how well the robot is doing. Higher is better. Designing the reward is the art of RL.
3. A policy — a neural network that takes the robot's current state (joint angles, sensors, target) and outputs the next action (joint torques, velocities). The policy starts random and improves over time.

The training loop: simulate an episode → record reward → tweak the policy's weights so good actions get more likely. Repeat for millions of episodes.

Why this is suddenly everywhere

Three things came together around 2018-2020:

• Big neural networks (deep learning) can represent complex policies that older RL couldn't.
• Fast GPU simulators can run thousands of robot lives per second on a single GPU.
• Sim-to-real techniques (domain randomisation, system identification) finally let policies trained in simulation work on real robots without re-training.

Before this convergence, RL was a research curiosity. After it, RL became how nearly every legged robot is trained.

Where it works (and where it doesn't)

Works well: locomotion (walking, running, recovering from pushes), dexterous manipulation (in-hand object rotation), drone flight controllers. Anywhere the goal is clear and you can simulate millions of attempts cheaply.

Doesn't work well (yet): open-ended tasks ("clean the kitchen"), long-horizon planning ("assemble a chair"), tasks involving humans (because humans behave unpredictably and can't be cheaply simulated).

For the open-ended problems, the field is moving toward VLA models — vision-language-action neural networks that combine RL with imitation learning from human demos. Tesla's Optimus, Figure 03, and 1X NEO all use VLA architectures.

Curious about the simulator side? NVIDIA Isaac is the most-used platform for sim-to-real these days.