Inverse reinforcement learning flips the usual problem: instead of learning behavior from a reward, it infers the hidden reward from expert behavior — a way to teach robots goals that are far easier to demonstrate than to write down.
Inverse reinforcement learning watches an expert and works out what they must have been trying to achieve — the reward behind their actions. It's useful when a goal is easy to show but nearly impossible to write as a formula.
Normal reinforcement learning starts with a reward and learns behavior. But for many real tasks the reward is the hard part — how do you write a formula for "drive courteously" or "fold the shirt neatly"? Inverse reinforcement learning turns the problem around: watch an expert, and infer the reward they must be optimizing.
The idea
Given demonstrations of good behavior, IRL asks: what reward function would make this behavior optimal? Recover that reward, and you can then optimize a policy against it — including in situations the demonstrations never showed.
From behavior back to its goal
Instead of hand-writing the reward, IRL reads it off the expert's choices — then a standard RL policy can pursue that recovered goal.
Why it beats plain imitation
Simple imitation learning copies the expert's actions. IRL learns the expert's intent — and intent generalizes. If the robot drifts into a state the demos never covered, an action-copier is lost, but a reward-driven policy still knows what it's trying to achieve and can recover. This is why IRL is attractive for driving, navigation, and manipulation where you can't demonstrate every situation.
The hard parts
IRL is fundamentally ill-posed: many different reward functions can explain the same behavior (including the trivial "everything is zero reward"). The influential fix, maximum-entropy IRL, resolves the ambiguity by preferring the reward that explains the demos while assuming the least beyond them. Modern deep variants (adversarial IRL, GAIL) scale it with neural networks and connect it to generative modeling. It's also compute-heavy — classically it runs RL in an inner loop for every candidate reward.
Where it fits
IRL is the principled cousin of reward shaping: rather than hand-crafting a reward and risking reward hacking, you learn the reward from people who already do the task well. It underpins learning-from-human-preference pipelines used across robotics and beyond.
Why it matters
Inverse reinforcement learning addresses one of RL's deepest practical bottlenecks — specifying the reward. By reading goals from behavior, it lets robots learn objectives that are natural to demonstrate but hopeless to formalize.
