Q-learning is a reinforcement-learning method that learns the value of each action in each state purely from trial and error — no model of the world needed — and forms the basis of deep RL breakthroughs like the Deep Q-Network.
Q-learning learns a big table of "how good is it to take this action in this situation?" It fills the table in by trying actions, seeing the reward, and nudging its estimates — no instructions and no model of the world required.
How does a robot learn a good behavior when nobody tells it the right move and it doesn't even have a model of how the world works? Q-learning was one of the first clean answers — learn by doing.
What it learns
Q-learning estimates a function Q(state, action) — "how much total future reward can I expect if I take this action here, then act well afterward?" Learn Q accurately and the policy is trivial: in any state, pick the action with the highest Q.
It fills Q in from raw experience. After each step, it nudges its estimate toward the Bellman target — the reward just received plus the best Q available in the new state:
The gap between predicted and observed value (the TD error) drives each update. Repeat enough and Q converges to the true action values.
Why it was a big deal
Q-learning is model-free (it needs no knowledge of the world's dynamics) and off-policy (it can learn the optimal behavior even while exploring with random actions). Under mild conditions it's proven to converge to the optimal Q. That combination — learn optimally, from messy experience, without a model — is what made it a cornerstone of reinforcement learning.
The catch and the sequel
Classic Q-learning stores Q in a table, which is hopeless for robots: a camera image has astronomically many states. The breakthrough was to replace the table with a neural network that approximates Q — the Deep Q-Network that learned to play Atari from pixels. For continuous actions (a robot's joint torques), value-based methods give way to policy-gradient and actor-critic approaches.
Why it matters
Q-learning is the intuitive entry point to reinforcement learning — the idea that you can learn what's worth doing from consequences alone. Every deep-RL agent that grasps, walks, or navigates traces its lineage here.
