Reinforcement learning

Training by trial, error and reward rather than by copying labelled examples.

Instead of being shown the right answer, the system tries something, receives a score, and adjusts. It is how AlphaGo improved past the human games it started from, by playing millions of games against copies of itself. A variant is used on chatbots: people rank competing answers, those rankings train a scoring model, and the scoring model is used to tune the chatbot toward replies people preferred.

Where it mattered in the record

  1. Turing Award goes to Barto and Sutton for reinforcement learning

    The Association for Computing Machinery named Andrew Barto and Richard S. Sutton, once advisor and student, recipients of the 2024 A.M. Turing Award for developing the conceptual and algorithmic foundations of reinforcement learning. The $1 million prize is funded by Google.

  2. DeepSeek releases R1, an open-weight reasoning model

    The Chinese laboratory DeepSeek released R1 on 20 January 2025, an open-weight model trained with reinforcement learning to work through problems step by step. It matched OpenAI’s o1 on several benchmarks at a small fraction of the reported training and inference cost.

  3. OpenAI releases ChatGPT as a free research preview

    OpenAI published ChatGPT on 30 November 2022: a conversational interface to a model trained with human feedback and described as a sibling of InstructGPT. It was free during the research preview and reached through a web page, with no API key and no waitlist.

  4. Meta’s Cicero reaches human-level play in Diplomacy

    Meta AI announced Cicero on 22 November 2022, an agent that paired a language model with strategic planning and reinforcement learning to play the negotiation game Diplomacy. Across 40 anonymous games on webDiplomacy.net it more than doubled the average human score.

  5. DeepMind’s MuZero plays Atari, Go, chess and shogi without rules

    Nature published “Mastering Atari, Go, chess and shogi by planning with a learned model” on 23 December 2020. Julian Schrittwieser and colleagues at DeepMind described MuZero, which planned using a model it learned rather than rules it was given.

  6. OpenAI Five defeats the Dota 2 world champions OG

    OpenAI Five, a team of five neural networks trained by reinforcement learning, beat OG, the reigning Dota 2 world champions, 2-0 at a public event in San Francisco on 13 April 2019. OpenAI said the system had played the equivalent of about 45,000 years of the game in ten months.

  7. AlphaStar defeats professional StarCraft II players

    DeepMind said on 24 January 2019 that its AlphaStar system had beaten the professional StarCraft II players Dario Wünsch and Grzegorz Komincz, winning 10 of 11 games played under professional conditions in December 2018. It was trained by league play among versions of itself.

  8. AlphaZero learns chess and shogi from the rules alone

    DeepMind posted a paper describing AlphaZero, one algorithm that, given only the rules, reached superhuman play at chess, shogi and Go within 24 hours of self-play training and beat the reigning computer champion program in each game.

Every term in the glossary