AlphaGo

I was fortunate to work on the team that created AlphaGo as well as its successors AlphaGoZero and AlphaZero - together with some incredibly talented individuals, and led by the brilliant David Silver. You can find more details about AlphaGo, its history, and further developments on the AlphaGo page at DeepMind and the AlphaGo Wikipedia page. You can watch me give a very short explanation of how AlphaGo works in the video How Google Deepmind AlphaGo works.

The original AlphaGo work is described in the paper Mastering the game of Go with deep neural networks and tree search. In our follow up work, we removed the necessity for human data and other heuristics, resulting in the even stronger player AlphaGoZero described in the paper Mastering the game of Go without human knowledge. In order to demonstrate that the methodology was not limited to the game of Go, but extended to other perfect information two player zero-sum games, we developed, AlphaZero, described in the paper A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play.

While the work up to this point had assumed that a simulator for the game was available, we recently extended AlphaZero to be able to estimate the transition model and rewards of a game, resulting in the MuZero algorithm described in the paper Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model. To demonstrate this new modelling capability, we show strong results on Atari games - a benchmark first introduced in the Arcade Learning Environment.

The wonderful director Greg Kohs made a wonderful feature-film length documentary about AlphaGo and its historic matches against Fang Hui and Lee Sedol, which is available on Youtube - AlphaGo. Watching it always makes me relive those incredible days in London and Seoul when AlphaGo was developed and made its mark.

The AlphaZero work has inspired a number of books including Max Pumperla’s and Kevin Furgeson’s Deep Learning and the Game of Go, for which I had the pleasure of writing the foreword. The book provides a wonderful introduction to the world of deep learning, reinforcement learning and tree search, all based on the game of Go. Also, Michael Redmond, professional 9 dan Go player, and Chris Garlock, who provided the engaging and competent on-site commentary for the AlphaGo vs Lee Sedol matches in Seoul, have written the book AlphaGo to Zero - The Complete Games, which is available here.

There are also some great videos with AlphaGo match commentaries on Youtube - a good starting point is the Youtube channel of the American Go Association (AGA), where Michael and Chris show some great commented games.

For chess fans, there is a wonderful book by Matthew Sadler and Natasha Regan with the title Game Changer: AlphaZero’s Groundbreaking Chess Strategies and the Promise of AI. It explores all the new chess knowledge that AlphaZero created, and puts it into the context of the play of human chess masters.

If you prepare videos, there are also some great AlphaZero chess videos on Youtube, for example Matthew Saddler’s commentary “Exactly How to Attack”. My favourite game is Alpha Zero’s “Immortal Zugzwang Game” against Stockfish - watch until the end!