Reinforcement Learning in
Classic Board Games
David Silver
Plan
I. State of the Art
II. Minimax search
III. Reinforcement learning and minimax search
IV. Case Studies: Alpha-Beta Search
V. Case Studies: Monte-Carlo search
VI. Conclusion: Similarities and Lessons
Part I:
State of the Art
Why Classic Board Games?
Simple rules, deep concepts
Studied for hundreds (or thousands) of years
Meaningful IQ test - e.g. Elo rating in chess
Drosophila of artificial intelligence
Microcosms encapsulating real-world issues
Games are fun!
AI in Games: State of the Art
Level of Play
First Program at this Level
Checkers
PERFECT
Chinook
Chess
WORLD CHAMPION
Deep Blue
Othello
WORLD CHAMPION
Logistello
Backgammon
WORLD CHAMPION
TD-Gammon
Scrabble
WORLD CHAMPION
Maven
9x9 Go
GRANDMASTER
MoGo
19x19 Go
MASTER
Zen