Many researchers have advocated game domains as highly useful testbeds where one can cleanly isolate and study important issues faced by RL and more general AI methods in tackling messy real-world problems. In this talk, I’d like to survey some of the highlights of the numerous studies of RL in various game domains since Samuel's seminal work of fifty years ago. My definition of "games" is broad and will include puzzles, competitions, simulated marketplaces, and video/online games. I will also talk about the relationship and differences between traditional single-agent RL and more recent multiagent learning algorithms, which are likely necessary in general multi-player games. The goal of the talk is to draw a larger perspective on what we have learned from studying RL in games, and where promising future opportunities may lie, not only as RL theory advances, but as "games" themselves continue to evolve with advancing technology.