AlphaGo family of AI programs grew from AMS simulation-based algorithms developed at UMD

AlphaGo family of AI programs grew from AMS simulation-based algorithms developed at UMD

AlphaGo family of AI programs grew from AMS simulation-based algorithms developed at UMD


In recent years, the Go-playing artificial intelligence AlphaGo and its successors AlphaGo Zero and AlphaZero have made international headlines with their incredible successes in game playing. They are part of a line of AI systems developed to beat humans at games like Go, checkers, chess, Scrabble and Jeopardy. Each successive challenge extends the boundaries of machine learning and its capabilities. The programs have been touted as evidence of the immense potential of artificial intelligence, and in particular, machine learning.

At the core of AlphaGo and its successors are the ideas related to adaptive multistage sampling (AMS) simulation-based algorithms for Markov decision processes (MDPs) first explored by four University of Maryland researchers in a 2005 Operations Research paper. Now, one of the researchers, Professor Michael C. Fu (BMGT/ISR), has written “Simulation-Based Algorithms for Markov Decision Processes: Monte Carlo Tree Search from AlphaGo to AlphaZero,” a review of the original ideas and the ensuing developments in the Asia-Pacific Journal of Operational Research, Vol. 36, No. 06, 1940009 (2019).

The deep neural networks of AlphaGo, AlphaZero, and all their incarnations are trained using a technique called Monte Carlo tree search (MCTS), whose roots can be traced back to an AMS simulation-based algorithm for MPDs  published in Operations Research back in 2005.

 “An adaptive sampling algorithm for solving Markov decision processes”  was written by Institute for Systems Research (ISR) Postdoctoral Researcher Hyeong Soo Chang, Professor Michael C. Fu, Electrical and Computer Engineering (ECE) Ph.D. student Jiaqiao Hu, and Professor Steven I. Marcus (ECE/ISR). The idea was introduced even earlier in 2002.

In the current review article, Fu reviews the history and background of AlphaGo through AlphaZero, traces the origins of MCTS back to simulation-based algorithms for MDPs, and examines its role in training the neural networks that essentially carry out the value/policy function approximation used in approximate dynamic programming, reinforcement learning, and neuro-dynamic programming. Fu also includes discussion recently proposed enhancements that build on statistical ranking and selection research in the operations research simulation community.

Related Articles:
Maryland research contributes to Google’s AlphaGo AI system
LEGOLAS participates at U.S. Senate Robotics Showcase on Capitol Hill
UMD and NIST develop LEGO-based robots for teaching artificial intelligence
Manocha Receives 2022 Verisk AI Faculty Research Award
Exploring the 'rules of life' of natural neuronal networks could lead to faster, more efficient computers
How tech can fill gaps in mental health care
A new way to monitor mental health conditions
Clark School faculty 'AIM-HI' to address major health challenges
Advancing Healthcare through Robotics and Machine Learning
The Battery Revolution

December 17, 2019


Prev   Next

Current Headlines

Ph.D. Student Receives Best Paper Award at VFS 80th Annual Forum

Maryland Engineering: Top 10 Among Public Graduate Programs, Six Years Running

Registration Open for UMD/NFPA Fire & Life Safety Ecosystem Symposium

Teaching Students Specialized Skills for Success

Boyce Highlights Promise of Soft Composites

Maryland Engineering to Highlight Educational Advances at the 2024 ASEE Annual Conference

UMD Roundtable Weighs Lessons Learned From Key Bridge Collapse

Innovating in Engineering Education: Join Us at the 2024 ASEE Annual Conference

News Resources

Return to Newsroom

Search News

Archived News

Events Resources

Events Calendar