A Monte Carlo Tree Search based bot for the game of Connect4
I wanted to learn Reinforcement Learning so I thought this would be a good first exercise. Plus, the algorithm used in this project is present in MuZero and AlphaGo so really wanted to learn about it!
Previously I did a minimal version with a simpler game: tictactoe.
Connect Four is a two-player strategy game in which players take turns dropping colored discs into a vertical grid, aiming to be the first to form a horizontal, vertical, or diagonal line of four discs of the same color.
Monte Carlo Tree Search (MCTS) is a decision-making algorithm that builds a search tree by simulating random playouts from a given state, progressively refining its choices based on the results. It combines exploration of new moves with exploitation of known successful moves to find optimal actions. In particular, we employ the Upper Bound Confidence to set weights between these two.
Depending on the amount of computation budget that you give the program it may behave better or worse as it can have the opportunity to explore a larger space of possible states.
To Alfredo de la Fuente for letting us know about this project and for putting very good references. See his repo.