#Go #AI #ArtificialIntelligence #ComputerGaming #BoardGames #Science Summary It's part 3 of our miniseries on teaching computers to play games. Today we're joined by special guest, Dr. Prithvi Akella, a roboticist and AI expert here to help us learn how to play Go, or at least how to teach a computer to do so. Timestamps 00:00 Introductions 02:20 Background on Go 06:52 Neural networks 09:50 Training the network 11:52 When (and how) computers won Go 18:38 Networks replacing brute force 21:31 Wrap-up Links Neural Networks, AlphaGo, and Alpha Zero (Wikipedia) Find our socials at https://www.gamingwithscience.net This episode of Gaming with Science™ was produced with the help of the University of Georgia and is distributed under a Creative Commons Attribution-Noncommercial (CC BY-NC 4.0) license. Splash image by Elena Popova via Unsplash https://unsplash.com/photos/a-close-up-of-a-board-game-with-black-and-white-balls-xdXxY5C9PUo. Full Transcript (Some platforms truncate the transcript due to length restrictions. If so, you can always find the full transcript on https://www.gamingwithscience.net/ ) Brian 0:06 Hello, and welcome to the Gaming with Science podcast, where we talk about the science behind some of your favorite games. Jason Wallace 0:11 In today's minisode about teaching computers to game, we'll be talking about Go neural networks and reinforcement learning. All right, everyone. Welcome back to Game with Science. This is Jason. Brian 0:23 This is Brian. Jason Wallace 0:24 And today we are on number three of our four-part miniseries on teaching computers to game. We're gonna be talking about Go and neural networks and deep reinforcement learning, and we have now officially gone beyond what I am capable of talking about on this show. And so we are joined by a special guest, Dr. Prithvi Akella, who is here to help us understand not only how we're training computers to play games, but how this actually applies to real life. So, Prithvi, could you please introduce yourself to our audience? Prithvi 0:50 Sure. Hello, everyone. My name is Prithvi. Pleasure to meet everyone, at least virtually. I finished my PhD from Caltech about three years ago. While I was there in grad school, I did a little bit of work in both learning-enabled systems with an emphasis on robotic systems. My specific focus was on trying to make these systems more robust, and now, as research scientist at Siemens, my goal is to apply these same methodologies in the robots that we put out in factories, and also for use in agentic systems that we're building internally as well. Jason Wallace 1:14 So, yeah, actually putting AI to use out in the real world, and so the colleague who introduced us mentioned, you've done some work recently on plants, right, which is the area that Brian and I work on. Prithvi 1:23 Yeah, so the work that we did with plants was with one professor at Berkeley, Ken Goldberg, and his lab. The idea there was, could we make 3D models in real time of plants for use in phenotyping and other identification aspects, specifically as it regards making sure and monitoring that plants are growing correctly, have certain markers, etc. things of this nature, Jason Wallace 1:42 and I could definitely use some of those. We have some traits that we measure in our lab that I've been going after a 3D scan of these plants for years, and we just don't have the skills to be able to put it together. Brian 1:53 A lot of the work on plants, we use this little model weed called Arabidopsis, which has the convenient thing of being very flat, so like you can just get a top-down image, and it's pretty good, but most plants, like what Jason works on, maize, there's a lot of verticality there, so like top down isn't going to pull it off. Jason Wallace 2:08 Yeah, and phenotyping is the process of actually measuring traits on plants, how tall it is, angles, colors, all sorts of stuff like that, any trait that we're interested in, really, Brian 2:16 blue eyes, red hair, you know, the classic plant phenotypes. Jason Wallace 2:20 All right, well, let's start talking about games. So, today's game is Go. Go is an ancient game, even older than chess. I think last time I said that chess was 1000s of years old. That's not quite true. It's more like 13, 1400 years old. Go, however, is 2500 years old, originally from China, and it's thought to be the oldest continually played board game. It even gets a mention in the Analects of Confucius, so it's an old game that is played on a board that traditionally is a 19 by 19 board, a grid where you place either black or white stones on the intersections. One player plays white, the other player plays black. You take turns placing them down, once they're down, they can't move, and your goal is to surround the other player's pieces and thus capture them, and to capture as much territory as you can on the board, the name Go, I'm not going to go all the way through the etymology, because it's complicated, but the name in original Chinese means essentially board game of surrounding, like you are surrounding your opponent and trying to capture them. Although professional Go is on 19 by 19, you can play on smaller boards, like 13 by 13, or even nine by nine, as a training board, that makes it easier, as far as learning goes, and pretty much the game goes until both players pass. As far as I'm aware, games generally don't go until you run out of spaces. They go until both players say, 'You know what, I'm good, I'm not going to be able to actually do anything better, or one concedes to the other. The reason we're talking about Go specifically is because Go is sort of the next evolution of hard games to get computers to play, so we talked about chess last time, and how this was the poster child of getting computers to play games up until like the mid 90s, when suddenly Deep Blue beat the world's best chess player, and that hurdle had been passed. In fact, I even remember way back in the Devonian, when I was in high school, I did a field trip with one of my classes to the local university, where we listened to some visiting professor talk about how Go was a better model for human cognition than chess, and he was arguing that when we got computers that could actually play Go, we would be much closer to understanding human neurology and psychology, or whatever. I don't remember all the details. I was 17 at the time, but it was basically Go is the better model to train on than chess, because Go is much more flexible. No piece is more valuable than another. The number of moves is much larger at any given point in a game of chess. There's maybe 30 to 40 moves you have to worry about, sometimes more, sometimes less. On go, it's closer to 150 to 250 and so there's more moves. Everything is very context dependent. How good a specific spot is on the board depends on the state of the board. There's probably a few spots that are slightly more powerful than others, but it's really very context dependent, and a move made at one point can have repercussions, 100 moves down the line, and so this is a very strategically deep game from a very simple principle, and I must admit I have not played Go, so I am not fit to talk about the strategies of it. I just understand from research that it is extremely deep, and the people who are really into Go, these world-class champions, are extremely good at it, and so once chess was vanquished, and once we basically had computers that could beat any human being at chess, the next obvious one was go. How do we do this? Because go, from the numbers I was throwing out, you probably figured out, is not really computationally tractable. We talked about how chess is not something that you can truly solve by brute force, that there are many more possible games of chess than there are atoms in the universe by 40 orders of magnitude. Well, for Go it's about 90 orders of magnitude. Jason Wallace 5:48 And I want to put this in context because we're not always good about explaining it. So when we say that the universe has 10 to the 80th atoms and that there are 2.1 times 10 to the 100 and 70th possible Go game states, that doesn't mean there's just over twice as many, that means there's 10 to the 90th universe's worth of atoms worth of go games. I looked at this number, it is 2.1 novemvigintillion. Brian 6:13 Jason, that's not a real word. Jason Wallace 6:15 it is a real word. Brian 6:16 All right, Jason Wallace 6:17 I have never heard of it before. Brian 6:19 Okay, Jason Wallace 6:20 there is some math nerd out there that has just gone and named everything as far as they can go, so anyway, so that's why go was the next level, and it pretty much was thought that it could not be solved by the same brute force methods that chess was, because the number of moves was too high, there were too many board states, and the value of the move is too hard to compute as far in the future as you need it. Master Go players do this intuitively. They are so experienced they can look at a board and they can intuit how things will play out, but we couldn't brute force a computer to do this. And so this then brings us to the next level of computation, which is neural networks and reinforcement learning. And now, Prithvi, I need you to do this part. Can you explain to us what is a neural network? Prithvi 7:03 Sure, I'll try my best. So, fundamentally, a neural network, like many machine learning models, is just one of multiple ways that we, as people who create machine learning models, try to fit or otherwise understand patterns that we see in general practice. So, specifically, with respect to neural nets, we define a neural net as one where, given an input, an input is just a vector of numbers. In this context, we apply a certain sequential set of operations to that vector of numbers, matrix op
