Computers are very good at the game of Go February 23, 2012 Theres an attraction between computer programmers and

the Asian game of Go. I think theres a lot to like about the game it has very simple rules, high comple ity !its "deeper# than chess$ and pleasing symmetry and aesthetics. I think the real reason programmers are so drawn to it might be a little more self%involved, though& being good at things that computers arent good at tends to make programmers happy, and computers are terrible at Go.' (r at least, thats the knowledge thats been true for most of my life. )omputers have always been worse than an average amateur with a few years of e perience, and incomparably bad to true professionals of the game. *omeone I was talking to brought up the ineptitude of computers at Go a few days ago, talking about new ideas for )A+T),As& just make the human solve Go problems, they said, and youre donecomputers cant do that, right. *o, Ive en/oyed this feeling of technological superiority to computers as much as anyone, and it hurts me a little to say this, but here I go& the idea that computers are bad at Go is not remotely true anymore . )omputers are e cellent at Go now. To illustrate this, theres some history we should go into& 0ack in '112 the year that 3eep 0lue beat chess world champion Garry 4asparov for the first time 3arren )ook asked )omputer Go enthusiasts for predictions on when computers will get to shodan !a strong amateur level$ and when theyll beat 5orld )hampion players. 6ohn Tromp, an academic researcher and appro imately shodan%level amateur, noticed the optimism of the guesses and wondered aloud whether the bets would continue to be optimistic if money were on the line, culminating in& 6ohn Tromp& I would happily bet that I wont be beaten in a 10 game match before the year 011! 3arren )ook took the bet for 7'888, and waited until 98'8 before conducting the match against the ":any ;aces of Go# program& Tromp won by < games to 8. That seemed to settle things, but the challenge was repeated last month, from 6anuary '=th%'>th 98'9, and things happened rather differently. This time Tromp was playing a rising star of a program named ?en'1, which won first place at the )omputer Go (lympiad in 9881 and 98''. The results are in, and& Zen1 !on by 3 games to 1" !;or more reading on the challenge, see 3avid (rmerods page or 3arren )ooks.$ 0eating an amateur shodan%level player is shocking by itself thats my strength too, and I wasnt e pecting a computer to be able to beat me anytime soon but that isnt nearly the limit of ?en'1@s accomplishments& its progress on the 4G* Go server looks something like this&

#ear 9881 98'8 98''

$G% &an' '%dan =%dan <%dan

98'9

A%dan

To put the A%dan rank in perspective& amongst the players who played American Go Association rated games in 98'', there were only '8A players that are B%dan and above. 9 This suggests that there are only around '88 active tournament players in the C* who are significantly stronger than ?en'1. Im sure Ill never become that strong myself.= 0eing able to gain four dan%level ranks in three years is incredible, and theres no principled reason to e pect that ?en'1 will stop improving it seems to have aligned itself on a path where it /ust continues getting better the more )+C time you throw at it, which is very reminiscent of the story with computer chess. Dven more reminiscent !and frustratingE$ is the techniFue used to get it to A%dan. 0efore I e plain how it works, Ill e plain how an older Go program worked, using my favorite e ample& GeuroGo. GeuroGo dates back to '11B, and has what seems like a very plausible design& its a hierarchical set of neural networks, containing a set of "D perts# that each get a chance to look at the board and evaluate moves. Its also possible for an e pert to override the other evaluators for e ample, a "Hife and 3eath D pert# module could work out whether theres a way to "kill# a large group, and an "(pening D pert# could play the first moves of the game where balance and global position are most important. This seems to provide a nice balance to the tension of different priorities when considering what to play. ?en'1, on the other hand, incorporates almost no knowledge about how to make strong Go movesE Its implemented using :onte )arlo Tree *earch, as are all of the recent strong Go programs. :onte )arlo methods involve, at the most basic level, choosing between moves by generating many thousands of random games that stem from each possible move and picking the move that leads to the games where you have the highest score- you wouldnt e pect such a random techniFue to work for a game as deep as Go, but it does. This makes me sad because while I wasnt foolish enough to believe that humans would always be better at Go than computers, I did think that the process of making a computer that is very good at Go might be eFuivalent to the process of acFuiring a powerful understanding of how human cognition works- that the failure of brute%force solutions to Go would mean that wed need a way to appro imate how humans approach Go before wed start to be able to beat strong human players reliably by implementing that same approach in silico. "pdate& "bitti# comments below that ?en'1 is choosing which move to play based on ma imiIing win probability, not ma imiIing score. I think that programs like ?en'1 have actually learned even less about how to play good Go than computer chess programs have learned about how to play good chess- at least the chess programs contain heuristics about how to play positionally and how to value different pieces !in the absence of overriding information like a path to checkmate that involves sacrificing them$. This lack of inbuilt Go knowledge shows up in ?en'1@s games it regularly makes moves that look obviously bad, breaking proverbs about good play and stone connectivity, leaving you scratching your head at how its making decisions. Jou can read a commented version of one of its wins against Tromp at GoGameGuru, or you could even play against it yourself on 4G*. "pdate& :atthew 5oodcraft comments below that ?en'1 does contain significant domain knowledge about Go. Its hard to know e actly how much- its closed%source. ?en'1 is beating e tremely strong amateurs, but it hasnt beaten professionals in games with no handicap yet. That said, now that we know that ?en'1 is using :onte )arlo strategies, the reason why it seems to be getting stronger as its fed more )+C time is revealed& these strategies are the most obviously paralleliIable algorithms out there, and for all we know this e act version of ?en'1 could end up becoming 5orld )hampion if a few more orders of magnitude of )+C time were made available to it.

5hich would feel like a shame, because I was really looking forward to seeing us figure out how brains work. %tephen %moogen said on February 23, 2012 at ()13 pm) :y understanding of how neurophysics has seen how brain methods fire off.. it is very much like how the Ien'1 works but a lot messier. 5hile we think we have one conscious decision making structure, we /ust have a high level virtual machine sitting on top of a ton of cheap parallel processors !lets say the K: is a =9 bit land but the processors are > bit and have been wired in various ways to look =9 bit$. ;or games like go and chess the visual processing parts of the brain make multiple guesses about how things will look if you make a move. (ne gets chosen by semi%random draw and sent up to the vm. The vm makes a move and then if the game works that choice gets weighted higher.. if that choice looses then the feedback tries to weigh it lower. ,owever since the brain has to use various wiring methods, it may or may not work. *ome people reach a level where they cant improve anymore and others dont. ;rom a programming standpoint this is a highly inefficient way to do things. 5e would rip out the > bit processors and use =9 or even B< bit. 5e would rip out bad programming and put in completely new programming. I think for computers to approach learning like a human, the programmers and engineers have to be stuck with various limitations that cant be changed out. Give them a billion > bit processors and circuits.. have them have to get them to interconnect and work. and then have them have an (* that is written in a different bit range and "thinks# it is talking to silicon that is that. Anyway, thanks for that info. I did not know about Go getting hit by silicon so Fuickly.