Did AI make human Go players better?

Yes, by the best measure anyone has. Across more than 5.8 million moves by professional Go players from 1950 to 2021, the quality of human decisions improved comparatively little for about sixty-six years and then rose after superhuman Go programs arrived in 2016. Players also began making more moves nobody had played before, and those novel moves were associated with better decisions. A second study pinned down when the gain began: it followed the release of a free program that ordinary professionals could study with in February 2017, not the televised defeat of Lee Sedol a year earlier. Both studies judge human play by a machine's yardstick, and that is the limit every reader of them should carry.

Two professionals, two defeats#

In October 2015, AlphaGo beat Fan Hui, the European champion, five games to nil. It was the first time a program had beaten a professional on a full board without a handicap. In March 2016 it beat Lee Sedol, one of the strongest players of his generation, four games to one. His single win, in game four, turned on move 78, a move so unexpected that commentators called it divine. In November 2019 Lee retired, telling Yonhap News Agency: "Even if I become the number one, there is an entity that cannot be defeated."

That is the story most people know, and it reads as a story about humans being made redundant. The data on what happened to the people who kept playing tells a different one.

Sixty-six years of little change, then a rise#

Minkyu Shin, Jin Kim, Bas van Opheusden and Thomas Griffiths, writing in PNAS in March 2023, scored 5,857,513 moves from professional games played between 1950 and 2021. For each human move they asked KataGo, a superhuman Go program, what the best available move would have been, and compared the two moves' chances of winning. The smaller the gap, the better the decision. They call the result a Decision Quality Index.

Their finding, in their own words, is that "human decision-making improved comparatively little before the advent of superhuman AI", and that afterwards "human experts began to make significantly better decisions." From 1950 to 2016 is sixty-six years. The rise begins across 2016 and 2017.

More novel moves, and better ones#

The same paper looked at novelty: a move that created a sequence never seen before in the record, measured across the first sixty moves of each game. After superhuman programs arrived, novel moves appeared more often and earlier in games. And novel moves "on average contributed significantly more to the increase in decision quality than non-novel moves."

The obvious objection is that players were simply memorising what the machine plays. The authors tested it several ways. Taking out every human move that matched the program's choice, about 40 per cent of decisions, the improvement remained. Moves 21 to 50, where memorised sequences rarely survive, also improved. So did responses to opponents who departed from machine play. And novelty rose even among moves the program would not have made. Copying explains some of the change, but not all of it.

The authors are careful about the mechanism. Their suggestion is that superhuman play may have pushed professionals out of inherited strategies and into exploring, and that the exploring may have improved their decisions. "May" is their word, and the right one: the data shows the association, not the cause.

It began with study, not with the spectacle#

Watching AlphaGo beat Lee Sedol gave players a shock and a published paper. AlphaGo itself was never released for anyone to train with. What changed ordinary professional practice was a program they could actually use.

Sukwoong Choi, Hyo Kang, Namil Kim and Junsik Kim, in a paper accepted by the Strategic Management Journal, studied 749,190 moves by 1,241 professional players in 24,973 major games between 2015 and 2019. They tied the change to one date: the public release in February 2017 of Leela, the first freely available program stronger than top human players. After it, the gap between players' moves and the best move shrank by about 30 per cent. Errors fell by about a third and the size of the largest critical mistake by about a fifth. Players' moves matched the program's top suggestions more often, which the authors read as learning rather than live help.

Three details make the timing hard to dismiss. There was no improving trend before the release. A weaker program released in 2015 produced no improvement, so extra practice opponents alone do not explain it. And younger players gained more than older ones, as you would expect if the gain came from changing how people trained.

Leela was released free in February 2017. The community-trained programs that followed, Leela Zero from late 2017 and later KataGo, put superhuman analysis on every serious player's laptop. The study dates the change to the first of them.

What the studies do not show#

The yardstick is the machine's. Both papers grade human moves by how close they come to a program's choice. Part of what reads as better judgement may be play becoming more like the machine's. In Go that is a fair test, because the machine's evaluation is close to the truth about who wins. Most work has no such scorer.

The gains are largest where study pays most. Choi and colleagues found the improvement concentrated in the opening and shrinking later in the game. The opening is where preparation and recall carry most weight, which keeps open how much of the gain is deeper judgement and how much is better-prepared openings. Shin and colleagues' checks on later moves push against the second reading, but they do not settle it.

Nobody observed who used what. Neither study knows which player studied with which program, or for how long. The effects are averages over a profession, inferred from timing.

Go is a closed game. Fixed rules, a complete record, a clear winner and a program that can score any position. Those are the conditions under which learning from a superhuman system is easiest to measure. They are rare at work, so the result shows that it can happen. It says nothing yet about any office.

What expert judgement does after a machine surpasses it#

The question underneath this one is the question this research keeps returning to: what happens to human judgement once a machine is better at the task. Go gives the most hopeful measured answer so far, and it comes with conditions attached.

The machine was a training partner. In a professional game, consulting a program at the board is cheating. Players prepared with the machine and then decided alone, against an opponent, with the result counting. That is attempt first, check after built into the rules of a sport. Most workplaces are adopting AI the other way round, with the tool inside the decision itself.

The gain came from interrogating the machine. Fan Hui and Lee Sedol met it as opponents. The profession improved when thousands of players could put a position to a program, see a different answer, and work out why. Access to a tool you can question is what moved the average.

And the best moves were new. If the machine had only been copied, play would have converged on it. Instead professionals explored, and exploration paid. That is the difference this research calls design rather than drift: a deliberate choice about where the machine sits in how people learn, rather than a default in which it does the deciding.

The counter-case sits on other pages here. Experienced endoscopists who used AI during procedures got worse at detecting adenomas without it, which is deskilling, measured. Junior patent lawyers whose drafting improved with AI showed no lasting gain without it, which is the subject of whether AI makes juniors better or only look better. What separates those cases from Go is whether the human still had to make the decision unaided, and whether anyone was keeping score.