Is the ranking system facing a crisis?

Published 6 May 2020 by antti (last edited 26 Jul 2020)
tags: tldr

You are probably familiar with the phrasal verb ‘level up’, and just how rewarding those two words can be after spending some effort (or at least time) on a game. Although the phrase is now abundantly used in the gaming industry, it originally came from role-playing games – a precursor to the pen-and-paper roleplaying game Dungeons & Dragons, Blackmoor, is the reported origin of the concept. The American dictionary publisher, Merriam-Webster, currently does not have an entry for ‘level up’, but it is on their watchlist – and I would be surprised if the situation does not change soon. 

Why do so many video games these days employ experience points and level ups? The base is most likely in marketing: as anybody who has played such games knows, the phrase ‘level up’ simply feels good when you have spent effort in order to see it. This work-reward loop is the heart of many addictive games, and beautifully follows the core principle of Mihály Csíkszentmihályi’s Flow theory.

I am probably not the only person who finds that the ‘gather experience points in order to reach the next level’ cycle greatly resembles the improving process in go. Of course there are no experience points involved, but if we assume that a player must learn a certain amount of new things to reach the next stage and level of play in their go career, ‘gathering experience’ sounds as good a phrase as any. The same obviously applies to martial arts with rankings – and when we start to think of the whole ranking system concept in video game terms, I cannot help but suspect that the system was not at least partly introduced as a marketing ploy to get practitioners more invested in their sport. In go, of course, the ranking system also has the added benefit of helping determine the correct number of handicap stones used.

The belt colour system used in martial arts originates from Kanō Jigorō, the founder of jūdō, but martial arts ranking systems existed before this. It is widely accepted that the dan-kyu ranking system was originated by the go player Hon’inbō Dōsaku in the 17th century, although most likely the oldest go ranking system (and perhaps the oldest game-related ranking system anywhere?) was the Chinese Nine Pin Zhi. The latter, however, does not seem useful for enabling handicaps – indeed, no handicap system may even have existed at the time – but Dōsaku may well have been an early marketing genius.

Even if Dōsaku’s invention did not sound particularly opportunistic, its present-day usage in Japan is something else. While the Japanese professional go world uses a fair ranking system that depends on the number of wins that one gets in professional tournament games, the amateur system is quite different. One can earn amateur ranking diplomae by participating in regular ranking tournaments and getting a good enough score on a particular level – or else you can just buy a diploma with money. Roughly 10,000 euro will get you the top amateur rank of 8 dan! And of course, since people take pride in the ranking they have, rank demotions are out of the question (even though a player’s skill might deteriorate), so in Japan there is a constant ranking inflation going on.

Recently I noticed a new phenomenon when playing a training game on the Tygem server. A few years ago I used to be comfortably ranked 9d, but now I am firmly grounded at 8d. This has happened at the same time when AI-using players have increased in number on online go servers. (Fun fact: the Chinese say that such players are ‘taking the dog out for a walk’, where the word ‘dog’ is the same that is used to refer to AI.) Having done a good deal of studying with the AI, myself, I believe I can fairly consistently tell whether my opponent is using the AI: I can guess what the AI might play in a specific situation, and there is also a telltale ‘lag’ in an AI-user’s time usage (because they constantly have to switch over to the AI program, or else a second device). Since I can play with the AI on my own computer, too, recently I have just started resigning my games whenever the opponent’s cheating is obvious to me. I have limited time, and when I play go online I want to play with a human.

I used to think that the ranking system was there in order to help facilitate games between players of similar level, but also as a rewarding system (as per the ‘level up’ concept). With the arrival of AI users, however, I find that I no longer care if I get to 9d – in fact, that would probably increase the number of times that I get paired with AI users, so I may well be better off at 8d.

For a time, I thought the cheating phenomenon would be a problem just for top players, but these days I am not so sure.

As everybody is familiar with, recently the coronavirus epidemic has stopped all live go tournaments from happening. This leaves a hole in tournament-loving go players’ lives, so recently Corona Cup, a fully online tournament, was held. The tournament was immensely popular with roughly 400 participants, and there were prizes for the top performers as well as players with five or six wins in the lower ranking groups. Even though I believe most participants recognise that the purpose of participating in such an event is to enjoy the game with others in a playfully competetive setting, the ease of cheating will unavoidably generate some cheaters. After all, money prizes are involved, cheating is as easy as downloading a program, and the risk of getting caught is small or almost nonexistent.

When helping with the anti-cheating work at the Corona Cup, I came upon several important findings. Although there is basically no way to prove who cheated and who did not, it started looking highly likely that some ‘probable’ cheaters at the Corona Cup had also cheated in their regular online games – the difference between individual games by the same player was at times simply too big. This happened among players that were not even near 9d level, so ‘getting to the top rank’ could not be a motivation; in fact, I could not think of any motivation at all save for ‘winning just feels good’.

A second problem that turned up at Corona Cup was that many participants had registered with a ranking that did not really correlate with their playing strength. EGF rankings were used, so if a player had not played in a live tournament in a long time but had kept training online, then the player could be 6–8 stones stronger than their ranking might suggest. The problem is not made any easier by the fact that different countries use different ranking systems: in some countries, players are free to designate their own ranking, while in other countries players have to amass rating points and use the corresponding ranking. Even if a player was free to designate their ranking, they might be motivated not to rank up, as that way they would likely get five or six wins and win a prize due to having been underranked – which, obviously, is against the spirit of the whole event.

Things may have been different in 17th century Japan, but the modern trend seems to be that people crave short-term rewards; winning individual games and small prizes at tournaments probably feels more worthwhile to most people than the ambigous idea of ‘getting to a higher level of understanding’. (Although I do know people who belong to the second group; you have my sincere respect!) This effect is further amplified by the fact that rankings just do not feel as valuable as before. If anybody can get any ranking they want with the help of an AI, what value does that number have? And if you want to get an idea of what level you are at currently, how motivated do you feel to mingle with possible AI-using players to find out? 

Comments (5)

tam1998 wrote 2 years, 4 months ago:

'the modern trend seems to be that people crave short-term rewards'
That seems pretty true to me, according to some other evidence. Just compare video games that have been released before ~2000 and the average game thats being released today. Todays average game is probably a mobile game with a reasonable play time of a few hours (maximum?), and most games are incredibly simple to get through. Compare that to older games, where one could be stuck at a certain level time and time again without progress. (I personally remember how long it took me to finish the original Super Mario^^).
And that doesn't only hold true for games in general being more oriented towards 'mobile gaming', but also in certain subgenres. I especially like to compare the Sid Meier Civilization series, where Civ5 just feels like the game makes things happen for you to exploit (via dropping massive amounts of freebies), compared to Civ4 where you have to push the game to make things happen. And don't even compare that to playing the original Master of Orion^^
So i'd say with my slightly biased view, your statement is super accurate nowadays.

antti wrote 2 years, 4 months ago:

The Diablo series is also a great example of video game evolution. Diablo 1 (1996) felt at times merciless; Diablo 2 (2000) had what I’d call a near-perfect balance of gameplay; and Diablo 3 (2012), well ... I enjoy the game, but I certainly don’t feel challenged by it (loot piñatas, anyone?).
Recently I’m also amazed by the fact that an increasing amount of people seem to like watching other people play more than playing the game themselves.

devinfraze wrote 2 years, 4 months ago:

In a hypothetical world where all Go players across the planet had a sort of unified "Go profile" which was linked to their national and online accounts and which had records of all their games played, do you think that this could solve the ranking issue?

What if I told you I was working on such a platform? Assuming it was possible and that people actually used such an unified platform, how would you feel about it from the perspective of a strong player?

antti wrote 2 years, 4 months ago:

Leaving the question of feasibility aside, I am still not sure if a strict worldwide system would be a good idea. Personally I like the possibility of playing a training game online without people knowing it’s me playing; and I am sure I am not alone in this. Therefore, for me a global solution would still need to leave space for anonymous playing accounts.

Following, the main use of such a system would be to enable validating a player’s playing strength when the player in question (or a tournament organiser) wanted to. This would somewhat reduce the number of cheaters, since everybody would know they can only have one personal account, but I believe there would be a number of people who would keep cheating regardless. For example, a new player might decide to create a flashy online fake career where he zooms from 30k to 9d in two years, winning a number of online tournaments, before disappearing and quitting go completely.

I think the core issue is in who distributes the rankings. Right now I cannot see a way to create an automatic online ranking system that is not on some level prone to cheaters, so I think ‘serious’ rankings should only be determined through proctored live games or tournaments. Of course it wouldn’t hurt to have a global database of such rankings, similarly to the European Go Database.

This still leaves the problem of what should be done with people who don’t have a recent ‘serious’ ranking and would like to participate in a tournament. Right now these cases are handled flexibly by tournament organisers – at least if the player in question brings up the topic. (If the player stays silent, they can of course sandbag freely.)

I wonder if it wouldn’t be possible to create some sort of smartphone quiz app that can fairly accurately test a player’s understanding of the game. Then, when a serious player does the quiz in front of a tournament organiser, the organiser gets a fairly good idea of the player’s current strength – though sandbaggers could of course still intentionally cheat.

tam1998 wrote 2 years, 4 months ago:

I might have come up with an easy explanation for the 'People like watching others play the game more than playing themselves' thing...
If you play the game yourself, winning is fun but losing might be not.
However, watching someone else play, you can enjoy both winning and losing the same.