Lance Wicks

Abstract

An experimental relative skill based ranking system for elite level Judo
Overview
The RWJL ranking system has determined the following rankings, presented here in comparison with the official IJF Ranking list and RWJL as ofAugust 16th 2011.

JudoCoach.com, United Kingdom ( lw@judocoach.com ).

This research explains the theories behind and the progress on an experimental system of ranking elite level Judo athletes based on their relative skills rather than their results in competitions. Qualification for the Olympic Games Judo tournament is now subject to the IJF ranking list. The list is based on results in a series of elite level competitions. The IJF ranking list awards points to athletes based on the position the player achieves in the category results. The system being tested in this research looks at the players met in the competition then awards and deducts points to the players based on the result of each match and the relative skill levels (as indicated by their ranking points). The system being experimentally used is based on the ELO ranking system used in Chess and many other sports and online in systems like Microsoft's Xbox Live rankings. The experiment is for the duration of the qualification period for the 2012 Olympic games, May 2010 - May 2012 It is hoped that the constructive criticism from the Judo research community at the European Judo Championships will provide valuable input that will improve the ranking system. By providing an alternative to the current IJF system it is hoped that this research highlights the strengths and weaknesses of both systems. The experimental system has been running online at http://www.rwjl.net and uses information obtained from Mathias Fischer's TTA software at http://www.ippon.org without which the research could not be conducted.

The hypothesis of this research is concisely summarised as "Getting 3rd on a bye & 1 win is not equal to beating the current world & European champs to get 3rd" (De Mars, 2011). This research proposes a system that awards ranking points based on the opposition beaten as opposed to the position in the tournament reached.

Differences between systems

The RWJL system includes only results from 2010 Tunis World Cup onwards. However, by the time this research project concludes the dates should be in sync. RWJL ranking points are based on the relative skill levels of the athletes in each match and the probability of a player beating another. The IJF system by comparison awards points to players based on the position gained in a tournament.The IJF system factors in time by decreasing the value of points earned on an annual basis. The RWJL system does not currently decrease the points over time.The IJF system ranks players on their top 5 performances in a 12 month period, the RWJL system counts every match.

Research methods
This research is being conducted using bespoke software written specifically for the research and hosted on the www.rwjl.net website. The www.RWJL.net was developed on the Limonade-PHP application framework, the software collects data from www.ippon.org, the result of each match is used to determine the ranking points awarded to the winning player and deducted from the losing player. The amount added or deducted is based on the predicted probability of one player beating the other. The result is that if a player beats another player with more ranking points than them, they are awarded more points than if the player they beat had less points than them. Currently, this site is using a modified version of the Elo ranking system. Using the following modifications: K factor of 40 The losing player loses half the normal amount The starting value for all athletes is 1500 The following extract of code details the Elo algorithm used and shows how the K factor is a multiplier applied to the probablity of winning. A larger K factor provides larger changes in ranking, a smaller K factor provides a smaller deviation in ranking. Please note this code extract is for when player 1 won the match.
// we calculate the probability of one player winning (between 0 and 1), // the probability of the other player winning is // 1 this amount), multiply by the K factor $E = $K*(1(1/(1+pow(10, (($RankingPlayer2$RankingPlayer1)/400))))); $E = round($E); //Add this number to the winner's score $R['R3']=round($R1+$E/2); //subtract it from the loser's score $R['R4']=$R2$E;

Discussion

The IJF system provides an objective method for ranking players. The points awarded from an event do not factor in the level or number of athletes in the event. This is particularly noticeable when comparing continental championships. The size of categories and level of athletes in EJU championships can be argued to be considerably larger and harder than the OJU championships, yet points awarded are identical. The RWJL system has been generating outputs that have been of interest outside of the original area of investigation. Specifically, charting of player ranking points over time has been of interest in showing consistently performing players. Equally charting of number of athletes from each nation over time has provided a visual representation of the rate at which new players enter the elite level of Judo. Finally, aggregating and averaging ranking points by nation has provided a way of determining the strength of Judo across nations. All athletes enter the system with 1500 ranking points, this causes players who have competed regularly at the elite level, but lost, to be ranked lower than players entering the system. The top ranked and lowest ranked players may be more accurate than the middle ranked players as a result.

Plans for the future

Main points

Data is presented as tables listing each players country, weight category, current ranking points, fights won and lost. A chart is presented of the athletes total ranking points overtime (total at the end of each event in the Olympic Qualification series). Another table is presented which shows the result of every match the athlete has participated in along with the change in ranking points and new total. The RWJL system also presents data aggregated by category, nation, and for all athletes across all categories. This includes the ability to sort by ranking points, number of wins, number of losses or alphabetically. The aggregated view by nation also presents a chart of players from that nation on the system after each event providing a line chart of total number of athletes in ranking system over time. Names of athles are programmatically corrected for spelling mistakes, currently all names are corrected with "UTF-8, ISO-8859-1//TRANSLIT//IGNORE" coding and 64 corrections for mis-spellings of names have been included in the software.

The current RWJL software can be improved to include ideas from the existing IJF system and from outside sources, most notably the "TrueSkill" and "TrueSkill Through Time" methodologies developed at Microsoft. These methods provide an mathematical model for decreasing the value of points awarded over time, much like the current IJF system. They are however in general more linear and points are lost over a longer period of time, typically 40 months. Investigation needs to be taken into the optimal K factor and if the losing player should lose half the amount or not. Initial rank on entry to elite level needs to be considered and models from Chess and other areas will be considered for determining an initial rank as opposed to the current method of all tahletes starting with 1500 points. At present the RWJL assigns an athlete to a category once, upon their first entry into the syste,. There is not ability to adjust to a player changing category. This needs to be considered as a successful player may or may not continue to be successful in a different category. Research should be conducted into if "skill" as defined by ranking points should transition across categories or be confined to a single weight category. The RWJL research is a "work in progress" the software algorithms will be altered based on the feedback from researchers at this symposium and the IAJR symposium in Paris later in 2011. Your feedback is appreciated and vital to this research reaching a successful and useful result in 2012.

RWJL system awards ranking points based on "relative skill" of two players in a match. IJF System awards point sbased on final position in category. IJF system decreases value of ranking points awarded over time. RWJL currently does not. The RWJL research project is planned to finish in August 2012 after the London2012 Olympic Games Judo Tournament, a full report on the RWJL research will be written then. Acknowledgements:
Matthais Fischer for his TTA software on www.ippon.org without which this research could not be done. Dr. Mahana Clutha for her assistance with the mathematical algorithms. Hans van Essen for his help in linking to www.judoinside.com.