Hello everyone! I’m Shelt and I’ve been working at Stunlock Studios (SLS for short) since the very beginning. My title at SLS is Creative Director. It’s a very fuzzy role, but within game development it often ties into the overall design vision of the game and its features.
I’ve been given the honor to share some of my work and thought process with you guys for this Dev Blog post. This post will be focused around matchmaking and the new ranking system, where I will give you some insight into our vision, summarize our first season and what you can expect in the future.
Designing and implementing a matchmaking system is a super exciting task, but also one of the most difficult in skill-based competitive games. A small difference in skill can make a huge impact on the match balance. Also having to take estimated skill, visual placement, server regions, champions, archetypes, game type, ranking type, party size, queue times etc into consideration when matching players doesn’t make it any easier. Finally, we also have the player needs; some players gladly sacrifice match balance for faster games but others do not.
As many of you know we launched our first season and new ranking system that introduced leagues and divisions in December 2016. This system was designed to be more visually understandable, appealing and with improved rating changes.
During the season several concerns were raised by our community which we tried to tackle along the way. I’ll go through some of them to explain our reasoning and solutions.
“Players got very low placement results”
Our first intention was to place people a lot lower than their estimated skill to give them a guaranteed positive progression. We wanted to prevent players from getting a better placement position than their end result for the season.
This algorithm was very hard on the players and many of you felt like you were treated as the worst players in the world by being placed in low bronze, even though you felt good at the game. We completely understand this and a hotfix was made to increase the initial placement.
“Players often got matched with people outside their own league”
This is a very interesting topic. What we did before was to value our estimated skill of the player (their hidden mmr) more than their visual position (their League placement) when it comes to matchmaking. Because some players are more active than others, the visual position could vary a lot even if we estimated the players to be equally skilled.
We got a lot of complaints on bad matchmaking and we applied a hotfix which made the visual placement more important than our estimated skill when matching players. We got some great responses from that hotfix, telling us games finally felt a lot more balanced. The ironic thing is they actually became slightly worse. Here are some match result data from the Solo Ranked Queue taken before and after the hotfix was in place.
We were aware that the match balance would statistically fall, but as we also guessed, the visual perception played a big part in how players experience match balance. We will keep a close watch on both community feedback and the match result data throughout the season to determine adjustments needed going forward.
“Players needed to win 3 matches to compensate for 1 loss in terms of rating even though the match was equally balanced”
Our system uses the estimated skill of a player to influence the visual changes from a match. Higher estimated skill allows players to climb faster until the two values are almost matched, at which point you hopefully see similar rating gains and rating losses from a match.
In a lot of cases our algorithm for doing this didn’t work as intended and the estimated skill fell below the visual position. This made progression close to impossible. We’ve already made some hotfixes for this, but you can expect more improvements for season 2 as well.
“Ranked and unranked players got matched against each other a lot”
This feature was added to give us extra possibilities to match players if there weren’t enough suitable active players playing your ranking type. It’s a nice feature to have at this point in Early Access, but may be an unnecessary feature in the future when Free-To-Play brings an increase of players.
Unfortunately we had a bug in the system, which allowed these cross queue matches to happen a lot more frequently than intended. This lead to some unwanted results where match balance got a lot worse and ranked players could farm rating from unranked players. This was later hotfixed and should now be working properly.
“Too many players reached Grand Champion”
We’ve seen concerns regarding “too many people in Grand Champion” and that it is “too easy to reach Grand Champion”. In some extent it has been easier than intended. The previously mentioned bug regarding ranked players feeding on unranked players allowed some players to climb higher than our system’s estimation of their skill.
Grand Champion players have also lacked a reason to continue to play once they reach Grand Champion, which allowed less skilled players to progress. Since these players faced fewer Grand Champions it made it easier for them to climb further.
Other than that we are very happy with the split among leagues. This is how the current split looks like by looking at the Solo players:
Many of the bronze players would probably end up higher, but haven’t played enough games to progress.
Season 2
We’ve recently announced the start of Season 2, which due to technical issues has been rescheduled to the 15th of February. We are looking into the following improvements.
Distribution of divisions
We will be doing some changes to the amount of divisions within each league, making the amount of divisions between leagues more similar. This will give some slight improvements to the time it takes to progress through the early leagues.
Demotions
We will enable demotions, so players/teams no longer have safe spots once they reach a new league. This is to allow players/teams who have been inactive for a while to be demoted to their current accurate skill. We have also experienced some players/teams who start trolling or leaving games once they reach a new league. This change will also prevent this to some extent.
Grand Champion League
We want to make the Grand Champion League more interesting. To begin with we will be handing out titles as rewards to the top players/teams in each bracket (Solo, 2v2, 3v3) in order to add some additional reason to keep playing once they’ve reached the highest league. We have more plans to make Grand Champion more interesting, but that will come at a later stage.
Promote playing League
We will be looking into doing some UI tweaks to promote more players to play the League System. This will create a more balanced and appealing experience. Quick Match is intended to turn slightly more into a casual mode; where game balance isn’t as important and where players should feel less stressed to perform and more focused on testing a new champion.
Reworks to how Placement works
We will be doing some major changes to our system for placing players. New players, which we have no previous ranking information about, will start in a new league called the Contender League, which is below Bronze. We will monitor players/teams in this league to quickly progress them to their correct league. These changes will improve new players’ entrance to the League System.
Experienced players, which we have reliable data on, will be placed in a league directly but their first games will still have an higher impact on how we estimate their skill. Meaning two Diamond solo players, who decides to create a team, might start directly in Platinum/Diamond League. The reason for this changes is to reduce the hassle of placement games for experienced players and also remove the issue we had with highly skilled players stomping others through their placement games.
Estimated Skill and Visual Rating
As mentioned earlier we’ve been having some troubles with how these two values synergize. You can expect to see better and more correct rating changes for Season 2. This will always be an ongoing task, so expect additional improvements to rating calculations during the season.
Rating Decay
There have been requests regarding rating decay as some teams are becoming inactive in order to guarantee specific season rewards.
This is something we are looking into but it won’t be implemented at the beginning of Season 2 due to technical limitations. We want to implement something to solve this issue, but I can’t tell when it will be ready yet.
Final Words
We have an even greater vision of matchmaking and ranking system that can (and hopefully will) revolutionize your experience of competitive games. The base of this idea is that you as a player will have more control in the matchmaking process by adjusting your preference of queue time lengths compared to the likelihood of a balanced game. But I’ll save more about that for another Dev Blog 🙂
I want to give my best regards to everyone that managed to read this far and I’d like to thank you for taking an interest in this week’s Dev Blog.
See you in-game!
//Shelt