Forums / Community / Matchmaking Feedback & Discussion

[Locked] Matchmaking Feedback Update – May 7

OP ZaedynFel

  1. 1
  2. ...
  3. 2
  4. 3
  5. ...
  6. 13
Definitions
There are a lot of words below, and before we dig in let me define a couple of important terms so less people get lost.

MMR
Stands for Matchmaking Rating. It’s a term I coined long ago at a different studio when we first introduced the idea of having a separate, more accurate, backend rating that persisted season to season.

Ideally, it is the best prediction of how well a player will perform in their very next match so that we can matchmake them correctly, regardless of how we eventually rank them.
In Halo 5, we have also use it for driving our Ranking system, which will be explained more under “CSR” below.

Also, in Halo 5, we just replaced the original MMR system with TrueSkill 2, which is record-breakingly better at predicting player performance than the old system was.
This is a raw number we use on the backend to drive both matchmaking and Ranking systems.

We keep it separate from the number we Rank you on in Halo 5 under the philosophy that MMR is our best “prediction” of your performance, whereas “Rank” should be what actually happened, and a measure of if you can prove you deserve your MMR.

The older MMR system was less accurate, and so we required a lot of matches to get your “Rank” to what your MMR predicted it should be. We did this by artificially initializing ranks below the MMR after placement.

CSR
Stands for Competitive Skill Rating. This is the visible measure you receive that indicates your current skill-level. You see this as Bronze, Silver, Gold Platinum, Diamond, and Onyx, with Champ just being the top Onyx players. It is initialized after 10 placement matches using the MMR you had at the end of those placement matches. It then goes up when you win and down when you lose, based on the difficulty of each match.

In the past, we also have subtracted 100 (and way in the past 200) off your MMR when we initialize it as a CSR in order to encourage players to “prove” they deserve their true CSR.

CSR initialization used to come from our older MMR system, but with the TrueSkill 2 update, now comes from an MMR created by TrueSkill 2 instead. This MMR is much more accurate and needs very few games to assess a player’s skill. Therefore, it is less important to enforce “proving” you deserve the MMR, but probably still psychologically useful.

In addition, CSR updates after a match have always been computed by comparing your current CSR to the MMR (NOT CSR) of everyone else in the match. This ensures that CSR converges on your actual performance, and isn’t stagnated by 50/50, matchmaking. With the new update, the CSR vs. MMR computations are more accurate than ever. Players still must win to go up, and will still go down when they lose, but the amounts will better reflect each player’s actual abilities than in the past.

Ranking System Updated
As we announced and as many of you have seen, we have rolled out TrueSkill 2 into Ranked play. It is now being used for both matchmaking and Ranking. It directly decides placement, and then indirectly biases the CSR updating from match to match.

This means Placements may end up quite different than in the past because it’s a new system with new opinions about how good each of you are. It’s also a more correct system, so even if you get a different Rank than you expected, that Rank is based on a system that is better at predicting winners. See more below.

Also, while TrueSkill2 by far prefers players who win, it also incorporates kill and death information, as well as quitting behavior. This means even if you lose, it’s possible that your MMR will go up. So, you can lose even most of your placement matches and still get a good Rank if you had much worse teammates and still played well. Once you are placed, you can’t gain CSR for losing, but you can have your MMR go up. If this happens, you will get that back over time as CSR when you next start winning. It will also mean you will lose less CSR on your losses because if your MMR is higher, the CSR update assumes you were more responsible for the win, and less for the loss. Likewise, MMR can go down if you perform poorly compared to the rest of the players on your team, or in your party, resulting in eventual CSR losses.

This also means that it will be hard to get carried or boosted to a higher Rank. TrueSkill 2 will likely give your better teammates the better Rank, while giving you one closer to what your actual performance was. While this may hurt some egos, it will overall create a more fair experience for everyone. It means when you aren’t playing with your better friends, you don’t find yourself in overly difficult matches, and you also don’t misrepresent yourself to those players who play against you.


Usual -100 and 1700 Restrictions Removed for this Season
Normally, we place players 100 CSR back from where their MMR would place them after placement. We also usually restrict the max possible CSR to 1700 to force Champs to grind and earn their final ranks. All of this is artificial however, and often results in CSR underestimating a player’s actual skill.

This season, with the rollout of TrueSkill 2, we have removed this restriction. This means a Champ-level player can place directly into, e.g. 2400 right out of placement.

This helps us to stress test the Ranks a lot faster than waiting for players to grind it out, especially given not all players will take the time to do that.

We have heard feedback that this makes it feel like a lot of these Ranks haven’t been earned. We do empathize with this feedback because it aligns with our original reasons for enforcing the grind in the first place. We will highly likely reenable the restrictions in the next season.

That said, the current Ranks are still appropriate. TrueSkill 2 does not need 10 matches to assess a player’s abilities, unlike the older system. This means we don’t really need the artificial grind as much anymore. Part of the reason we had it was because we knew the older system couldn’t be precise enough in only 10 games, and the grind would ensure only the best really got there.

TrueSkill 2 doesn’t have this drawback. Every one of these players have turned in worthy performances over the course of their placement matches, and any one of them could be, e.g., Champ players.

But, again, we agree that it should feel more earned from a design point of view and will likely reinstate the previous restrictions.


I was Onyx for the last three systems, but now I’m Diamond!
The short answer is, the new system is right and the old one was wrong all along. The new system doesn’t know at all what the old system said about you in previous seasons. It looks at the same exact data (all of your previous matches) and makes its own judgement about your skill.

Some of you will find yourselves with higher ranks than before, and some lower. The new ranks are more accurate though, see below for more on that.
But I went W-L, had K/D, and got a different Rank than usual, and my friend got Onyx, etc.
The new Ranks are correct even if they don’t match your intuition. How do I know they’re correct? Here are two main reasons:
  • We take all the matches that happened over a period of time (e.g. 1 week, 2 weeks, 1 month, etc.) and ask both the new and old systems to tell us who should win. The new method is always significantly more accurate. The improvement in accuracy that we get in the new system is one of the largest improvements I’ve seen in this industry. Since the new method is more correct at saying who will win those matches, that means it is also more correct at saying who is the better player. Much more accurate. In addition, when we ask the new method how many kills each player will have and then average that over all our matches, the error is infinitesimal. Same for deaths. So, if TrueSkill 2 can both predict who will win and how many kills and deaths they will have, it must have a good grasp of how good each player is.
  • I have personally hand checked dozens of these placement situations and have yet to find one that is incorrect, usually to the embarrassment of the petitioner. If you post a request like this and I have time to investigate it, be advised I will post it publicly and it may not be what you’d hoped. I don’t do this to embarrass you personally, I do it because it’s educational for everyone. Every situation I’ve checked so far has shown players getting the number of kills TrueSkill 2 predicts and winning and losing as it expects. In addition, situations where players claim to have “played against Onyx” etc. have shown them playing usually against Platinum on average, and when they do play Onyx, they lose and / or have a significantly lower K/D.
So, again, I have yet to see an incorrect placement. TrueSkill 2 is heavily weighing current placements based on recent performance, tuned by seeing all matches since the beginning of time.


But I saw this one guy go 0-10 in his placement and still get Champ, how is that possible?

As discussed in the previous update thread, this happens when:
  • The player played amazingly pre-season.
  • The player throws his placement matches intentionally. When skillful players intentionally throw their matches, TrueSkill 2 ignores them completely and they don’t impact their placement matches at all. So, the player places at whatever Rank they were at pre-season. This combined with us removing the 1700 restriction this season, results in high placement. The placement is still correct. Those players really did turn in a solid Champ-level performance over a sustained number of matches. TrueSkill 2 is not only accurate, as we discussed, but also needs very few matches to identify player skill.
Keep in mind that when players do this, they are flagged by TrueSkill 2 as exploiting. We reserve the right to action those who do this. We’ve been relatively lenient so far this first season, but I can’t speak for how enforcement will choose to act in the future. Keep in mind they can ban not only a gamertag, but an entire console, and they can easily see which gamertags are associated with which consoles.


I’m in a full party and can’t find matches

Like all past Halo games, we now have a system in place to prevent full parties from playing solo players because they do have an advantage. So, if you are a high-ranking or high-skilled full party, and there are no other full parties around to play, you can end up waiting indefinitely --- just like in previous Halos.

The difference in Halo 5 is that TrueSkill 2 has correctly identified the skill advantage that parties of each size have, learned separately for each game mode. This means we can, in fact, match a full party against non-full parties and even solo players if those players are good enough to overcome the party advantage. In our data, we are seeing these non-full parties consistently defeat full parties. In fact, they win 50/50 whenever we can correctly matchmake them.

So, it’s basically a soft version of the same party restrictions Halo has always had and allows better matchmaking than ever in those situations.


But I'm a high-skilled player and played a full party recently while playing solo, and we got stomped

This can sometimes still happen in less popular playlists if you get “skill-capped” by the matchmaker. In those playlists, the matchmaker ignores skill above a certain level to make timely matches. If we don’t do this, even an all solo-player matchup won’t be possible at the top.

Absolute vs. Relative Ranks

In past seasons, we have made Ranking relative to the previous season’s skill distribution. This means your Rank could get worse even if your skill stayed the same. It could even happen if you got better over the season. This is because with a relative Rank, you will fall behind anytime the rest of the population improves faster than you do. It can also change as the population shifts season to season.

This season we are using TrueSkill 2’s suggested skill distribution. This is learned over the entire lifetime of each Playlist, so your skill is measured against the average player ever in that list. For most players, this means if you have improved in that playlist at all, your Rank will go up.

Most players in most playlists are better now than when that playlist first appeared, so their Ranks will also be better.

This means your Rank is now more reflective of your journey as a player. As you get better, it will go up, regardless of the recent population.

It does also mean that your CSR is no longer a pseudo-real-time measure of you percentile Rank among the whole population. A Diamond player used to be only the top 15% or so of the current players playing. Instead, it is now the top 15% of all time players in that playlist.

I think that’s OK personally. You will still be Ranked [above] people worse than you, and [below] those better, you’ll just all have a different label. The only place the relative ranking really matters is Onyx anyways, where everyone has a visible absolute value of their Rank and can stack rank themselves against the rest of Onyx. That stank ranking is still in the correct order.


Playlist Popularity

The new rotationals are fairing well as they usually do in their first weeks. What’s not apparent here but can be seen in the raw numbers is with the conversion of a few playlists to rotationals, the remaining lists at the bottom had better population numbers than they’ve seen in a while. Trimming the lists a bit has resulted in a slight improvement across the board.

Super Fiesta
Slayer
Warzone
Heroic Warzone Firefight
Shotty Snipers
Quick Play
[Big Team Battle]
Legendary Warzone Firefight
Infection
Triple Team
SWAT
Elimination
Team Arena
Doubles
HaloWC 2018
Mythic Warzone Firefight
Free For All
[Warzone Assault]
Action Sack
Breakout
Snipers
HaloWC FFA
Grifball
ZaedynFel wrote:
I have personally hand checked dozens of these placement situations and have yet to find one that is incorrect, usually to the embarrassment of the petitioner. If you post a request like this and I have time to investigate it, be advised I will post it publicly and it may not be what you’d hoped.
This is the grown up version of me posting people's game links back to them. :D

gonna be gud.
ZaedynFel wrote:
I have personally hand checked dozens of these placement situations and have yet to find one that is incorrect, usually to the embarrassment of the petitioner. If you post a request like this and I have time to investigate it, be advised I will post it publicly and it may not be what you’d hoped.
This is the grown up version of me posting people's game links back to them. :D

This is the TrueSkill equivalent of "I want to speak with your manager" while being the manager.
I don’t know if this unpopular, it may be. But what is the possibility of getting 1-50 ranks again.
I really enjoy the new skill matching. I feel as though I've had some really fun games. It's gotten me back into this game lately, especially Warzone.
ZaedynFel wrote:
  • I have personally hand checked dozens of these placement situations and have yet to find one that is incorrect, usually to the embarrassment of the petitioner. If you post a request like this and I have time to investigate it, be advised I will post it publicly and it may not be what you’d hoped. I don’t do this to embarrass you personally, I do it because it’s educational for everyone. Every situation I’ve checked so far has shown players getting the number of kills TrueSkill 2 predicts and winning and losing as it expects. In addition, situations where players claim to have “played against Onyx” etc. have shown them playing usually against Platinum on average, and when they do play Onyx, they lose and / or have a significantly lower K/D.
Where are these publicly posted, I'm interested to see the evidence. As someone who ranked Onyx for the first time under the new system, I'm more than satisfied, deserved or not. If you want another guinea pig, feel free to publicly post your investigations behind my ranking if you have time.

Thank you for all of the attempted clarity behind the new system, I've read every word in the buildup and it makes sense to me as far as I can understand it.
This in my opinion broke the social playlist... Now I'm matching people with higher MMR and getting absolutely stomped... In a social playlist, it supossed that those gamemodes are separated to have fun and chill out for a while... Not to get stressed and try hard on something everyone enjoyed
Elliminist wrote:
ZaedynFel wrote:
  • I have personally hand checked dozens of these placement situations and have yet to find one that is incorrect, usually to the embarrassment of the petitioner. If you post a request like this and I have time to investigate it, be advised I will post it publicly and it may not be what you’d hoped. I don’t do this to embarrass you personally, I do it because it’s educational for everyone. Every situation I’ve checked so far has shown players getting the number of kills TrueSkill 2 predicts and winning and losing as it expects. In addition, situations where players claim to have “played against Onyx” etc. have shown them playing usually against Platinum on average, and when they do play Onyx, they lose and / or have a significantly lower K/D.
Where are these publicly posted, I'm interested to see the evidence. As someone who ranked Onyx for the first time under the new system, I'm more than satisfied, deserved or not. If you want another guinea pig, feel free to publicly post your investigations behind my ranking if you have time.

Thank you for all of the attempted clarity behind the new system, I've read every word in the buildup and it makes sense to me as far as I can understand it.
I'd check out his twitter feed. :)
I don’t know if this unpopular, it may be. But what is the possibility of getting 1-50 ranks again.
Hopefully never because it's outdated. It was a poor representation of skill at the low and top ends and matchmaking suffered from its general limitations.
Elliminist wrote:
ZaedynFel wrote:
  • I have personally hand checked dozens of these placement situations and have yet to find one that is incorrect, usually to the embarrassment of the petitioner. If you post a request like this and I have time to investigate it, be advised I will post it publicly and it may not be what you’d hoped. I don’t do this to embarrass you personally, I do it because it’s educational for everyone. Every situation I’ve checked so far has shown players getting the number of kills TrueSkill 2 predicts and winning and losing as it expects. In addition, situations where players claim to have “played against Onyx” etc. have shown them playing usually against Platinum on average, and when they do play Onyx, they lose and / or have a significantly lower K/D.
Where are these publicly posted, I'm interested to see the evidence. As someone who ranked Onyx for the first time under the new system, I'm more than satisfied, deserved or not. If you want another guinea pig, feel free to publicly post your investigations behind my ranking if you have time.

Thank you for all of the attempted clarity behind the new system, I've read every word in the buildup and it makes sense to me as far as I can understand it.
Mostly on twitter.
ZaedynFel

Playlist Popularity

The new rotationals are fairing well as they usually do in their first weeks. What’s not apparent here but can be seen in the raw numbers is with the conversion of a few playlists to rotationals, the remaining lists at the bottom had better population numbers than they’ve seen in a while. Trimming the lists a bit has resulted in a slight improvement across the board.

Super Fiesta
Slayer
Warzone
Heroic Warzone Firefight
Shotty Snipers
Quick Play
BTB Unranked
Legendary Warzone Firefight
Infection
Triple Team
SWAT
Elimination
Team Arena
Doubles
HaloWC 2018
Mythic Warzone Firefight
Free For All
24p Warzone Assault
Action Sack
Breakout
Snipers
HaloWC FFA
Grifball

Kinda noticed you put BTB Unranked this week instead of Big Team Battle. Any reason for this? You guys planning on a BTB refresh and/or bringing back Ranked BTB?
Vluniverse wrote:
This in my opinion broke the social playlist... Now I'm matching people with higher MMR and getting absolutely stomped... In a social playlist, it supossed that those gamemodes are separated to have fun and chill out for a while... Not to get stressed and try hard on something everyone enjoyed
If you were a good player, you will lose more now, yes, but only because you are going from winning 60% down to 50%.

Meanwhile, the bottom players are going from 30% up to 50%.

We can't give you fun and chill games without giving someone else tryhard games or games they just get stomped in.

In order to be fair to the bottom, we have to be fair to the top.
The matchmaking across the board has been much much slower for me, even on Expanded setting. However, I am experiencing far fewer runaway victories, or blowout losses. I think if that's the goal, then mission accomplished. But I think I'd rather get games faster, rather than wait as long as I have been recently. And I'm on the East coast of US. Can't imagine what others are experiencing.
ZaedynFelPlaylist PopularityThe new rotationals are fairing well as they usually do in their first weeks. What’s not apparent here but can be seen in the raw numbers is with the conversion of a few playlists to rotationals, the remaining lists at the bottom had better population numbers than they’ve seen in a while. Trimming the lists a bit has resulted in a slight improvement across the board.
Super Fiesta
Slayer
Warzone
Heroic Warzone Firefight
Shotty Snipers
Quick Play
BTB UnrankedLegendary Warzone Firefight
Infection
Triple Team
SWAT
Elimination
Team Arena
Doubles
HaloWC 2018
Mythic Warzone Firefight
Free For All
24p Warzone Assault
Action Sack
Breakout
Snipers
HaloWC FFA
Grifball

Kinda noticed you put BTB Unranked this week instead of Big Team Battle. Any reason for this? You guys planning on a BTB refresh and/or bringing back Ranked BTB?
Mistake there, used the backend name by accident, I'll fix it.
I don’t know if this unpopular, it may be. But what is the possibility of getting 1-50 ranks again.
Have to agree with eLantern, I much prefer the current tiers as a representation of where you fall within skill levels.
There are equivalents that Dr. Menke posted a while back if you are interested:
https://www.halowaypoint.com/en-us/forums/58b8518e005f432381ab99fbcaf931e0/topics/matchmaking-feedback-update-%e2%80%93-march-27/7d7a1605-3aab-41ff-9950-95a9afbc29bc/posts?page=1#post2
Vluniverse wrote:
This in my opinion broke the social playlist... Now I'm matching people with higher MMR and getting absolutely stomped... In a social playlist, it supossed that those gamemodes are separated to have fun and chill out for a while... Not to get stressed and try hard on something everyone enjoyed
After looking at your arena stats it seems like you are a slightly above average player. Therefore there are going to be a decent amount of people who are better than you searching social. Social is just meant as a chill place, however for the players better than you chilling could be at the same level as you full out sweating and trying. Social isn't for trying super hard but it also isn't free wins. Think of the games you do well and how you barely trying could be equivalent to a gold or silver player really trying hard and still getting stomped. Its the same as your situation just more common since more people are better than them. If anything this system is now better at understanding your actual skill and trying to match within the social boundaries, which are pretty wide skill wise.
ZaedynFel wrote:
I have personally hand checked dozens of these placement situations and have yet to find one that is incorrect, usually to the embarrassment of the petitioner. If you post a request like this and I have time to investigate it, be advised I will post it publicly and it may not be what you’d hoped.
This is the grown up version of me posting people's game links back to them. :D

Is it wrong that I kinda want to be roasted?
ZaedynFel wrote:
Super Fiesta
Slayer
Warzone
Heroic Warzone Firefight
Shotty Snipers
Quick Play
[Big Team Battle]Legendary Warzone Firefight
Infection
Triple Team
SWAT
Elimination
Team Arena
Doubles
HaloWC 2018
Mythic Warzone Firefight
Free For All
24p Warzone Assault
Action Sack
Breakout
Snipers
HaloWC FFA
Grifball
Is there a reason that warzone assault is listed as 24 players ? Is there maybe a smaller version coming later ?
I've thought of this as a better understanding of skill. I've always excelled at Objective game modes more than slayer. Understanding the spawns and strategic value of objective gameplay has suited me back to Halo 2.

Because of this, I found myself ranked much higher in these playlists than Slayer ones. Elimination and Team Arena both started me at Champ #4, while Slayer kept me at the paltry 1780s.

You're right about it affecting the egos of people, because they sure weren't happy getting Diamond/low Onyx when I hit these higher ranks at placement. The matches are definitely closer and better organized, but I also felt the wrath of playing 2s in 4v4 Slayer as the only champ/onyx and running into a team of known high champ players still in placement matches. We were steamrolled lol. It happens a bit less now but even less often when you're with an actual team.
  1. 1
  2. ...
  3. 2
  4. 3
  5. ...
  6. 13