Forums / Community / Matchmaking Feedback & Discussion

[Locked] Matchmaking Feedback Update – April 10

OP ZaedynFel

  1. 1
  2. ...
  3. 2
  4. 3
  5. ...
  6. 4
CSR Update Change
A change went in last week that fixes two issues with how we update your CSR
  • We were comparing a deflated version of CSR to the opponent’s MMR. This resulted in too much CSR on a win, and too little removed on a loss. This has been fixed. Players with already inflated CSRs will see a steady net loss while their CSR restabilizes at the correct value. In FFA, this also fixes an issue where 4th and 5th place would earn CSR even on a loss. Those players with FFA CSR inflated because of this will lose CSR even in 2nd place until their CSRs float down to where they belong. There will still be some cases of earning CSR even in 4th place, etc., but only if, e.g., the 5th and 6th place players have much higher MMR than the 4th
  • In general, we consider FireTeam MMR in the CSR update. This meant that if you had a high MMR or CSR, it was better to not play with other high MMR players because it would always make the CSR update small --- even at the beginning of a new season. In addition, this rewarded high MMR players teaming with highly-skilled but lower MMR players to artificially inflate their CSRs. We have changed this so that very high MMR players can now team up with their highest-skilled friends and earn more CSR at the beginning of a season. In addition, players trying to team up with much lower MMR teammates will end up not getting much CSR.
#1 means the CSR system will deflate everyone a little and more accurately reflect your actual skill. Most playlists won’t notice this much, but higher-skilled FFA players probably will.

#2 means smurf boosting no longer works well at the Champion level. It also means Champions can earn more CSR playing with other Champions, especially at the beginning of the season.

#2 also has some implications for Champion-level players playing with their noob friends. Basically, something has to give. Extremely high CSR players playing with low ones causes systemic problems, even when done honestly.

Many other competitive games put a hard block on high ranked players playing with low ones. Instead of doing that, we'll allow you to play together with the same matchmaking experience, but with the softer penalty of losing some rank if you’re at or near the Champion level.

If a Champion level player decides to play with their noob friends, their Rank will descend gradually to the average skill between the Champion and who they play with. Which I think it a fair compromise.

Or, said another way, if you want to keep your Champion-level Rank, you have to play with Champion teammates, which I also think is fair.

Overall, this update also means that CSR now more accurately reflects the skill of our players than ever before. There will still be some lingering inflation this season, but I'm hoping next season results in the best ranks we've seen yet. This does mean that the higher Ranks like Onyx and even to a lesser extent, Diamond, will be harder than ever to attain, but I think that's for the best.

What about a party-up option?
We like it, but I don’t think it’ll get in any time soon.

What will happen if a high Onyx player parties up with a legitimately Silver player?
This totally fine, we don’t mind it at all. Don’t worry about this if you are doing it legitimately.
If the gap gets too big, then you will see something like this:
  • Matchmaking is the same as today.
  • The lower-skilled players’ CSR will move like normal
  • The higher-skilled players’ CSR will go down towards the middle of the two MMRS.
Basically, instead of straight up blocking you from playing together, we just make it harder to rank up in the extreme cases. Most of the time it will be fine though. As I said above, if you really want that top Champion spot, you need to team with other Champions.

Why don’t we show population counts?
This question has come up on several different game teams that I have been a part of. There’s this careful balance because showing a number tells players where they should play. This means if the number is healthy, the list stays healthy, but if the number is low, people might not try it and it becomes a self-fulfilling prophecy of sorts.

On one hand, I like setting expectations about a playlist’s wait time, but on the other hand there’s this argument that low numbers don’t give the playlist a chance to grow.
A common compromise I’ve seen for this is to give approximate wait times, but that has some of the same issues.

I don’t remember 100% how we landed on the current implementation, but it’s something we still discuss.

Low pop region matchmaking
I am looking at how we can improve this within our current framework. Searching expanded is currently your best bet. I can’t currently change the allowed skill gap based on location, and if I could, it would mean you couldn’t leave those places once the skill gap expanded. So it would do this:
  • Try worldwide at the normal skill gaps, not find enough folks
  • Try regionally at a wide skill gap, but still not find enough because there aren’t enough at this particular time in the region.
I could just widen the skill gap overall, but that would mean allowing, e.g., Gold vs. Onyx some of the time. There may be a way to balance both of these though, so I am still investigating.

HCS Settings and Matchmaking
Some of you have probably been following the HCS settings discussion. I’ve been fortunate enough to have been included in the internal process on this, so I may answer some questions on that, but except for playlist and matchmaking-related questions, it would be over on this thread and not here.

What playlists will the new HCS Settings affect?
The current plan it to put the new settings only in Proving Grounds, and we will evaluate how well they work out. That evaluation will include feedback from you as a community as you play on those settings, the HCS players as they also play on them, and also watching the data itself. We may also discover there are few more adjustments we can make to improve them, and then we'll evaluate those as well. If the evaluation suggests these settings are better overall for everyone, we could see some of them migrate into Team Arena, and maybe other lists. If not, then we'll consider the best place for them at that point. I think the ideal is to always have the HCS settings in at least one playlist. If it turns out the HCS settings are by far the best for the HCS scene and players, but aren't working out for even the highly competitive portion of the mainstream, that may suggest we would need to maintain both lists as has been done in past Halo titles.
Glad to see features being implemented to prevent smurfing. I really appreciate the continued support and updates on the issue.

The whole deflating CSR thing kinda seems like a cheap fix to me. I find little motivation to actually play the game knowing it will be impossible to progress at all. Not to mention the deflation started several days before any explanation leaving a lot of players confused and discouraged. I know it's a difficult situation, but it definitely could have been handled better.
ZaedynFel wrote:
CSR Update Change: Champion level player decides to play with their noob friends, their Rank will descend gradually to the average skill between the Champion and who they play with. Which I think it a fair compromise.

What will happen if a high Onyx player parties up with a legitimately Silver player? This totally fine, we don’t mind it at all. Don’t worry about this if you are doing it legitimately.
If the gap gets too big, then you will see something like this:
  • Matchmaking is the same as today.
  • The lower-skilled players’ CSR will move like normal
  • The higher-skilled players’ CSR will go down towards the middle of the two MMRS.
Basically, instead of straight up blocking you from playing together, we just make it harder to rank up in the extreme cases. Most of the time it will be fine though. As I said above, if you really want that top Champion spot, you need to team with other Champions.
Overall regarding the CSR Update Change I liked what you said, but I do have a specific question that somewhat relates to the idea of deranking. Let's say that a champion level player has some realisticly bad people they like to play with so they realize playing on their primary account isn't a smart method to keep their championship CSR while spending time in matchmaking with their legitimately bad friends or family members therefore they create a second account -- I imagine this is the ideal situation.

Under this new method of calculating CSR within the system that second account's CSR is obviously going to get pulled down or be less than their champion level primary account because of those who the champ level player is playing with and against, but what about when that champion level player takes his or her secondary account and starts solo queing with it? Will he or she be accurately matchmade per their true skill level or will the suppressed CSR (and potentially suppressed MMR) allow them to absolutely pub-stomp groups of players who are below their actual skill level? I assume so, but the question then becomes how long would it take for the system to properly re-adjust them to play with and against people more accurate to their true skill level? I guess what I'm asking is whether this change makes a player's MMR take more of a precedence in the search system than their CSR so that situations like the one I just laid out won't easily allow pub-stomping for an extended period of time. Also, how could the system accurately indicate their proper MMR in the initial few matches of solo queuing if they've been mainly matching with and against lower level players thanks to the legitimately bad players who they play with bringing the competition level way down? It would be a bit of a process until the system can accurately place them against people of their true skill level, right?
I wanna know why on saturday I played FFA and won but my CSR did not go up; not even one point.
I'm gonna play later and try to win to taste how it is going, but, losing against unranked-lvl10s-and ending second with 12 kills to see my CSR drop down feels like BS.
Haven't played much since the update rolled out last week but last night I ran with a couple buddies, I'm usually Onyx and my friends are usually diamonds and then grind to onyx by the end of each season.

We had this game (https://www.halowaypoint.com/en-us/games/halo-5-guardians/xbox-one/mode/arena/matches/9bfe39c3-bdde-45a3-99b6-eeb2be417235/players/jolly%20josh?gameHistoryMatchIndex=6&gameHistoryGameModeFilter=All).

A champ running with an unranked (who is most likely also a champ) and facing our group of onyx's and diamonds.

I noticed throughout the night that I'd drop like 17-18 Onyx ranks when we lost to opponents whose average skill was higher than ours based on the CSR (Obviously IDK their hidden MMR). Then, we would beat teams whose average skill "appeared" higher than our average skill and I'd only get a minimal +12 Onyx rank increase. references below. https://www.halowaypoint.com/en-us/games/halo-5-guardians/xbox-one/mode/arena/matches/8d998460-f58d-4a3b-9cb2-29140444e9cc/players/jolly%20josh?gameHistoryMatchIndex=10&gameHistoryGameModeFilter=All
https://www.halowaypoint.com/en-us/games/halo-5-guardians/xbox-one/mode/arena/matches/7875560e-2589-49c3-ac35-41ee31dad6e4/players/jolly%20josh?gameHistoryMatchIndex=3&gameHistoryGameModeFilter=All

Am I being penalized for playing with players lower rank than me or is this only really affecting the champ/gold smurf types?

I tended to lose more than I was gaining while generally losing and winning against competition whose average skill appeared higher than my teams average skill. If this is the case shouldn't a win be worth more than a loss?

We ended up losing 4/10 and I began the night at Onyx 1805 and ended at Onyx 1817.
eLantern wrote:
Overall regarding the CSR Update Change I liked what you said, but I do have a specific question that somewhat relates to the idea of deranking. Let's say that a champion level player has some realisticly bad people they like to play with so they realize playing on their primary account isn't a smart method to keep their championship CSR while spending time in matchmaking with their legitimately bad friends or family members therefore they create a second account -- I imagine this is the ideal situation.

Under this new method of calculating CSR within the system that second account's CSR is obviously going to get pulled down or be less than their champion level primary account because of those who the champ level player is playing with and against, but what about when that champion level player takes his or her secondary account and starts solo queing with it? Will he or she be accurately matchmade per their true skill level or will the suppressed CSR (and potentially suppressed MMR) allow them to absolutely pub-stomp groups of players who are below their actual skill level? I assume so, but the question then becomes how long would it take for the system to properly re-adjust them to play with and against people more accurate to their true skill level? I guess what I'm asking is whether this change makes a player's MMR take more of a precedence in the search system than their CSR so that situations like the one I just laid out won't easily allow pub-stomping for an extended period of time. Also, how could the system accurately indicate their proper MMR in the initial few matches of solo queuing if they've been mainly matching with and against lower level players thanks to the legitimately bad players who they play with bringing the competition level way down? It would be a bit of a process until the system can accurately place them against people of their true skill level, right?
The matchmaker only looks at MMR. We have tested CSR matchmaking in the past and it was measurably inferior. Both in terms of the skill gap and player engagement. MMR is just a better measure of who should play vs. whom.

Currently, we have a few tricks to separate a superior player in early matches. This means if a Champ makes a new account and plays with his noob friends, his MMR will still be higher than theirs, and as they keep playing together, it will rise significantly. So for the most party, when the Champ solo queues, things will be "OK" as long as they don't keep creating new accounts. I think we'd like to eventually make creating lots and lots of new accounts less profitable.

There will still be some cases where we don't currently catch folks like this fast enough under the current implementation. We are working on an update that can separate out a high-skilled player from his noob friends much more efficiently than we can now, so we are working to address that as well. Once that goes live, we'll evaluate its effectiveness and see if there's more to be done there.
Thanks for the update. One question I have is what did you learn about the wait time between starting the next search for a game? Will a feature be implemented? Or is the plan to leave it be?
Why I understand not showing population counts for playlists. It would be nice to know what the general population is. Some of my mates call halo a dead game, while halo tracker shows an increase in ranked players. But Halo trackers isn't precise. I would love to see just a population statement, doesn't have to be live updated. Just give us a general statement on how big the populations.

Mostly because I know my mates have issues finding because of their internet.
Apologies for this not being about competitive, but I can't not ask:

Along the same lines as testing and matchmaking updates, could we PLEASE get some feedback on the status of potential AR/mag starts in BTB? At this point I'm assuming there has been some kind of internal testing of this, because people have definitely been wanting this for the better part of a year. Is there a population concern? Did testing show it didn't balance well? Is there a technical issue that the community couldn't have known about?
Jolly Josh wrote:
Haven't played much since the update rolled out last week but last night I ran with a couple buddies, I'm usually Onyx and my friends are usually diamonds and then grind to onyx by the end of each season.

We had this game (https://www.halowaypoint.com/en-us/games/halo-5-guardians/xbox-one/mode/arena/matches/9bfe39c3-bdde-45a3-99b6-eeb2be417235/players/jolly%20josh?gameHistoryMatchIndex=6&gameHistoryGameModeFilter=All).

A champ running with an unranked (who is most likely also a champ) and facing our group of onyx's and diamonds.

I noticed throughout the night that I'd drop like 17-18 Onyx ranks when we lost to opponents whose average skill was higher than ours based on the CSR (Obviously IDK their hidden MMR). Then, we would beat teams whose average skill "appeared" higher than our average skill and I'd only get a minimal +12 Onyx rank increase. references below. https://www.halowaypoint.com/en-us/games/halo-5-guardians/xbox-one/mode/arena/matches/8d998460-f58d-4a3b-9cb2-29140444e9cc/players/jolly%20josh?gameHistoryMatchIndex=10&gameHistoryGameModeFilter=All
https://www.halowaypoint.com/en-us/games/halo-5-guardians/xbox-one/mode/arena/matches/7875560e-2589-49c3-ac35-41ee31dad6e4/players/jolly%20josh?gameHistoryMatchIndex=3&gameHistoryGameModeFilter=All

Am I being penalized for playing with players lower rank than me or is this only really affecting the champ/gold smurf types?

I tended to lose more than I was gaining while generally losing and winning against competition whose average skill appeared higher than my teams average skill. If this is the case shouldn't a win be worth more than a loss?

We ended up losing 4/10 and I began the night at Onyx 1805 and ended at Onyx 1817.
No, I don't think you're being affected by the update here. Unless you were used to getting a little more CSR than you were supposed to. But looking at your MMR on the backend and comparing it to your CSR, you are not inflated. You're about where you should be so far.

+12/-18 means that, looking at your CSR vs. their MMR, you are playing opponents that you beat 60% of the time (which, probably coincidentally, lines up with your exact win% for the night, hah hah). +30 is the max, so if you take (+12/30) you get 40%. This is because if you beat a team it expects you to beat 60% of the time, you get (Actual - Expected) = (100 - 60) = 40% of the update.

If we can only find 60/40 opponents for you, we give out a bit less CSR so you break even going 60/40, rather than going up. Basically, if you perform at the level of your CSR, whether that's 50/50 or 60/40, you stay where you are. If you outperform it, you go up, underperform, down.
SCDF Zero wrote:
Thanks for the update. One question I have is what did you learn about the wait time between starting the next search for a game? Will a feature be implemented? Or is the plan to leave it be?
It was really informative, but I don't think we'll see changes based on that for awhile.
What happens to the other side? When you soloQ and match against a champ and a noob, chances of winning will still be slim... what will happen to the CSR of the losing side so a plat 3 matched with anorher plat 3 losing to a bronze and a champ?
Good to see that people ranking down for 2nd and 3rd in FFA is just part of their CSR evening out, and not just a permanent change across the board. I was worried it would end up hurting the population even more.

Having said that, I still REALLY would like to see FFA 2nd and 3rd places count as wins on the service record. It's the small things that motivate people to remain invested in that hopper for much longer. No one likes just racking up losses, even more so newcomers who might just be learning to get familiar with it. Even HCS tournaments reward top 3, and so do most FFA tournaments with a reasonable prize pool. No real reason for Matchmaking not to do the same.
MJBfox wrote:
What happens to the other side? When you soloQ and match against a champ and a noob, chances of winning will still be slim... what will happen to the CSR of the losing side so a plat 3 matched with anorher plat 3 losing to a bronze and a champ?
So far, across our data, when a legit Champ and a legit Bronze play together, the win% are consistent with the MMR model. So treating them as Plat so far has resulted in good predictions, and the correct CSR updates.

It's the cases when you are play against a Champ and someone who is not actually a noob that results in that "slim chance" of winning, and those are the cases we are making less effective.
Why don’t we show population counts?

I think the not showing population counts isnt the best even titanfall 2 shows population counts and they have like 20k players online at a time. I understand if the numbers are sort of low and 343 doesnt want to give the information if they dont really like it. Though I feel like the population is very healthy especially for how many playlist there are plus i find firefight matches literally in 3 seconds no matter when im playing which is good!

I think we could maybe do a % of players for this? like in Ranked section split the % between people playing in the ranked playlist then do for social then do for warzone?
When we said we wanted a population counter, we didn't mean for all playlists. We meant for the overall population of the game. Like when you go into Arena, its overall population will be like 5,000 or when you go into Warzone, its overall population will be 7,000. It just seems very weird how every Halo game before Halo 5 had a population counter while Halo 5 doesn't. The Halo Community would love to know how many people play Halo 5.
ZaedynFel wrote:
CSR Update ChangeA change went in last week that fixes two issues with how we update your CSR
  • We were comparing a deflated version of CSR to the opponent’s MMR. This resulted in too much CSR on a win, and too little removed on a loss. This has been fixed. Players with already inflated CSRs will see a steady net loss while their CSR restabilizes at the correct value. In FFA, this also fixes an issue where 4th and 5th place would earn CSR even on a loss. Those players with FFA CSR inflated because of this will lose CSR even in 2nd place until their CSRs float down to where they belong. There will still be some cases of earning CSR even in 4th place, etc., but only if, e.g., the 5th and 6th place players have much higher MMR than the 4th
  • In general, we consider FireTeam MMR in the CSR update. This meant that if you had a high MMR or CSR, it was better to not play with other high MMR players because it would always make the CSR update small --- even at the beginning of a new season. In addition, this rewarded high MMR players teaming with highly-skilled but lower MMR players to artificially inflate their CSRs. We have changed this so that very high MMR players can now team up with their highest-skilled friends and earn more CSR at the beginning of a season. In addition, players trying to team up with much lower MMR teammates will end up not getting much CSR.
#1 means the CSR system will deflate everyone a little and more accurately reflect your actual skill. Most playlists won’t notice this much, but higher-skilled FFA players probably will.

#2 means smurf boosting no longer works well at the Champion level. It also means Champions can earn more CSR playing with other Champions, especially at the beginning of the season.

#2 also has some implications for Champion-level players playing with their noob friends. Basically, something has to give. Extremely high CSR players playing with low ones causes systemic problems, even when done honestly.

Many other competitive games put a hard block on high ranked players playing with low ones. Instead of doing that, we'll allow you to play together with the same matchmaking experience, but with the softer penalty of losing some rank if you’re at or near the Champion level.

If a Champion level player decides to play with their noob friends, their Rank will descend gradually to the average skill between the Champion and who they play with. Which I think it a fair compromise.

Or, said another way, if you want to keep your Champion-level Rank, you have to play with Champion teammates, which I also think is fair.

Overall, this update also means that CSR now more accurately reflects the skill of our players than ever before. There will still be some lingering inflation this season, but I'm hoping next season results in the best ranks we've seen yet. This does mean that the higher Ranks like Onyx and even to a lesser extent, Diamond, will be harder than ever to attain, but I think that's for the best.
So what Im getting from this basically is that playing the rest of the season is pointless as a FFA player because of this CSR Loss/Gain change, which only took place because someone at 343 failed to notice an obvious flaw that caused people's CSR to inflate. And now even though you place Top 3 even 2nd or 3rd causes CSR loss because of this inflation and the fix used to correct it. Sorry to say this but with the way these changes take place I can see Halo 5 doing a repeat of Halo 4 and seeing a huge population drop off due to inconsistency with how the matchmaking and ranked arena works.
What I don't get is why would you roll this out in the middle of a season, especially without telling anyone until a couple days after it happened? Just seems like you could have given everyone at least a weeks notice before ranks reset next season. I understand that people were being ranked incorrectly and that it needed to be fixed, but your timing was very poor and left a lot of people demoralized and not wanting to touch ranked at all.
Just wanted to stop by and say thanks for everything you do. Its greatly appreciated that you share your logic and opinions on why and how everything is changing.
What I don't get is why would you roll this out in the middle of a season, especially without telling anyone until a couple days after it happened? Just seems like you could have given everyone at least a weeks notice before ranks reset next season. I understand that people were being ranked incorrectly and that it needed to be fixed, but your timing was very poor and left a lot of people demoralized and not wanting to touch ranked at all.
Yes, normally I don't like to make CSR changes mid-season.

The reason this happened this time was that it was part of the exploit fix, and fixing exploits is something I try and do as soon as I can, if possible.
  1. 1
  2. ...
  3. 2
  4. 3
  5. ...
  6. 4