Forums / Community / Matchmaking Feedback & Discussion

[Locked] Matchmaking Feedback Update – July 3

OP ZaedynFel

  1. 1
  2. ...
  3. 2
  4. 3
  5. 4
  6. ...
  7. 6
ZaedynFel wrote:
Thanks for the updates, really intrigued by the changes coming to FFA. My CSR and MMR must really be in sync because I've been stuck at Plat 2 for forever despite winning multiple games and mostly placing top 3. I'm not sure that I agree that the best player usually wins, that may be the case but it definitely doesn't feel like it. This makes not gaining hardly any CSR for winning a game in which you were the highest ranked person in the lobby, feel really frustrating. I wish there was a way to select whether you'd rather the system try to match you against higher ranked players rather than those ranked below you, or at least try to do this first, then match with lower ranked players if it takes too long. For example, I'm only a Platinum 2 in FFA and it seems that I'm usually the highest ranked player in the lobby, so when I do win, I gain next to no CSR. I would much rather match higher Platinums and Diamonds than Golds and Silvers so I can learn how higher ranked players play and try to improve, gaining more CSR if I do manage to beat higher ranked players would be a nice bonus as well. I know matches are based off of MMR rather than CSR, so these Gold players I'm matching constantly aren't really any worse than me, but for me anyway, that makes the low CSR gains even more frustrating because it feels like I still had to fight pretty hard to get that win, like if one thing had gone differently, a lower ranked player could've easily won, so to not really be rewarded for getting the win kinda sucks. It's definitely better than it was though, it just feels like before we were gaining way too much CSR even when we didn't perform well, but now we're not earning enough CSR even when we do perform well.

I also had a quick question about ranked Snipers (super stoked this is coming back by the way). Is this going to be a kind of featured ranked playlist only for the July-August season, or is it going to be permanent (assuming the population is high enough)? Sorry if that's a stupid question, it just seems like every time I see it mentioned, it's in relation to the upcoming season rather than making it sound permanent. Really hope enough people will play it so it can stick around, Snipers has always been one of my favorite playlists and I've definitely missed it.
Oh, as to your matching vs. higher players question, the problem we run into there is the higher up you go, the less players there are, and so when we do have higher players, we really try and get them into a higher match first.

This makes it really hard for us to find room for them in Plat matches unless there really are no higher ones around.
Good point, hadn't thought of that. Obviously the higher ranked players should come first when it comes to fair matches, just kinda frustrating being such a low rank (talking about myself)and not gaining much CSR because I'm the highest ranked player in the lobby fairly consistently. And I know a lot of people don't like matching players that are ranked a lot higher than them, so that's probably an unpopular opinion anyway. I guess it's just because when I first started playing Halo I was playing with friends who were much better than me, so I feel like I get better faster when I'm playing with and against better players. No big deal though, like I said, it's a lot better than it was when everyone was gaining a crap ton of CSR just for not getting last. Excited to see how the changes affect things :)
I played Halo for around 3 and a half hours today and had nine 3v4s. Three of them started as 3 versus 4, which is just completely unacceptable, and I was on the team w/ three for 6 of the games. I even had times where I had two 3v4s in a row. Matchmaking desperately needs to get fixed, because running into this type of completely broken matches makes people stop playing the game, and that's not even taking the ghost melees, blank shots, heavy aim, disappearing rockets, thrust melees, and server disconnects into account.

To improve matchmaking, first off, don't start a game as a 3v4; it's really that simple. Second, as the game developers, figure out a way to tell if somebody scrolled through their menu to press leave game/dashboarded out or just disconnected, and penalize the people who purposefully quit out much more. Third, provide more incentives to not quit out of matches, such as the way Halo Reach does it with credit rewards. Fourth, ban repeat offenders by IP/Xbox and not just account, so that they can't just hop on an alt or a smurf. Finally, if a player is quitting out of a designated competitive playlist, such as HCS, don't derank the people unfortunate enough to be on her/his team, unless they're in a fireteam.
We never start a game 3v4. If a game starts 3v4, that means someone left during the map loading.

The game waits till there's 4v4 before going to load, but if someone leaves during load, there's nothing we can do.

Also, knowing whether or not they went to the menu to leave the match vs. disconnected doesn't help because savvy players just pull the plug or dashboard, and they're just as bad as any other quitter. We can't tell the difference between a network problem and you pulling your ethernet cable.

As for not deranking, any not deranking results in easy exploits, even if you're not in a fireteam (social engineering, etc.). If someone leaves early, we treat the rest of the leavers as forfeiting and they lose points as if they lost rather than the full penalty.
With how powerful forge is now, it's REALLY disheartening to see no dedicated playlist that stays where it is.

This is no longer a community that can be sequestered in a monthly spotlight for a specific gametype either.

If I had my way, there would be a system to "vet' maps by a decent amount of staff and hand picked players/forgers. I understand that it's hard to think that devoting a good chunk of time and resources into what is basically a smallish customs community would bring in enough bang for your buck. In reality, however, the amount of quality content and growth inherent in such a "smallish community" would exponentially increase, and tons more maps available to the public. As it stands right now (excluding grifball and community gametype maps such as Extermination and the MCC) we are at the lowest amount of fan made material in halos forging history.

And darn it if it just doesn't make sense when you consider how MASSIVE the strides in forge have come.

Give us our own bar, like warzone, social, and ranked, and let us do the heavy lifting for you.

Please, I am quite literally begging you to at least bring this up in the next meeting.

Please
ouv wrote:
Thanks for the updates, really intrigued by the changes coming to FFA. My CSR and MMR must really be in sync because I've been stuck at Plat 2 for forever despite winning multiple games and mostly placing top 3. I'm not sure that I agree that the best player usually wins, that may be the case but it definitely doesn't feel like it. This makes not gaining hardly any CSR for winning a game in which you were the highest ranked person in the lobby, feel really frustrating. I wish there was a way to select whether you'd rather the system try to match you against higher ranked players rather than those ranked below you, or at least try to do this first. For example, I'm only a Platinum 2 in FFA and it seems that I'm usually the highest ranked player in the lobby, so when I do win, I gain next to no CSR. I would much rather match higher Platinums and Diamonds than Golds and Silvers so I can learn how higher ranked players play and try to improve, gaining more CSR if I do manage to beat higher ranked players would be a nice bonus as well. I know matches are based off of MMR rather than CSR, so these Gold players I'm matching constantly aren't really Amy worse than me, but for me anyway, that makes the low CSR gains even more frustrating because it feels like I still had to fight pretty hard to get that win, like if one thing had gone differently, a lower ranked player could've easily won, so to not really be rewarded for that kinda sucks. It's definitely better than it was though, it just feels like before we were gaining way too much CSR even when we didn't perform well, but now we're not earning enpuhh CSR even when we do perform well.

I also had had a quick question about ranked Snipers (super stoked this is coming back by the way). Is this going to be a kind of featured ranked playlist only for the July-August season, or is going to be permanent (assuming the population is high enough)? Sorry if that's a stupid question, it just seems like every time I see it mentioned, it's in relation to the upcoming season rather than making it sound permanent. Really hope enough people will play it so it can stick around, Snipers has always been one of my favorite playlists and I've definitely missed it.
I'm in the same exact boat actually. Plat 2 in FFA and even winning I barely even saw my bar go up.
Same here. Extremely frustrating.
This post has been edited by a moderator. Please do not post spam.
*Original post. Click at your own discretion.
Spoiler:
Show
ZaedynFel wrote:
New Season This Week
30 seconds, 40 seconds, 2.5 minutes, 5 minutes…

How do you arrive at these numbers? I assume they're not arbitrary. How have you determined their value and validity?

Additionally, I prefer fair matches all the time. If I’m willing to wait, why not allow me to wait or give me the ability to wait?
Can you please look into or fix join in progress.

I joined a losing game with around 20 seconds left of the clock. Winning or losing, no one should be joining games with that little time left.

Link
LUKEPOWA wrote:
Can you please look into or fix join in progress.

I joined a losing game with around 20 seconds left of the clock. Winning or losing, no one should be joining games with that little time left.

Link
Nobody should join a game after the first 2-3 minutes of a game ever, especially if one team has a significant lead on the other. Nobody likes a loss added to their stats that they didn't even contribute to the majority of the game.
ZaedynFel wrote:
New Season This Week
30 seconds, 40 seconds, 2.5 minutes, 5 minutes…

How do you arrive at these numbers? I assume they're not arbitrary. How have you determined their value and validity?

Additionally, I prefer fair matches all the time. If I’m willing to wait, why not allow me to wait or give me the ability to wait?
Because if you wait, then someone else doesn't get to play because you don't join them. No one lives in a vacuum in matchmaking.

The numbers come from me stretching the platform limitations as far as I can.
ZaedynFel wrote:
ZaedynFel wrote:
New Season This Week
30 seconds, 40 seconds, 2.5 minutes, 5 minutes…
How do you arrive at these numbers? I assume they're not arbitrary. How have you determined their value and validity?
Additionally, I prefer fair matches all the time. If I’m willing to wait, why not allow me to wait or give me the ability to wait?
Because if you wait, then someone else doesn't get to play because you don't join them. No one lives in a vacuum in matchmaking.
The numbers come from me stretching the platform limitations as far as I can.
They do get to play, they just wait longer. However, if I’m not offered fair games, I won’t play at all. How does that help matchmaking?

Again, why 30 seconds? Why 40 seconds? etc. What justifies these decisions? What’s the rationale?
ZaedynFel wrote:
ZaedynFel wrote:
New Season This Week
30 seconds, 40 seconds, 2.5 minutes, 5 minutes…
How do you arrive at these numbers? I assume they're not arbitrary. How have you determined their value and validity?
Additionally, I prefer fair matches all the time. If I’m willing to wait, why not allow me to wait or give me the ability to wait?
Because if you wait, then someone else doesn't get to play because you don't join them. No one lives in a vacuum in matchmaking.
The numbers come from me stretching the platform limitations as far as I can.
They do get to play, they just wait longer. However, if I’m not offered fair games, I won’t play at all. How does that help matchmaking?

Again, why 30 seconds? Why 40 seconds? etc. What justifies these decisions? What’s the rationale?
Stats show people are more likely to stop playing due to longer waits than due to uneven matches, within a threshold. The waits times and thresholds are based on experience, experiments, and millions to near billions of matches worth of analysis.

They strike a balance between wait time and fairness that has shown to be accurate over that data.
ZaedynFel wrote:
New Season MMR Clipping The skill rating system (MMR) is supposed to clip everyone’s MMRs at the beginning of each season. We do this to trim off inflated player skills. We fixed an issue before last season that was preventing this clipping from happening.
Then, we discovered that the clipping was too aggressive in many playlists, making it so higher-skilled players had unusually large gaps between their MMR and CSR values, especially in FFA where players were starting at 1200 and having to climb up to 1800 (+600) just to get champ.
We have tuned that correctly for the new season coming, so the starting gaps for higher-skilled players will be more reasonable.
For those curious about the details:
  • We were clipping players to +/- 4.0 MMR
  • This would be 4 standard deviations IF the mean was 0 and the standard deviation 1.0
  • This is not the case in many playlists. For example, FFA has a mean AND standard deviation of around 2.0 each. So FFA was being clipped to only 1.0 standard deviations on the top (hence 1200 CSR).
  • We have changed it to clip 4 standard deviations from the mean, by playlist. So, using FFA as an example again, it will clip to 2.0 +/- 8.0 or [-6, 8].
Quick question -- what's the trimmed value for slayer and HCS? I'm just curious because those are the ranked playlists I play most. Thanks again for the great stuff Josh!
ZaedynFel wrote:
Social and Warzone Max MMR GapSo after “giving up” on skill, Social will still have at least a safety net.

This is still a huge gap, but roughly half what we allowed before. This means top players won’t see worse than high-Gold / Platinum players in their matches. Bottom players usually won’t see players better than those Gold / Plat ones.
We did the same thing in Warzone as well. We also kept Warzone’s expansion time to 30 seconds before it gives up on skill and allows the new, large, max gap.
This gap as large as you may describe is stopping a lot of players from finding games though. If you win all your games, the matchmaking construct tries to match me and my friends with the worst players, and some nights, only sweaty players are online, thus we don't load into any games because you cannot disadvantage us as the system usually would.

Can you please go into further depth about how the warzone mmr works?

Thanks.
I'm all for a more polished and focused MP experience but I have to question trimming down the Doubles map count. There's only so many as it stands. How can less be better?
I'm all for a more polished and focused MP experience but I have to question trimming down the Doubles map count. There's only so many as it stands. How can less be better?
There's at least 25 that I can think of that would be awesome in the Doubles playlist
ZaedynFel wrote:
ZaedynFel wrote:
ZaedynFel wrote:
New Season This Week
30 seconds, 40 seconds, 2.5 minutes, 5 minutes…
How do you arrive at these numbers? I assume they're not arbitrary. How have you determined their value and validity?
Additionally, I prefer fair matches all the time. If I’m willing to wait, why not allow me to wait or give me the ability to wait?
Because if you wait, then someone else doesn't get to play because you don't join them. No one lives in a vacuum in matchmaking.
The numbers come from me stretching the platform limitations as far as I can.
They do get to play, they just wait longer. However, if I’m not offered fair games, I won’t play at all. How does that help matchmaking?
Again, why 30 seconds? Why 40 seconds? etc. What justifies these decisions? What’s the rationale?
Stats show people are more likely to stop playing due to longer waits than due to uneven matches, within a threshold. The waits times and thresholds are based on experience, experiments, and millions to near billions of matches worth of analysis.
They strike a balance between wait time and fairness that has shown to be accurate over that data.
How many? How much more likely? My friends and I are surprisingly corporeal and multitudinous for people who don’t exist.

Make the “focused” search mean something or add “super focused” that guarantees a fair match. Give players the ability to decide how they want to play the game - not a black box.

The alternative is to continue to ignore a pool of highly desirable potential players not captured by the data and analysis.
Game 1
Game 2
Game 3
Game 4
Game 5

Why am I getting matches this lopsided? These are just the past few games I've played, there are even more examples of matches similar to these. I'm searching in Focused, but matches feel like social. Are there not enough players for me to have gotten a full lobby in Platinum? Basically, all the matches I played yesterday were where I would lose a large amount of CSR because I had players who don't know how to play the map, or the wins are meaningless because the other team was very low ranked. Why am I getting put in matches like these? I could probably find more examples from when I was in Diamond 1 yesterday, then got deranked all the way down to plat 4 because I wasn't getting enough CSR for each win, in some cases, it looked like I got nothing.

Thanks.
ZaedynFel wrote:
New Season MMR Clipping The skill rating system (MMR) is supposed to clip everyone’s MMRs at the beginning of each season. We do this to trim off inflated player skills. We fixed an issue before last season that was preventing this clipping from happening.
Then, we discovered that the clipping was too aggressive in many playlists, making it so higher-skilled players had unusually large gaps between their MMR and CSR values, especially in FFA where players were starting at 1200 and having to climb up to 1800 (+600) just to get champ.
We have tuned that correctly for the new season coming, so the starting gaps for higher-skilled players will be more reasonable.
For those curious about the details:
  • We were clipping players to +/- 4.0 MMR
  • This would be 4 standard deviations IF the mean was 0 and the standard deviation 1.0
  • This is not the case in many playlists. For example, FFA has a mean AND standard deviation of around 2.0 each. So FFA was being clipped to only 1.0 standard deviations on the top (hence 1200 CSR).
  • We have changed it to clip 4 standard deviations from the mean, by playlist. So, using FFA as an example again, it will clip to 2.0 +/- 8.0 or [-6, 8].
Quick question -- what's the trimmed value for slayer and HCS? I'm just curious because those are the ranked playlists I play most. Thanks again for the great stuff Josh!
Well, these numbers won't tell you a whole lot, but I'll post them for fun?

Slayer will be -4.85 to 6.21
HCS will be -3.98 to 5.18
Trengrove wrote:
ZaedynFel wrote:
Social and Warzone Max MMR GapSo after “giving up” on skill, Social will still have at least a safety net.

This is still a huge gap, but roughly half what we allowed before. This means top players won’t see worse than high-Gold / Platinum players in their matches. Bottom players usually won’t see players better than those Gold / Plat ones.
We did the same thing in Warzone as well. We also kept Warzone’s expansion time to 30 seconds before it gives up on skill and allows the new, large, max gap.
This gap as large as you may describe is stopping a lot of players from finding games though. If you win all your games, the matchmaking construct tries to match me and my friends with the worst players, and some nights, only sweaty players are online, thus we don't load into any games because you cannot disadvantage us as the system usually would.

Can you please go into further depth about how the warzone mmr works?

Thanks.
No, that's not how it works.

If you win a lot of games, your MMR is high, so the matchmaker does NOT try and find noobs for you to play with.

It tries to find people as close as possible to your MMR at first, and then after 30 seconds it opens the search up to any player within 4 standard deviations of your MMR.

If your Fireteam is in the middle, this will cover the entire population of players.

If your Fireteam is more than 4 standard deviations above the middle (1 in 31600 players will be like this), then you can match against anyone between the middle and the top 4 standard deviations, which is half the population.

But that doesn't really matter because most of the players are in the middle. Once the search expands, there are plenty of players near the middle that you will have access to.

Once you are actually in a match with 24 people, the team balancer will then form teams that create the smallest MMR gap possible given the players and Fireteams. That's where you may end up with the worst players on your team if your Fireteam is by far better than anyone else in the match.

But that has to be the case, otherwise it becomes a blowout.
l Jinxed I wrote:
Game 1Game 2Game 3Game 4Game 5Why am I getting matches this lopsided? These are just the past few games I've played, there are even more examples of matches similar to these. I'm searching in Focused, but matches feel like social. Are there not enough players for me to have gotten a full lobby in Platinum? Basically, all the matches I played yesterday were where I would lose a large amount of CSR because I had players who don't know how to play the map, or the wins are meaningless because the other team was very low ranked. Why am I getting put in matches like these? I could probably find more examples from when I was in Diamond 1 yesterday, then got deranked all the way down to plat 4 because I wasn't getting enough CSR for each win, in some cases, it looked like I got nothing.

Thanks.
Two things:
  • Pop is not amazing right now in HCS, so sometimes it will have to dig deep. There may not always be enough Plat to fill a match entirely with Plat. Also, if a party searches and their average MMR is Plat, then you will look equal to them even if they have a spread of players above and below Plat.
  • I looked into these matches. MMR gaps were (0.14, 0.29, 0.73, 0.89, 0.71). Which are all well within the old tolerance (1.0) and the new one (1.5). So despite the range of CSRs you see on the PGCR, the actual average MMR gaps were fine.
  1. 1
  2. ...
  3. 2
  4. 3
  5. 4
  6. ...
  7. 6