[Locked] Matchmaking Feedback Update – August 21

Tighter Matchmaking

As part of last Thursday’s blog post we mentioned that based on community feedback and our own data, we decided to tighten matchmaking in the Ranked playlists. Instead of allowing matches within 2- and sometimes 4-tiers, we now only allow matches within 1-tier of your MMR across all Ranked lists.

Keep in mind this is MMR and not CSR, so there will be the occasional exception where you see a gap that looks like it’s more than 1-tier (e.g. Gold v. Diamond), but this can only happen in cases where the Gold is already expected to play as Plat, or the Diamond is expected to drop soon to Plat. Or both.

For those used to Halo 3 numbers, this is equivalent to 7-12 levels, depending on the playlist (e.g., HCS is 9.5 levels). H5 tiers are resized according to the actual population and skill dynamics, whereas H3’s were never “corrected” after ship, this accounts for the differences, and why the level to CSR map changes over time. For example a “50” in HCS was 1800 Onyx the first season, but is now around 1600 Onyx because more people have passed that point. We shifted Onyx up higher to preserve the actual percentage of people who can get that far into Onyx, whereas in Halo 3 we never re-scaled to prevent more and more people from getting 50s.

The “1-tier” rule also applies to Onyx, but considers all of Onyx and Champion as 1 wide tier. It’s wide in skill gap terms, but very narrow in population terms. Keep in mind that Champs are just the top 200 Onyx players, so they are technically still Onyx.

To do this, the matchmaker now considers any player at 1800 MMR (not CSR) or higher to be the same skill. This means, the top Champ can instantly match with an 1800 MMR player because they “look the same” to the matchmaker. This allows us to make sure the top players can still actually play, while simultaneously keeping the mid-tier of players in tight matches. Note that 1800+ players will almost never if at all see Diamond unless parties are involved.

This is similar to Halo 3 where the matchmaker ignored skill differences above 50. In Halo 3, this meant that a true party of 75s could immediately match with a party of 50s and destroy them. In Halo 5, this means a party of Champs can match a party of mid-Onyx and do likewise.

This isn’t ideal, so we plan to add a feature down the road that allows us to preserve the 1-tier rule for anyone lower than 1500, and allow anyone 1800+ to first try for closer matches before giving up and hitting 1800 players, and then eventually reaching down to 1500 players like today.

I don’t have an ETA on this, but it’s “on the list” per se. I’m not going to rush it because it affects a very small group of players who are a little more “thick-skinned” than the rest. Usually.

One more note with this change. We will probably still have to tweak a playlist or two in the coming weeks to make sure everyone can match.



12-player Fireteam Warzone

We’ve heard some of you wanting 12-player Warzone in some form. We have the concern that while players say they want to wait long periods of time, they actually have more fun getting immediate matches than not playing at all, even if they are in parties of 6.

Nevertheless, we have heard your requests and feel it would be worth testing this to see how well it actually does versus just speculating.

We still have some verifying to do before actually turning the switch on, but I think we have the ability to allow 12-player fireteams to matchmake in the normal Warzone playlist with the following caveats:

• We are willing to try it as a test. If it doesn’t work well, we would remove it quickly. If it works well, we might just leave it.
• 12-player fireteams would be restricted to only playing vs. other 12-player fireteams, for the time being. This is because our data clearly show that the current matchmaker can’t fairly match fireteams of this size otherwise. We are working to address this, but until then, this restriction must hold.
• We would only allow parties of sizes 1-6, and 12 because our party restriction system can only restrict full parties, not partial ones. This is currently our only option if we want to do this at all.
• There will be no way to tell if there are other available full fireteams, but you are welcome organize this on your own.
• If there are no other 12-player fireteams available, you will not find any matches whatsoever. You will instead get a red error screen every 5 minutes and must re-search manually.
• If you want to play but there are no other full fireteams around, your only option will be to split your fireteam down to a size of 6 or less

If you feel you can live with these restriction, then this might be a nice feature for you. We’ll be listening to your feedback, as well as monitoring stats on our side.

This is also only if we can verify beforehand that this is possible. If not, we won’t be able to do this. But we’re pretty confident right now that it will work.



Warzone Changes vs. Matchmaking

The last blog post also mentioned some Warzone gameplay changes. I just wanted to give a little motivation from a matchmaking point of view on these changes.

From monitoring the matchmaking data, we noticed that around 75% of Warzone matches were won by whichever team had the higher total MMR. This is despite having a pretty large population of players playing in Warzone.

We also found that tightening matchmaking didn’t have a strong enough effect on this, at least not without making it way tighter than people are currently used to, and drastically increasing wait times.

Usually, when both population is high and tighter matching doesn’t immediately help, it means that the game mode itself could use some attention.

So the matchmaking designers met with the original Warzone designers and asked what they thought. The settings changes we are seeing are the results of that cooperation, and part of an effort to increase match quality in Warzone.


Why am I Platinum or even Diamond in most lists, but Gold in HCS?

When the HCS list originally formed, we grabbed all the HCS players who had played a lot in Team Arena previous to the split and looked at what their MMRs had been in the Team Arena list. We found that the foundational population in HCS was around 1- to 1.5-tiers higher than the mean in Team Arena. This means that the mean of the player base that moved over and established the Ranks in HCS was around Diamond level.

Since we don’t use other playlist MMRs to inform new playlists (see below), this meant that the average player moving over from Team Arena (where they were Diamond) would now find their Rank right in the middle of the system, which is the border between Gold and Platinum.

In fact, even former Pros were starting in Platinum coming off their Placement matches, which would be the same as starting in Onyx in Team Arena.

So, yeah, HCS’s ranking system is shifted quite a bit from Team Arena’s and will probably stay that way since the foundation has already been established.

The old thread got locked, literally as I was replying, so here we try again:

Quote:

With that, we'll be able to both place smurfs incredibly fast into the right slots, and also detect those that intentionally underperform very easily.

So you're saying there will be less placement matches due to this faster method? But detecting underpreformers seems like it would need past history to judge the current performance on, which could lead to a rise of new accounts, rather than reusing old accounts. Is that taken into consideration as well?
(For example, last night I was playing with a To3, and our PUG kept intentionally betraying us (giving kills to the enemy), commiting suicide, etc. Taking a look at his service record, he was Spartan Rank 2 with 3 games, all 0-10, 0-13, 0-13. Based on his knowledge of the map, i.e. clambering and boosting places any new player wouldn't realise is there, I have 100% no doubt he was a new account smurf.)

Quote:

• We don't matchmake on CSR, we match on MMR, which you don't see. It is far more accurate than CSR.

Forgive me if you've already said it, but you're saying with the current system, CSR is inaccurate because it's limited to a maximum gain/loss of 20, while MMR isn't limited to such values? Something along those lines?

ZaedynFel wrote:

12-player Fireteam Warzone We’ve heard some of you wanting 12-player Warzone in some form. We have the concern that while players say they want to wait long periods of time, they actually have more fun getting immediate matches than not playing at all, even if they are in parties of 6.
Nevertheless, we have heard your requests and feel it would be worth testing this to see how well it actually does versus just speculating.
We still have some verifying to do before actually turning the switch on, but I think we have the ability to allow 12-player fireteams to matchmake in the normal Warzone playlist with the following caveats:

• We are willing to try it as a test. If it doesn’t work well, we would remove it quickly. If it works well, we might just leave it.
• 12-player fireteams would be restricted to only playing vs. other 12-player fireteams, for the time being. This is because our data clearly show that the current matchmaker can’t fairly match fireteams of this size otherwise. We are working to address this, but until then, this restriction must hold.
• We would only allow parties of sizes 1-6, and 12 because our party restriction system can only restrict full parties, not partial ones. This is currently our only option if we want to do this at all.
• There will be no way to tell if there are other available full fireteams, but you are welcome organize this on your own.
• If there are no other 12-player fireteams available, you will not find any matches whatsoever. You will instead get a red error screen every 5 minutes and must re-search manually.
• If you want to play but there are no other full fireteams around, your only option will be to split your fireteam down to a size of 6 or less

If you feel you can live with these restriction, then this might be a nice feature for you. We’ll be listening to your feedback, as well as monitoring stats on our side.
This is also only if we can verify beforehand that this is possible. If not, we won’t be able to do this. But we’re pretty confident right now that it will work.

One thing I would suggest is, if this is possible, allow 11-12 person fireteams to search. The 11 person fireteam would know that they'll be playing a man down the entire time unless a friend Joins in Progress.

This offers flexibility for lobby leads. Organized teams of 11 should be able to compete against an organized team of 12 fairly easily.

ZaedynFel wrote:

Tighter Matchmaking As part of last Thursday’s blog post we mentioned that based on community feedback and our own data, we decided to tighten matchmaking in the Ranked playlists. Instead of allowing matches within 2- and sometimes 4-tiers, we now only allow matches within 1-tier of your MMR across all Ranked lists.
Keep in mind this is MMR and not CSR, so there will be the occasional exception where you see a gap that looks like it’s more than 1-tier (e.g. Gold v. Diamond), but this can only happen in cases where the Gold is already expected to play as Plat, or the Diamond is expected to drop soon to Plat. Or both.
For those used to Halo 3 numbers, this is equivalent to 7-12 levels, depending on the playlist (e.g., HCS is 9.5 levels). H5 tiers are resized according to the actual population and skill dynamics, whereas H3’s were never “corrected” after ship, this accounts for the differences, and why the level to CSR map changes over time. For example a “50” in HCS was 1800 Onyx the first season, but is now around 1600 Onyx because more people have passed that point. We shifted Onyx up higher to preserve the actual percentage of people who can get that far into Onyx, whereas in Halo 3 we never re-scaled to prevent more and more people from getting 50s.
The “1-tier” rule also applies to Onyx, but considers all of Onyx and Champion as 1 wide tier. It’s wide in skill gap terms, but very narrow in population terms. Keep in mind that Champs are just the top 200 Onyx players, so they are technically still Onyx.
To do this, the matchmaker now considers any player at 1800 MMR (not CSR) or higher to be the same skill. This means, the top Champ can instantly match with an 1800 MMR player because they “look the same” to the matchmaker. This allows us to make sure the top players can still actually play, while simultaneously keeping the mid-tier of players in tight matches. Note that 1800+ players will almost never if at all see Diamond unless parties are involved.
This is similar to Halo 3 where the matchmaker ignored skill differences above 50. In Halo 3, this meant that a true party of 75s could immediately match with a party of 50s and destroy them. In Halo 5, this means a party of Champs can match a party of mid-Onyx and do likewise.
This isn’t ideal, so we plan to add a feature down the road that allows us to preserve the 1-tier rule for anyone lower than 1500, and allow anyone 1800+ to first try for closer matches before giving up and hitting 1800 players, and then eventually reaching down to 1500 players like today.
I don’t have an ETA on this, but it’s “on the list” per se. I’m not going to rush it because it affects a very small group of players who are a little more “thick-skinned” than the rest. Usually.
One more note with this change. We will probably still have to tweak a playlist or two in the coming weeks to make sure everyone can match.
12-player Fireteam Warzone We’ve heard some of you wanting 12-player Warzone in some form. We have the concern that while players say they want to wait long periods of time, they actually have more fun getting immediate matches than not playing at all, even if they are in parties of 6.
Nevertheless, we have heard your requests and feel it would be worth testing this to see how well it actually does versus just speculating.
We still have some verifying to do before actually turning the switch on, but I think we have the ability to allow 12-player fireteams to matchmake in the normal Warzone playlist with the following caveats:

• We are willing to try it as a test. If it doesn’t work well, we would remove it quickly. If it works well, we might just leave it.
• 12-player fireteams would be restricted to only playing vs. other 12-player fireteams, for the time being. This is because our data clearly show that the current matchmaker can’t fairly match fireteams of this size otherwise. We are working to address this, but until then, this restriction must hold.
• We would only allow parties of sizes 1-6, and 12 because our party restriction system can only restrict full parties, not partial ones. This is currently our only option if we want to do this at all.
• There will be no way to tell if there are other available full fireteams, but you are welcome organize this on your own.
• If there are no other 12-player fireteams available, you will not find any matches whatsoever. You will instead get a red error screen every 5 minutes and must re-search manually.
• If you want to play but there are no other full fireteams around, your only option will be to split your fireteam down to a size of 6 or less

If you feel you can live with these restriction, then this might be a nice feature for you. We’ll be listening to your feedback, as well as monitoring stats on our side.
This is also only if we can verify beforehand that this is possible. If not, we won’t be able to do this. But we’re pretty confident right now that it will work.
Warzone Changes vs. MatchmakingThe last blog post also mentioned some Warzone gameplay changes. I just wanted to give a little motivation from a matchmaking point of view on these changes.
From monitoring the matchmaking data, we noticed that around 75% of Warzone matches were won by whichever team had the higher total MMR. This is despite having a pretty large population of players playing in Warzone.
We also found that tightening matchmaking didn’t have a strong enough effect on this, at least not without making it way tighter than people are currently used to, and drastically increasing wait times.
Usually, when both population is high and tighter matching doesn’t immediately help, it means that the game mode itself could use some attention.
So the matchmaking designers met with the original Warzone designers and asked what they thought. The settings changes we are seeing are the results of that cooperation, and part of an effort to increase match quality in Warzone.
Why am I Platinum or even Diamond in most lists, but Gold in HCS?When the HCS list originally formed, we grabbed all the HCS players who had played a lot in Team Arena previous to the split and looked at what their MMRs had been in the Team Arena list. We found that the foundational population in HCS was around 1- to 1.5-tiers higher than the mean in Team Arena. This means that the mean of the player base that moved over and established the Ranks in HCS was around Diamond level.
Since we don’t use other playlist MMRs to inform new playlists (see below), this meant that the average player moving over from Team Arena (where they were Diamond) would now find their Rank right in the middle of the system, which is the border between Gold and Platinum.
In fact, even former Pros were starting in Platinum coming off their Placement matches, which would be the same as starting in Onyx in Team Arena.
So, yeah, HCS’s ranking system is shifted quite a bit from Team Arena’s and will probably stay that way since the foundation has already been established.

Who could ask for more? Thank you sincerely for the help and consideration.
Your awesome!

radar3301enigma wrote:

ZaedynFel wrote:

Is MMR cross-playlist? Or do you use it to inform the MMR of other playlists? No, we don’t do that. I have done this in other games and found it useful, but we don’t currently do it in Halo 5.
We do have a prototype we’ve tested against Halo 5 data that suggests we should be doing some form of MMR sharing across game modes but not necessarily playlists.
The prototype does two interesting things:

• Has an MMR per game mode
• Tracks how much each game mode is correlated with each other game mode.

Let’s say you’ve only ever played Slayer. This means you only have a Slayer MMR. But now, let’s say you go to play SWAT for the first time. Well, the system has millions of data points showing the relationship between Slayer and SWAT players and can then correctly construct a first pass MMR for your when you go into SWAT.
Not only that, but if you’ve played many game modes, it can correctly pull from all of them to form a sort of weighted-average that estimates where you should at least start in a new game mode.
Once you establish your actual skill in the new mode, though, this initial impression is swamped by your performance in the mode instead of looking at other modes.
This may bring up a concern that how you play in one mode will affect your Rank in another. The answer is:

• Only indirectly since Ranks are based on performance and not predictions
• Only as much as actual real data suggests it should
• Only initially and lightly since it trusts your actual performance in the mode over other modes once it sees enough

What effect would this have on mixed game mode playlists, such as Arena and HCS?

The MMR used for matchmaking would be an average weighted by mode frequency.

The MMR used for CSR would be whichever mode was played.

Just thought of this: can we remove the "don't match against blocked players" flag that was discussed during the warlords thread? A couple people with large block lists might make this a short test unfortunately

I have a slight suggestion for the WZ 12 man test.
Alright, so with this test.... How would two separate Warzone playlists work? It would be akin to the Heroic and Legendary firefight splits. Here's what I am imagining roughly:

"Heroic" or "Base" Warzone. This would be the 'social' warzone as it is, with a cap of 6 or even 4 or 8.

"Legendary" Warzone. This would be the 'competitive' warzone, as it was on launch with allowing fireteams of 12. There could be a minimum size, but I'd suggest none. Players would be warned that the playlist is intended for large teams, but solo or small-group searching would be allowed. Due to the more frantic nature of large parties clashing, this playlist may feature a slightly higher EXP/RP output like Legendary FF does to Heroic FF.

ZaedynFel wrote:

Sylxeria wrote:

I have a slight suggestion for the WZ 12 man test.
Alright, so with this test.... How would two separate Warzone playlists work? It would be akin to the Heroic and Legendary firefight splits. Here's what I am imagining roughly:

"Heroic" or "Base" Warzone. This would be the 'social' warzone as it is, with a cap of 6 or even 4 or 8.

"Legendary" Warzone. This would be the 'competitive' warzone, as it was on launch with allowing fireteams of 12. There could be a minimum size, but I'd suggest none. Players would be warned that the playlist is intended for large teams, but solo or small-group searching would be allowed. Due to the more frantic nature of large parties clashing, this playlist may feature a slightly higher EXP/RP output like Legendary FF does to Heroic FF.

Splitting the population always makes matchmaking worse for each sub-population, and, in our data, also always makes the sub-populations shrink and grow unhealthy over time. This has been the case without fail, no matter how logical the split seems to be.

Mm well hopefully this test will work out, there's been a lot of people who have been looking forward to something like this.

One thing I hear about Warlords is the search times ended up rising after teams began blocking each other so they wouldn't deal with a team that was constantly beating them. After a while, nobody could find matches because people were leaving, and everyone else had each other mutually blocked.

So you can search with 12 in the playlist which is great but can you remove blocking for any fireteams searching with 12 people?

Sylxeria wrote:

One thing I hear about Warlords is the search times ended up rising after teams began blocking each other so they wouldn't deal with a team that was constantly beating them. After a while, nobody could find matches because people were leaving, and everyone else had each other mutually blocked.

Rhinos Back wrote:

So you can search with 12 in the playlist which is great but can you remove blocking for any fireteams searching with 12 people?

Ruined Empire wrote:

Just thought of this: can we remove the "don't match against blocked players" flag that was discussed during the warlords thread? A couple people with large block lists might make this a short test unfortunately

I'll look into it. The problem is, we can't remove the blocks for 12-player parties only, and leave them in place for everyone else.

But I'll see.