Forums / Community / Matchmaking Feedback & Discussion

[Locked] MATCHMAKING FEEDBACK UPDATE – November 5

OP ZaedynFel

  1. 1
  2. ...
  3. 23
  4. 24
  5. 25
  6. 26
  7. ...
  8. 27
The quit discussion is interesting, but like I said, there aren't a lot of players who consistently quit.

Creating a quit score and matching on it would result in no material difference in experience to the players because the top quitters only account for a tiny fraction of the overall quitting.

The rest of the quitting is spread out among everyone, so everyone would have a similar score and just match same as today.

Except for maybe those who, for whatever reason, NEVER quit, which is a teeny portion. If we matched on a quit score, that tiny portion of stalwart players would have much longer wait times, which doesn't seem like a good reward for their behavior.
ZaedynFel wrote:
Except for maybe those who, for whatever reason, NEVER quit, which is a teeny portion. If we matched on a quit score, that tiny portion of stalwart players would have much longer wait times, which doesn't seem like a good reward for their behavior.
D:
That would be terrible. "Hey, thanks for never quitting! As a reward, please enjoy extended queue times!"
eLantern wrote:
NightClerk wrote:
The “quit score” would be another metric used during MM but would never lead to imbalanced matches.
Just to clarify, when you say "...would never lead to imbalanced matches" you refer to the max limit of 60:40, correct?

In my opinion, 60:40 odds does not equate to a "balanced" match. But, it is an acceptable max limit regarding allowable imbalanced matches.
Quote:
MMR would be the primary driver of matches...
How so?

Please feel free to use the breakdown that I provided of the matchmaking system to explain how.
Spoiler:
Show
Quote:
...and no one would experience games outside of acceptable odds.
As I noted here:
Spoiler:
Show
It seems as though I was correct as you make some conflicting comments here:
Quote:
“Quit score” would yield to MMR balancing. MMR takes precedence...
You say SBMM takes precedence ^here^, but then follow that up with a statement (below) that suggests it doesn't truly take precedence.
Quote:
...and odds could very well be different because of a “quit score”
Stating that MMR will be used to prevent match odds from going beyond the max allowable limit does not equate to it taking precedence within team balancing.
I have explained this ad nauseam. I have stated repeatedly that MMR would be the primary driver of MM. "Quit score" would never lead to matches outside of acceptable MMR odds (I see the 60/40 figure frequently so maybe it's that, whatever it is, this system would remain in that parameter). It may lead to a match being 53/47 instead of 51/49. It would be adjusted to balance odds, search times, etc.

I'm laying the groundwork here, not writing a paper on how the system would work like the Microsoft research team did for TS2. It's a quality of life feature to reduce the negative impacts of quitters on good players. I have not made conflicting comments. You are trying to twist my words for the sake of argument. See my previous posts as I have addressed all questions at least once.
Chimera30 wrote:
ZaedynFel wrote:
Except for maybe those who, for whatever reason, NEVER quit, which is a teeny portion. If we matched on a quit score, that tiny portion of stalwart players would have much longer wait times, which doesn't seem like a good reward for their behavior.
D:
That would be terrible. "Hey, thanks for never quitting! As a reward, please enjoy extended queue times!"
It would take balancing, especially regarding search times, match odds, etc. As I mentioned previously, this is more or less a re-skin of what Valve did for CS:GO and its cheating problem, but here instead of cheating we look at quitting. Valve's system has been pretty successful.

Obviously almost all players quit sometimes, but ranking them based on frequency of quitting can establish tiers. I'm providing an outline, not making a comprehensive proposal. The actual implementation would be much more in-depth.
NightClerk wrote:
eLantern wrote:
...
I have explained this ad nauseam.
Yes, you've repeated your rough (general) idea to ad nauseam. Though, I have been trying to get you to think about it more thoroughly and critically instead of repeating it ad nauseam.
Quote:
I have stated repeatedly that MMR would be the primary driver of MM.
Yes you have. But, you don't back your statement up with any form of support. In fact, when I've pressed you to explain further, and with more detail, you've simply given an example of the opposite.
Quote:
"Quit score" would never lead to matches outside of acceptable MMR odds (I see the 60/40 figure frequently so maybe it's that, whatever it is, this system would remain in that parameter).
Understood, but that parameter is not the same thing as stating that MMR will remain the *primary* driving factor behind the team picker/balancer.
Quote:
It may lead to a match being 53/47 instead of 51/49.
Exactly my point. MMR is no longer the *primary* driving factor. The "Quit Score" effectively takes a form of precedence over SBMM within the role of the team picker/balancer.
Quote:
I'm laying the groundwork here, not writing a paper on how the system would work like the Microsoft research team did for TS2.
I understand you're trying the lay the groundwork, but I'm trying to help you think through the groundwork. Nobody's asking you to write the actual formula. And understanding the systematic process that the matchmaker and team picker/balancer goes through is not nearly as complex as trying to develop the formula for TrueSkill2.
Quote:
It's a quality of life feature to reduce the negative impacts of quitters on good players.
Yes I get that and appreciate what you're trying to do. It's also why I gave an example of how I foresaw your idea potentially working while "actually" maintaining MMR as the *primary* driving factor. And I did that here:
Spoiler:
Show
I don't know if you understood what I was explaining or you just dismissed it because the trust factor metric wouldn't be as influential as you desired it to be.
Quote:
I have not made conflicting comments.
Well, yes you did and I've point them out, but let's not focus on that too much. It's more or less besides the point now. The reason I continue to respond as I do is to help you better understand what you're actually requesting. The merit or value and consequences.
Quote:
You are trying to twist my words for the sake of argument. See my previous posts as I have addressed all questions at least once.
I'm not twisting your words. I'm simply responding to them. I've done so in order to (1) seek further clarity from you regarding your idea's request, and (2) to help bring awareness to you regarding the potential unintended consequences as well as to highlight other such conflicts that perhaps you might not have been aware of. Your groundwork idea was quite general (as you reference) and sounds good on the surface, but when given more in-depth thought it becomes questionable.

Not only was I trying to get a better understanding of what you were seeking, but once I was able to better understand I could see the conflict it caused and faced. Then I was trying to help you think more deeply about it and how it would work within Halo 5's matchmaking system so you too could better see the issues it caused or faced. I did this by asking you to further explain it within the matchmaking process example that I broke down for you. And you never did that. I just got a lot of repeated statements regarding the general groundwork that included contradictions.
NightClerk wrote:
Chimera30 wrote:
ZaedynFel wrote:
Except for maybe those who, for whatever reason, NEVER quit, which is a teeny portion. If we matched on a quit score, that tiny portion of stalwart players would have much longer wait times, which doesn't seem like a good reward for their behavior.
D:
That would be terrible. "Hey, thanks for never quitting! As a reward, please enjoy extended queue times!"
It would take balancing, especially regarding search times, match odds, etc. As I mentioned previously, this is more or less a re-skin of what Valve did for CS:GO and its cheating problem, but here instead of cheating we look at quitting. Valve's system has been pretty successful.

Obviously almost all players quit sometimes, but ranking them based on frequency of quitting can establish tiers. I'm providing an outline, not making a comprehensive proposal. The actual implementation would be much more in-depth.
That's the thing though, if you recall from the Microsoft Research paper, they thoroughly analyzed each stat before adding it to the model and excluded stats that wouldn't help.

In this case, we've pretty thoroughly analyzed quit behavior and can say that it also wouldn't help precisely because of how it's distributed.

Quitting is much much much much different than cheating, so regardless of Valve's success, that example isn't comparable.

Quitting has the pattern of pretty much every player in the game going from "good" to "quitter" within a game. It is not a premeditated action.

Cheating is a much smaller segment of the community, and requires premeditated actions. It requires players install cheats and have the intent to use them, and is a clear different segment of the population. It isn't something that "everyone does" now and then. It's not like every CS player sometimes "rage cheats"

So that approach doesn't work here, the distribution doesn't fit.

There's no "Quit score" system that would do anything other than increase wait times for everyone without actually reducing quit rates, and that's based on analyzing our distribution of quitting behaviors.
ZaedynFel wrote:
NightClerk wrote:
Chimera30 wrote:
ZaedynFel wrote:
Except for maybe those who, for whatever reason, NEVER quit, which is a teeny portion. If we matched on a quit score, that tiny portion of stalwart players would have much longer wait times, which doesn't seem like a good reward for their behavior.
D:
That would be terrible. "Hey, thanks for never quitting! As a reward, please enjoy extended queue times!"
It would take balancing, especially regarding search times, match odds, etc. As I mentioned previously, this is more or less a re-skin of what Valve did for CS:GO and its cheating problem, but here instead of cheating we look at quitting. Valve's system has been pretty successful.

Obviously almost all players quit sometimes, but ranking them based on frequency of quitting can establish tiers. I'm providing an outline, not making a comprehensive proposal. The actual implementation would be much more in-depth.
That's the thing though, if you recall from the Microsoft Research paper, they thoroughly analyzed each stat before adding it to the model and excluded stats that wouldn't help.

In this case, we've pretty thoroughly analyzed quit behavior and can say that it also wouldn't help precisely because of how it's distributed.

Quitting is much much much much different than cheating, so regardless of Valve's success, that example isn't comparable.

Quitting has the pattern of pretty much every player in the game going from "good" to "quitter" within a game. It is not a premeditated action.

Cheating is a much smaller segment of the community, and requires premeditated actions. It requires players install cheats and have the intent to use them, and is a clear different segment of the population. It isn't something that "everyone does" now and then. It's not like every CS player sometimes "rage cheats"

So that approach doesn't work here, the distribution doesn't fit.

There's no "Quit score" system that would do anything other than increase wait times for everyone without actually reducing quit rates, and that's based on analyzing our distribution of quitting behaviors.
Thanks for elaborating on the data you’ve examined. I was also thinking of people who are AFK from the start, etc. But this explanation is helpful.
I think a reasonable way to deal with quitting is to have a rejoin feature, but also a feature that you can't search for a new game until the game you were in has finished.

Won't stop quitting when people are truly fed up for whatever reason but will alleviate connection based quitting for sure.
Thorulfr wrote:
I think a reasonable way to deal with quitting is to have a rejoin feature, but also a feature that you can't search for a new game until the game you were in has finished.

Won't stop quitting when people are truly fed up for whatever reason but will alleviate connection based quitting for sure.
Or even the "welp, Halo decided to be a jerk and crash in the middle of my match" while still being comnected to your party chat.
Would you be willing to tell us the variables that Trueskill 2 uses to assess MMR and CSR? I am curious as to what parameters are used.
Would you be willing to tell us the variables that Trueskill 2 uses to assess MMR and CSR? I am curious as to what parameters are used.
The white paper will give you the most in-depth description on how TS2 measures skill.

The super generalized version: TS2 looks makes a prediction of whether you'll win/lose, your kill rate, and your death rate. If TS2 predicts right, it means it has accurately placed you and adjustment to your MMR isn't needed. If TS2 predicts wrong, it will adjust your MMR accordingly. CSR then follows MMR closely; CSR will always go down on a loss and always go up on a win, but the amount CSR goes up/down is dependent on how close to your MMR your CSR is.

In designing TS2, additional variables were tested in the prediction model, but they either didn't make the prediction better or made the prediction worse. This is why TS2 doesn't look at things like medals, damage, accuracy, etc.
Chimera30 wrote:
Would you be willing to tell us the variables that Trueskill 2 uses to assess MMR and CSR? I am curious as to what parameters are used.
The white paper will give you the most in-depth description on how TS2 measures skill.

The super generalized version: TS2 looks makes a prediction of whether you'll win/lose, your kill rate, and your death rate. If TS2 predicts right, it means it has accurately placed you and adjustment to your MMR isn't needed. If TS2 predicts wrong, it will adjust your MMR accordingly. CSR then follows MMR closely; CSR will always go down on a loss and always go up on a win, but the amount CSR goes up/down is dependent on how close to your MMR your CSR is.

In designing TS2, additional variables were tested in the prediction model, but they either didn't make the prediction better or made the prediction worse. This is why TS2 doesn't look at things like medals, damage, accuracy, etc.
Thanks for posting the link to the white paper on TS2. I read it and it did confirm some of what I suspected about rankings for any mode having an influence on rankings for the others. Good to know. I am still looking for individual skill parameters. There must be more to how TS2 predicting beyond Kill and Death rates in relation to winning a match. I watched an Onyx/Champion player create a new profile and start from scratch. He played Free-For-All and only killed using Spartan Charge or Melee. He didn't win any of his matches, but by the third match he was playing other Champion level players. In theory, TS2 should believe it to be a new soul playing and using those parameters he should be playing low level players. However, TS2 has to also be factoring the method that players engage in the game. (i.e., the type of movements they use..are the advanced movements or basic, where on the map they move and when). I am assuming that better players maneuver around the map much differently that low level players. These are the types of data points I am looking for. What characteristics or data points does the system target as a "Champion Level" player vs an average "Gold Level" player. Also, I am incredibly curious about scenarios in Super Fiesta or Big Team Battle where an individual has a much higher number of kills vs his teammates or even those on the opposing team, and his team still loses. Does that individuals MMR still take a hit even though that person carried more weight that anyone else in the game? ZaedynFel
Thorulfr wrote:
I think a reasonable way to deal with quitting is to have a rejoin feature, but also a feature that you can't search for a new game until the game you were in has finished.

Won't stop quitting when people are truly fed up for whatever reason but will alleviate connection based quitting for sure.
Or even the "welp, Halo decided to be a jerk and crash in the middle of my match" while still being comnected to your party chat.
Yes there definitely needs to be a rejoin the match that you were just in but got disconnected for whatever reason feature for sure!! This better be a thing in Halo 6.
Why is the game so inconsistent, I mean there has always been an issue with shot reg but of late it's been shocking.
Even people who still stream the game moan about it.

Isn't just a moan because I'm losing, I'm a mid plat and fine with that, but the lag, shot reg, boost not working in a fight etc... is so infuriating.
Obviously it's not just me, see diamonds play games and struggle to get kills and even onyx guys run a negative K/D by a lot.
Then the next game it's fine and then it -Yoink-.

I do a grenade test at start of a game to rest it.
ZaedynFel wrote:
Call Fire wrote:
To the people that matter for matchmaking settings,

I lurk a bit and have posted only a few times.... basically when I feel like something needs to be said. Whoever is in charge should really consider a playlist (ranked HCS or social core play or social HCS) WITHOUT RADAR or at least with reduced power to radar. I'll tell you why.

You have made a game that has weapons and map layout; that is the biggest reason. To have default radar the way it is really undercuts the players need to understand key concepts of map control and spawning. Honestly it's silliness for me to make a move for control or to set up crossfire and flank when the whole endeavor is undercut because I'm being broadcast to the other team. One might think that having radar forces the action which is true in a sense...but you lose a HUGE dimension of DEPTH to this game. As with respect to forcing the action, I would submit that it actually does as much to slow action as players are compelled now to crouch with the scattershot, or crouch walk up to corners. If there was no radar, all else being equal...there is less reason to sit crouch and wait for that red dot AND you have the added incentive to COMMUNICATE in MULTIPLAYER LIVE games.

Call Fire, MD MS
Team Arena uses the ability tracker, so as long as you don't use a spartan ability, you won't show up on the radar. If you don't sprint, etc.

The majority consensus internally, as well as testing with Pros was that, given H5's abilities, we need at least the Ability Tracker or else ability usage gets way too spammy. If there's no radar at all, there's no punishment for using H5's spartan abilities, and the gameplay degenerates with folks flying all over the place. Sprint, thrust, slide can move you across entire maps, rocketing players instantly away from engagements or instantly to another with no warning.

Keep in mind I'm not currently working on the gameplay side here, so can't go in depth on this one. I'm just passing along the feedback we got from internal and pro testing.
I know what the current playlist settings are, but thanks... You guys should consider having more lists at least with "team arena" settings .... I don't know who can play "slayer" with all the radar and nonsense for more than like two games with any seriousness....
I was wondering when the season for ranked playlist would be over.
I was wondering when the season for ranked playlist would be over.
Season is going to reset on March 5th
Chimera30 wrote:
Would you be willing to tell us the variables that Trueskill 2 uses to assess MMR and CSR? I am curious as to what parameters are used.
The white paper will give you the most in-depth description on how TS2 measures skill.

The super generalized version: TS2 looks makes a prediction of whether you'll win/lose, your kill rate, and your death rate. If TS2 predicts right, it means it has accurately placed you and adjustment to your MMR isn't needed. If TS2 predicts wrong, it will adjust your MMR accordingly. CSR then follows MMR closely; CSR will always go down on a loss and always go up on a win, but the amount CSR goes up/down is dependent on how close to your MMR your CSR is.

In designing TS2, additional variables were tested in the prediction model, but they either didn't make the prediction better or made the prediction worse. This is why TS2 doesn't look at things like medals, damage, accuracy, etc.
Thanks for posting the link to the white paper on TS2. I read it and it did confirm some of what I suspected about rankings for any mode having an influence on rankings for the others. Good to know. I am still looking for individual skill parameters. There must be more to how TS2 predicting beyond Kill and Death rates in relation to winning a match. I watched an Onyx/Champion player create a new profile and start from scratch. He played Free-For-All and only killed using Spartan Charge or Melee. He didn't win any of his matches, but by the third match he was playing other Champion level players. In theory, TS2 should believe it to be a new soul playing and using those parameters he should be playing low level players. However, TS2 has to also be factoring the method that players engage in the game. (i.e., the type of movements they use..are the advanced movements or basic, where on the map they move and when). I am assuming that better players maneuver around the map much differently that low level players. These are the types of data points I am looking for. What characteristics or data points does the system target as a "Champion Level" player vs an average "Gold Level" player. Also, I am incredibly curious about scenarios in Super Fiesta or Big Team Battle where an individual has a much higher number of kills vs his teammates or even those on the opposing team, and his team still loses. Does that individuals MMR still take a hit even though that person carried more weight that anyone else in the game? ZaedynFel
In FFA, as long as he places in the top 3, his MMR is going to increase.
Chimera30 wrote:
Would you be willing to tell us the variables that Trueskill 2 uses to assess MMR and CSR? I am curious as to what parameters are used.
The white paper will give you the most in-depth description on how TS2 measures skill.

The super generalized version: TS2 looks makes a prediction of whether you'll win/lose, your kill rate, and your death rate. If TS2 predicts right, it means it has accurately placed you and adjustment to your MMR isn't needed. If TS2 predicts wrong, it will adjust your MMR accordingly. CSR then follows MMR closely; CSR will always go down on a loss and always go up on a win, but the amount CSR goes up/down is dependent on how close to your MMR your CSR is.

In designing TS2, additional variables were tested in the prediction model, but they either didn't make the prediction better or made the prediction worse. This is why TS2 doesn't look at things like medals, damage, accuracy, etc.
Thanks for posting the link to the white paper on TS2. I read it and it did confirm some of what I suspected about rankings for any mode having an influence on rankings for the others. Good to know. I am still looking for individual skill parameters. There must be more to how TS2 predicting beyond Kill and Death rates in relation to winning a match. I watched an Onyx/Champion player create a new profile and start from scratch. He played Free-For-All and only killed using Spartan Charge or Melee. He didn't win any of his matches, but by the third match he was playing other Champion level players. In theory, TS2 should believe it to be a new soul playing and using those parameters he should be playing low level players. However, TS2 has to also be factoring the method that players engage in the game. (i.e., the type of movements they use..are the advanced movements or basic, where on the map they move and when). I am assuming that better players maneuver around the map much differently that low level players. These are the types of data points I am looking for. What characteristics or data points does the system target as a "Champion Level" player vs an average "Gold Level" player. Also, I am incredibly curious about scenarios in Super Fiesta or Big Team Battle where an individual has a much higher number of kills vs his teammates or even those on the opposing team, and his team still loses. Does that individuals MMR still take a hit even though that person carried more weight that anyone else in the game? ZaedynFel
In FFA, as long as he places in the top 3, his MMR is going to increase.
Not true! Only if said players were at a very similar MMR or higher! Getting top 3 will not give you a rank up everytime. Coming from a Champion with a 60% win iN FFA at first and top 3 98% of my games.
Chimera30 wrote:
Would you be willing to tell us the variables that Trueskill 2 uses to assess MMR and CSR? I am curious as to what parameters are used.
The white paper will give you the most in-depth description on how TS2 measures skill.

The super generalized version: TS2 looks makes a prediction of whether you'll win/lose, your kill rate, and your death rate. If TS2 predicts right, it means it has accurately placed you and adjustment to your MMR isn't needed. If TS2 predicts wrong, it will adjust your MMR accordingly. CSR then follows MMR closely; CSR will always go down on a loss and always go up on a win, but the amount CSR goes up/down is dependent on how close to your MMR your CSR is.

In designing TS2, additional variables were tested in the prediction model, but they either didn't make the prediction better or made the prediction worse. This is why TS2 doesn't look at things like medals, damage, accuracy, etc.
Thanks for posting the link to the white paper on TS2. I read it and it did confirm some of what I suspected about rankings for any mode having an influence on rankings for the others. Good to know. I am still looking for individual skill parameters. There must be more to how TS2 predicting beyond Kill and Death rates in relation to winning a match. I watched an Onyx/Champion player create a new profile and start from scratch. He played Free-For-All and only killed using Spartan Charge or Melee. He didn't win any of his matches, but by the third match he was playing other Champion level players. In theory, TS2 should believe it to be a new soul playing and using those parameters he should be playing low level players. However, TS2 has to also be factoring the method that players engage in the game. (i.e., the type of movements they use..are the advanced movements or basic, where on the map they move and when). I am assuming that better players maneuver around the map much differently that low level players. These are the types of data points I am looking for. What characteristics or data points does the system target as a "Champion Level" player vs an average "Gold Level" player. Also, I am incredibly curious about scenarios in Super Fiesta or Big Team Battle where an individual has a much higher number of kills vs his teammates or even those on the opposing team, and his team still loses. Does that individuals MMR still take a hit even though that person carried more weight that anyone else in the game? ZaedynFel
Based on numerous responses by Josh true skill 2 only takes into account KPM, DPM and whether you won or not. Movement is not a "stat" used to determine skill as far as we know. The streamer you watched is probably very good at the game. If they are already a champion level FFA player then even with only using Spartan abilities and melee they could most likely reach diamond level. Now because FFAs population is so low the matchmaking range is quite high and allows champions to quite easily match diamonds and platinums. Therefore it isn't a stretch that the streamer was matching champions even on his new account. You don't have to win to have your MMR go up/go down by very little. A decent portion of how much your MMR changes is based on how well you did in terms of KPM compared to what was expected of you before the game. For example if you had a KPM of 3 but the system expected you to only achieve a KPM of 1 that game your MMR is going to probably rise, even if you lose that game.
  1. 1
  2. ...
  3. 23
  4. 24
  5. 25
  6. 26
  7. ...
  8. 27