Skip to main content

Forums / Community / Matchmaking Feedback & Discussion

[Locked] MATCHMAKING FEEDBACK UPDATE – APRIL 24

1
...
2
3
...
4

4/25/2017

Noodle 2x

Recruit - Bronze

ZaedynFel wrote:
Confusion happens because the only things we show are CSR and SR. Neither of these are used for matchmaking. We use MMR. We’ve actually run tests to look at CSR and SR and both are significantly worse than MMR for the purposes matchmaking. However, they are also both better than MMR for Ranking and Progression respectively.

I'm having a hard time understanding why CSR works better for ranking than MMR. If MMR is what our matchmaking is based on why isn't displaying everyone's MMR numbers more accurate or better for ranking than displaying a different ranking system that hides the MMR?

Show MoreShow Less

4/26/2017

thebanginho

Recruit - Gold

ZaedynFel wrote:
Apoll0 wrote:
My main question is in regards to the "Soft Forfeit" idea that was mentioned a while ago so that as long as you aren't the first (or second, not sure where you want to draw the line) person to quit you dont have to worry about getting a quit ban, but it still counts as a loss.

Is that being worked on?
Yes. That is really close. I'll have more news soon. I kind of want to wait until both the ban piece and the CSR penalty piece are in before announcing it.

Tangential question: So we know that there's a full 30 CSR loss for quitting and lagging out, but is there an MMR effect? Do you lose the full amount in MMR as well or since you didn't finish the match does it just stay the same or does it take into account how well you did prior to disconnecting? Also, is MMR affected solely by winning and losing like CSR is? Thanks again for all the great updates Josh!

Show MoreShow Less

4/26/2017

Lazar7666

Recruit - Gold

A quick question (I hope) when I was playing Doubles with a friend I noticed they were gaining more CSR and losing less. I was wondering why this was happening because it had nothing to do with our performance in individual matches. I know this because we got a couple of double quit/lagouts before the game started and he still got more CSR. We have a few ideas on why this happened but were hopping you could give us a definitive answer.

https://www.halowaypoint.com/en-au/games/halo-5-guardians/xbox-one/mode/arena/matches/6c4beb23-9647-49d2-8531-35476f061a7b/players/lazar7666?gameHistoryMatchIndex=37&gameHistoryGameModeFilter=All

Show MoreShow Less

OP 4/26/2017

ZaedynFel

Master of Unicorns

LUKEPOWA wrote:
Like I said earlier, I don't want strict matching, but just closer skill matching with full parties which I think is doable in a playlist like BTB for example since it's traditionally been the more populated playlist in past Halos (my previous post assumption). The thing that's up in the air for me is if that kind of social change can be applied to individual playlists or is it universal.

I run into full parties everyday which is why I responded the way I did in my previous post. Granted, it's usually the same dedicated ones, but there are the random ones as well that I'll run into more than once. I communicate sometimes, but the majority of the time it's not worth it especially when no one else is talking either. When you're playing against onyx level players and most of your teammates are gold or lower, communicating isn't going to do much if they or me can't get kills. I'll let Josh determine if it's hard to do or not, but I think he could do it.

Edit: First game of the night in BTB. Get matched against a full party who farm us. They intentionally didn't score the last flag even when we only had 3-4 people left. Lol, you can't make this up.
Link

Out of curiousity, how do you tell if it's a full party? Level of coordination? Playing the same exact people again?

The problem with Party matching in BTB isn't so much the overall population size, but the amount of that population that runs in large parties. It's pretty tiny, so facing one of those is generally pretty rare.

Show MoreShow Less

OP 4/26/2017

ZaedynFel

Master of Unicorns

Noodle 2x wrote:
ZaedynFel wrote:
Confusion happens because the only things we show are CSR and SR. Neither of these are used for matchmaking. We use MMR. We’ve actually run tests to look at CSR and SR and both are significantly worse than MMR for the purposes matchmaking. However, they are also both better than MMR for Ranking and Progression respectively.
I'm having a hard time understanding why CSR works better for ranking than MMR. If MMR is what our matchmaking is based on why isn't displaying everyone's MMR numbers more accurate or better for ranking than displaying a different ranking system that hides the MMR?

MMR can run way ahead of what you've actually accomplished. It's a forward looking prediction at how we *think* you can perform, but not what you've actually done yet.

We can think way ahead of time that someone is probably Onyx, but then they go and completely prove us wrong by playing the next 20 games like a Platinum player.

MMR is also not a single number that shows your skill, it's a distribution. It's a complete bell curve. The middle of the bell curve is our best guess at how you're going to perform in your next game, so that's where we match make you.

But your actual Rank could be anywhere along that bell curve with some probability, so rather than assume you immediately deserve the middle of that MMR curve as your Rank, we make you prove us right first.

According to the math around that bell curve (normal distribution) your actual Rank has a 50% chance of being lower than we thought it was, and a 50% chance of being higher. So we don't want to just give you that Rank.

We want more data to prove our guess is right.

Another way to think about it if you are familiar with NCAA type stuff. MMR is, e.g., the BCS Ranking and CSR is the trophy / winning the bowl.

Show MoreShow Less

OP 4/26/2017

ZaedynFel

Master of Unicorns

thebanginho wrote:
Tangential question: So we know that there's a full 30 CSR loss for quitting and lagging out, but is there an MMR effect? Do you lose the full amount in MMR as well or since you didn't finish the match does it just stay the same or does it take into account how well you did prior to disconnecting? Also, is MMR affected solely by winning and losing like CSR is? Thanks again for all the great updates Josh!

That's a good question.

We want MMR to be our best guess at what your skill can be. On the one hand, when you quit early, we don't get much info about your performance, so it wouldn't make sense to trust that behavior much towards updating our belief about your skill.

On the other hand, according to our analysis, most quits early into a game happen because a player is performing poorly. That kind of information IS useful towards updating our belief about your skill --- if you quit because you are playing worse than we thought you would, we should lower our belief about your skill. We don't currently do this, but we may in the future.

CSR, on the other hand, is what you deserve and what you've earned. If you are forfeiting matches, even ones you probably would have won, you don't deserve the win or a trophy.

That's another reason we like separating CSR and MMR.

Show MoreShow Less

OP 4/26/2017

ZaedynFel

Master of Unicorns

Lazar7666 wrote:
A quick question (I hope) when I was playing Doubles with a friend I noticed they were gaining more CSR and losing less. I was wondering why this was happening because it had nothing to do with our performance in individual matches. I know this because we got a couple of double quit/lagouts before the game started and he still got more CSR. We have a few ideas on why this happened but were hopping you could give us a definitive answer.

https://www.halowaypoint.com/en-au/games/halo-5-guardians/xbox-one/mode/arena/matches/6c4beb23-9647-49d2-8531-35476f061a7b/players/lazar7666?gameHistoryMatchIndex=37&gameHistoryGameModeFilter=All

In this case, the system thinks your teammate is a little better than you, so it assumes when you win that he did a little more of the work than you. So he gets a few extra points, and you a few less.

Show MoreShow Less

4/26/2017

LUKEPOWA

Spartan - Onyx

ZaedynFel wrote:
Out of curiousity, how do you tell if it's a full party? Level of coordination? Playing the same exact people again?
The problem with Party matching in BTB isn't so much the overall population size, but the amount of that population that runs in large parties. It's pretty tiny, so facing one of those is generally pretty rare.

It's both. BTB is my main playlist so it's easy to recognize names and the people that they play with. If it is rare, then I must be getting the short end of the stick. The only thing I can think of is that it's probably my region and the time that I'm playing which is most likely when the daily population isn't at its highest.

Edit: I'll stop mentioning that complaint in future updates if it's getting repetitive.

Last edit about this: 9 games in BTB tonight with 1 win and 6 were against full parties. 3 were against the same party and I normally would've quit the build in the lobby, but I just wanted to highlight the problems. Oh, and hi Flex.

Show MoreShow Less

4/26/2017

thebanginho

Recruit - Gold

ZaedynFel wrote:
thebanginho wrote:
Tangential question: So we know that there's a full 30 CSR loss for quitting and lagging out, but is there an MMR effect? Do you lose the full amount in MMR as well or since you didn't finish the match does it just stay the same or does it take into account how well you did prior to disconnecting? Also, is MMR affected solely by winning and losing like CSR is? Thanks again for all the great updates Josh!
That's a good question.

We want MMR to be our best guess at what your skill can be. On the one hand, when you quit early, we don't get much info about your performance, so it wouldn't make sense to trust that behavior much towards updating our belief about your skill.

On the other hand, according to our analysis, most quits early into a game happen because a player is performing poorly. That kind of information IS useful towards updating our belief about your skill --- if you quit because you are playing worse than we thought you would, we should lower our belief about your skill. We don't currently do this, but we may in the future.

CSR, on the other hand, is what you deserve and what you've earned. If you are forfeiting matches, even ones you probably would have won, you don't deserve the win or a trophy.

That's another reason we like separating CSR and MMR.

That's great to know! I'm actually pretty encouraged by this because now I know if I lag out of a game and drop the 30 csr, my MMR isn't affected and I'll still be able to get into competitive games against people within my MMR gap. Also, by extension, since my CSR was artificially lowered it shouldn't be that difficult to regain those 30 points as long as I perform to the expectations the MMR bell curve has set for me over the next dozen or so games. Thanks Josh!

Show MoreShow Less

4/26/2017

eLantern

ODST - Silver

ZaedynFel wrote:
Noodle 2x wrote:
ZaedynFel wrote:
...
...
...

Another way to think about it if you are familiar with NCAA type stuff. MMR is, e.g., the BCS Ranking and CSR is the trophy / winning the bowl.

I took this and applied it to the NCAA's March Madness concept in case it helped some people better understand the metaphor:

In other words, MMR sort of acts like the seeding process for the Big Dance (NCAA Tourney) and CSR acts as the representation of how a team actually performed within the Big Dance; were they only good enough to reach Round 1, Round 2, Sweet Sixteen, Elite Eight, Final Four, Runner-up, or did they become the Champions regardless to what their seeding indicated was to be expected?

Show MoreShow Less

4/26/2017

Lazar7666

Recruit - Gold

ZaedynFel wrote:
Lazar7666 wrote:
A quick question (I hope) when I was playing Doubles with a friend I noticed they were gaining more CSR and losing less. I was wondering why this was happening because it had nothing to do with our performance in individual matches. I know this because we got a couple of double quit/lagouts before the game started and he still got more CSR. We have a few ideas on why this happened but were hopping you could give us a definitive answer.

https://www.halowaypoint.com/en-au/games/halo-5-guardians/xbox-one/mode/arena/matches/6c4beb23-9647-49d2-8531-35476f061a7b/players/lazar7666?gameHistoryMatchIndex=37&gameHistoryGameModeFilter=All
In this case, the system thinks your teammate is a little better than you, so it assumes when you win that he did a little more of the work than you. So he gets a few extra points, and you a few less.

So basically no matter how he preforms in the game what the system thought going into the game is how the points are awarded?

Show MoreShow Less

OP 4/27/2017

ZaedynFel

Master of Unicorns

Lazar7666 wrote:
ZaedynFel wrote:
Lazar7666 wrote:
A quick question (I hope) when I was playing Doubles with a friend I noticed they were gaining more CSR and losing less. I was wondering why this was happening because it had nothing to do with our performance in individual matches. I know this because we got a couple of double quit/lagouts before the game started and he still got more CSR. We have a few ideas on why this happened but were hopping you could give us a definitive answer.

https://www.halowaypoint.com/en-au/games/halo-5-guardians/xbox-one/mode/arena/matches/6c4beb23-9647-49d2-8531-35476f061a7b/players/lazar7666?gameHistoryMatchIndex=37&gameHistoryGameModeFilter=All
In this case, the system thinks your teammate is a little better than you, so it assumes when you win that he did a little more of the work than you. So he gets a few extra points, and you a few less.
So basically no matter how he preforms in the game what the system thought going into the game is how the points are awarded?

Currently, yes, because that's the only information we give the system. All else being equal, that's the more likely explanation.

In the future, we may look at actual in game performance to influence that. It's a tricky thing to get right though.

Show MoreShow Less

4/27/2017

Lazar7666

Recruit - Gold

ZaedynFel wrote:
Lazar7666 wrote:
ZaedynFel wrote:
Lazar7666 wrote:
A quick question (I hope) when I was playing Doubles with a friend I noticed they were gaining more CSR and losing less. I was wondering why this was happening because it had nothing to do with our performance in individual matches. I know this because we got a couple of double quit/lagouts before the game started and he still got more CSR. We have a few ideas on why this happened but were hopping you could give us a definitive answer.

https://www.halowaypoint.com/en-au/games/halo-5-guardians/xbox-one/mode/arena/matches/6c4beb23-9647-49d2-8531-35476f061a7b/players/lazar7666?gameHistoryMatchIndex=37&gameHistoryGameModeFilter=All
In this case, the system thinks your teammate is a little better than you, so it assumes when you win that he did a little more of the work than you. So he gets a few extra points, and you a few less.
So basically no matter how he preforms in the game what the system thought going into the game is how the points are awarded?
Currently, yes, because that's the only information we give the system. All else being equal, that's the more likely explanation.

In the future, we may look at actual in game performance to influence that. It's a tricky thing to get right though.

Thanks for clearing that up, it is slightly frustrating in that regard though.

Show MoreShow Less

4/28/2017

iTz Dogzilla

Marine - Silver

LUKEPOWA wrote:
ZaedynFel wrote:
Out of curiousity, how do you tell if it's a full party? Level of coordination? Playing the same exact people again?
The problem with Party matching in BTB isn't so much the overall population size, but the amount of that population that runs in large parties. It's pretty tiny, so facing one of those is generally pretty rare.
It's both. BTB is my main playlist so it's easy to recognize names and the people that they play with. If it is rare, then I must be getting the short end of the stick. The only thing I can think of is that it's probably my region and the time that I'm playing which is most likely when the daily population isn't at its highest.

Edit: I'll stop mentioning that complaint in future updates if it's getting repetitive.Last edit about this: 9 games in BTB tonight with 1 win and 6 were against full parties. 3 were against the same party and I normally would've quit the build in the lobby, but I just wanted to highlight the problems. Oh, and hi Flex.

Dude it's only repetitive because it keeps happening. Games like the one you linked may not be the norm, but they also are not the one in a million, almost non-existent aberration some would have us believe. TBH I would characterize that game as embarrassing to anyone involved in matchmaking when viewed just on it's face. When you start looking closer at the experience and ranks of the opposing players it gets a lot worse. I can't call out individual players, but compare the recent BTB record of one of the two Champs in that game to LUKEPOWA's. From a MM perspective how does he keep getting matched into win after win after win while Luke gets stomped 1-8? Is that player so good he's carrying seven random, Silver players every game? Because it looks like that's what Luke needs to do to get a win (slight exaggeration). Rather than asking how to tell if it's a pre-made team I'd be more interested in hearing explanations for some of these issues. I mean, team or not, you can't disagree that this shouldn't be happening, even in Social, especially to what appears to be a predominance of players new to the game. I mean who buys Halo, gets to SR 4 and says, "Now I'm ready to face a Team of Diamonds, Onyxs and Champss"? When you queue up a game for eight of your players and twenty of them quit that game it does not bode well for the future.

I'm not trying to roast Josh. He's the only one here to answer for a team of devs, but WTH?? And the stat padding when it's 3v8 couldn't be more lame. Some people have no shame, but they will blame 343 for matching them.

Show MoreShow Less

4/28/2017

RzR J3ST3R

Marine - Onyx

This is a slippery slope, and probably too prone to abuse, but.... It would be interesting if H6 had an internal reporting system, which you could use to report kill-farmers. That's gotta be the thing that most drives most average players away from this game.

As players, we don't have any right to get mad at an enemy team for beating us very badly - that's a flaw of the matchmaking system, or a consequence of low population, that created a mismatched game. But we DO have every right to be furious at a team who farms us for kills in a coordinated, egregious manner. Like 2 days ago when I played a To6 in WZ who trip-capped us pretty quickly on Noctus. An enemy killed me in my core room and took my full Light Of Urs, and didn't shoot ONCE at the core. He took it back outside the core and their team kept spawn-farming us. Or a To5 (all champ/high onyx) who played me in social assault a couple months ago, got 2 quick scores and then refused to score the final goal, holding the ball and just farming us for 12 full minutes. Behavior like that makes the halo community worse, and should be reported within the H6 system. Not yet sure how those folks should be punished... but 343 should at least start tracking the reports, and evaluate what to do eventually. The report should reference the actual MATCH where farming occurred, so that there's evidence of the behavior.

Show MoreShow Less

4/28/2017

ZippySkippy

Marine - Onyx

Ive been reading through a lot of these posts and have experienced a lot of them. I have said this once before but I personally think the heavy aim/slow Spartan movement in games is causing some if not a lot of this. One thing ive noticed when against higher ranks isn't the fact that there skill is better but they move around the map like there super man. Nothing effects them. They move so fluid and there shots deplete shields at a much faster rate. They can take much more damage and come clean from very little damage. In those same games I move like I'm caring another 50 pounds. I know my skill cant be represented properly just because of this issue. And the sad part is any new player coming into H5 wouldn't even know there was a problem because this heavy aim/slow Spartan movement doesn't expose itself as lag or anything of the sort. So most don't even know its there. But if a player watches close at players that do extremely well its pretty easy to spot. I know players may have a bad day and may not be on there game. But when I see players go 3-17 game after game or even worse some not getting a kill at all there just is no way even on a bad day a player should walk away with that low of kills. I know many may not believe this issue or just think its fabricated but I'm sorry its alive and well. This has such a direct impact to matchmaking and ranking it just very painful to see. I want H5 to thrive just like a lot of you. But if this issue doesn't get fixed I'm afraid this will be a vicious circle. And making any adjustments in the ranking system or matchmaking in general would not be a good idea until the above issue is resolved. Because the way they currently collect data doesnt show this issue or take in account of this issue. Which imo does reflect a lot in matchmaking issues. Again I cant stress enough how I want H5 to succeed and halo to grow positively moving forward. We just need to get to the bottom of this issue and move forward from there.
Thanks!
Zippy.

Show MoreShow Less

OP 4/28/2017

ZaedynFel

Master of Unicorns

ZippySkippy wrote:
I have said this once before but I personally think the heavy aim/slow Spartan movement in games is causing some if not a lot of this.

Heavy aim is definitely a real and reproducible issue. We are at this very moment working on a solution and hope to have good news soon, so stay tuned to the main page for updates.

It's not something I work on, but what I'm seeing looks promising.

Show MoreShow Less

4/28/2017

RzR J3ST3R

Marine - Onyx

ZaedynFel wrote:
ZippySkippy wrote:
I have said this once before but I personally think the heavy aim/slow Spartan movement in games is causing some if not a lot of this.
Heavy aim is definitely a real and reproducible issue. We are at this very moment working on a solution and hope to have good news soon, so stay tuned to the main page for updates.

It's not something I work on, but what I'm seeing looks promising.

I was playing warzone with a buddy this week - he REQed speed boost as soon as we cleared home base, i just sprinted normally. Yet we were running at the exact same speed... I should have taken a video capture of it, but it was really weird. I'd never really noticed this slow movement issue before, so I'm curious to learn more. Is there a thread discussing it?

Show MoreShow Less

4/28/2017

ZippySkippy

Marine - Onyx

RzR J3ST3R wrote:
ZaedynFel wrote:
ZippySkippy wrote:
I have said this once before but I personally think the heavy aim/slow Spartan movement in games is causing some if not a lot of this.
Heavy aim is definitely a real and reproducible issue. We are at this very moment working on a solution and hope to have good news soon, so stay tuned to the main page for updates.

It's not something I work on, but what I'm seeing looks promising.
I was playing warzone with a buddy this week - he REQed speed boost as soon as we cleared home base, i just sprinted normally. Yet we were running at the exact same speed... I should have taken a video capture of it, but it was really weird. I'd never really noticed this slow movement issue before, so I'm curious to learn more. Is there a thread discussing it?

The slow movement and heavy aim seem to be tied together or I should say if your experiencing one your likely experiencing the other. It is one of the most strangest things ive ever seen or can explain to be honest. There is a thread on the main page of H5 Guardians called Heavy Aim and CPU stress. A lot has happened since it was originally started. But feel free to read through it. You will likely see some similarities and understand it better from reading through the posts. If you have more questions feel free to pm me.

Show MoreShow Less

4/28/2017

ZippySkippy

Marine - Onyx

ZaedynFel wrote:
ZippySkippy wrote:
I have said this once before but I personally think the heavy aim/slow Spartan movement in games is causing some if not a lot of this.
Heavy aim is definitely a real and reproducible issue. We are at this very moment working on a solution and hope to have good news soon, so stay tuned to the main page for updates.

It's not something I work on, but what I'm seeing looks promising.

Thanks ZaedynFel for your reply. I greatly appreciate it. Anything that looks promising can be a very good thing!
Again thanks!
Zippy.

Show MoreShow Less

1
...
2
3
...
4