MCC Insider Flight One Recap

As the dust settles and we look fondly back upon our inaugural flight for the MCC Insider Program, we wanted to circle back with our community and share some of the results and outcomes. Today we’re going to breakdown how it all went and go over some of the game team’s key takeaways and we’ll hear from Dr. Jessie Thomas - our partner on the Xbox Research team who’s driving the MCC Insider Program and helping to capture all of your great feedback.

Before we dive into that though, we first want to extend a huge THANK YOU to everyone who jumped in and participated during our first flight. We really appreciate your support and the time you invested into helping us improve MCC!

But we also want to thank EVERYONE who has signed up for the MCC Insider Program – we’re equally appreciative of both your passion and commitment to help shape the future of MCC. We know the vast majority of our Insiders weren’t able to get into this first flight but don’t despair – there will definitely be more opportunities. As we progress down our development roadmap, there will be additional flights that will increase in both scope as well as participant pool. The team remains committed to ensuring that at some point before all is said and done, anyone and everyone who wants to help play an MCC flight and share feedback to be able to.

Flight #1 Spec & Goals

In case you missed this in our last development blog, here’s a quick catch-up on what the first flight entailed and what our key goals were:

The first public flight was designed to help stress test all the core matchmaking updates the team has been working on over the past few months. This major foundational overhaul and modernization work is really at the core of the biggest changes to MCC – many of the issues and areas of feedback fans have been most vocal about stem from the game's original matchmaking systems.

One important note is that this flight was intentionally setup to run with peer-to-peer networking. You may recall that we’ve already announced that following the official retail update, all MCC MP matchmaking games will run via dedicated server. However, for this flight, the dedicated servers weren’t quite fully ready for prime time and we still needed to validate and test peer-to-peer networking (which is what will be used for Custom Games, among other scenarios).

Additionally, this first flight also included several other changes such as the slightly updated main menu UI, the removal of the “vote” button, and the inclusion of “continuous matchmaking.” HDR video settings were also enabled just prior to the flight officially kicking off.

To help validate updates to our network transport layer and improvements to the matchmaking and MP experience for players, the team looked at two key data points:

• Matchmaking success rate
• Time-to-match

In both of these metrics, the team set baseline goals that would represent improvements over the current retail MCC offering and validate that these new updates are working as intended. To facilitate this data, the first flight was setup with a targeted content and experiential offering:

• Halo 2 Anniversary Multiplayer on four maps
• Team Slayer to 25 kills / 5 min game timer

The flight build was setup this way to really promote throughput above all else – the act of searching and joining games and continuously rolling from game to game was the most important aspect of the flight. Approximately 2,150 players were invited to participate in the first flight and were chosen based on a few different criteria (one of which being those who were most active in MCC MP in the past six months). We decided to leave the build accessible full time once it was available, but to help focus efforts and the population we landed on three official playtime windows:

• Friday, April 27:  7PM to 10PM PST
• Saturday, April 28:  10AM to 1PM PST
• Sunday, April 29:  4PM to 7PM PST

Additional Goals

Outside of the specific goals for the game itself, this first flight also carried some unique goals of its own:

• Establish the proper processes and procedures to release game builds (and rapidly iterate fixes/updates as needed)
• Ensure our MCC Message of the Day (rebuilt on new/updated internal services) worked as intended
• Validate our Xbox Insider Hub dependencies and toolsets
• Activate our MCC Insider Program participant pool and run the complete communications cycle (initial confirmation, build availability outreach, survey follow-up)

Flight #1 Results

Thanks to you all, we did it! The team has deemed the first flight a resounding success. Not only did we hit our key goals for time-to-match and matchmaking success, we also feel pretty good about the results of the other misc. goals noted above.

Let’s break it down:

• More than 14,000 total matchmaking attempts over the course of the three official playtest sessions
• Matchmaking success rates significantly exceeded Flight 1 goals
• Time-to-match results were significantly faster than our Flight 1 goals. The vast majority of players were into matches in less than 38s. (that’s pretty quick!)

From a purely technical level, metrics and the story the data told exceed expectations. We’re still waiting on a final apples-to-apples comparison between some of this flight data and the current retail MCC equivalent (it’s complex to align the various telemetry pieces) but it’s clear that the matchmaking and networking updates have already offered some big improvements. Our data folks also need to take into account things like population size and geographic locale, among other factors.

Like me, you may be wondering, “why would there ever be a case of an unsuccessful match?” It could really boil down to all sorts of things from legitimate errors (which we're actively hunting for), to NAT incompatibilities, to just the nature of the internet itself (particularly in the case of a few of our participants outside of North America who surely had some challenges connecting to a peer-to-peer lobby across the world). 

Game Build Release Process

Overall the process of compiling and releasing builds went smoothly. You may not even realize this but from the initial time the first flight became available for download in the marketplace several updates were released before the weekend.

First, thanks to feedback from players who jumped in right away on Wednesday, the team was able to identify and resolve a ‘back fill’ issue that was sometimes resulting in an error with the 8^th member of a lobby wasn’t correctly filled. This issue was quickly addressed with an internal build, tested, and then deployed quickly and flawlessly on Thursday.

The team also released another build update just prior to the Friday official playtest that enabled the new/updated HDR video settings in the main menu. That wasn’t so much driven by response to an issue as it was just a feature that was now ready and they were eager to get it out into the wild.

Lastly, we also received feedback and validated that somehow an audio issue was introduced that resulted in the main menu music being muted. This too was quickly fixed, tested, and released late Friday night following the first session, ensuring things were in good order for the weekend.

To get a sense of the types of feedback we received from the first flight, please welcome Dr. Jessie Thomas – a member of the Xbox Research Team who not only administers the MCC Insider Program for us but is also at the helm of the Halo Community Feedback Program. Take it away, Jessie!

Key Takeaways & Learnings

Thanks Jessie!

While it may not seem like much, successfully executing this first flight was a big milestone for the team. Not only was it the first public testing of significant networking and system work at the foundation of MCC, it was also the first of what will be an ongoing public flight process that we wish to continually streamline and improve upon.

Communication – We are working on improved processes and means with which to communicate with flight participants. As we try to maintain confidentiality, it does create challenges where we can’t just use a Waypoint forum to post known issues and important news. We’re exploring options to do better at two-way communication next time. However, participants should also get in the habit of checking the Xbox Insider Hub app regularly, as that is a key source of up-to-date information.

Latency – One thing we heard from some players is that the game was “laggy” for them. Since we were using peer-to-peer networking with a small, geographically diverse population, this is generally to be expected and in many cases somewhat unavoidable. The team will continue to keep a close eye on this as we move into Flight #2.

Performance – Some players reported performance drops in certain areas on some of the Halo 2 Anniversary Maps. The team is aware and will continue to monitor.

Sweaty – When we picked some of the most active MCC players to participate – and there was no skill-based matchmaking in play - we expected things to get a bit sweaty. And boy did they. If you’re like me, you likely got accustomed to losing badly as you found the flight #1 player population to be full of lethal killers that never missed a shot.

Early Access – We pushed the flight build out on Wednesday, a full two days before the first official playtest, and it was great to see so many people immediately jumping in. In fact, that initial response exceeded our expectations, helped us uncover a few early issues, and is something we’ll look to do going forward.

Content & Engagement – The first official play session on Friday had the largest number of players with a slight drop across each of the next two days. We’re going to look at ways to leverage more/different content and/or further focus activities to help players stay interested over the span of the entire flight.

We also heard your feedback that you’d really like it if there was a greater variety of content. For this flight, we focused the offerings to really drill into our specific goals while also juggling a desire to keep the download size manageable and keep the initial experience aligned with what we felt was one of MCC’s more broadly liked and consistent offerings. Going forward, we’ll definitely be mindful of these tradeoffs and when possible, include more content to keep things a bit more interesting.

GGs - For me personally, I’d just like to add that this first flight was a really cool experience. In many ways, it really reminded me of the old, early days of playing Halo 2 or Halo 3 right before launch. We had a small, tight group of players who were super passionate and really wanted to play and participate. People were using voice chat and it was pretty common to face the same opponents and teammates multiple matches over which made the whole thing feel unusually quaint. Total strangers would send me “GG” messages after thoroughly kicking my ass and overall the whole thing just had a special, tight knit vibe to it that’s harder to come by nowadays with massive online populations. It was such a communal experience, and a weird deja vu experience for me, that I literally didn't mind only winning 2 out of my first ~25 games.

Special Thanks - Over Achievers

While we’re supremely grateful and thankful for everyone who put time in to help out with our first MCC Insider Flight, we do want to give a few super duper extra special shout-outs to some players we feel really overachieved in this area!

Officially, it’s not a contest - and of course it’s hard to single out just one member of our beloved community - but if we have to, then it simply has to be “Fishmvn.” They managed to play the most matches (209), rack up the most wins (156), accumulate the most kills (2070), and get the most assists (498) across our three official MCC Insider flight play sessions.

Fishmvn, we salute you!

Here's a look at the top 5 players with the most matches played:

1. Fishmvn - 209
2. NitsudustiN - 165
3. FRiCTiON just - 154
4. Halo Pro - 134
5. Vengeful Vadam - 132

Next Flight - Coming Soon!

The team is continuing to make progress on the overall MCC update (now working on “Milestone 6” for those keeping score) and they’re starting to prepare for Flight #2. We’re still locking on the finer details and specifics but here’s a quick snapshot of some of the current thinking:

• High level goal: Test dedicated server deployment and ongoing stability work
• Participants: Overall flight population will be increased (specifics TBD but looking at ~2X the first pool). Yes, participants from the first flight will continue to roll into subsequent flight rings.
• Metrics: Dedicated server performance, time-to-match, match success, # of errors
• Content: Based on your feedback, we intend to mix things up next flight and offer different/more content. We'll have locked details to share soon.

Remember, anyone and everyone can join the MCC Insider Program to help shape the future of MCC. Whether you get into this next flight or not, we’ll have plenty more opportunities ahead.

You can read more about the MCC Insider Program and sign up HERE. Also, remember that one way you can improve your odds of getting selected for a flight is to PLAY MCC!

Thank you again to everyone who joined us for this first flight and who has offered their support as the team works through this ambitious update to MCC. We’ll have more info to share soon!

Additional Bonus Reading

Stay up to date on the latest happenings with the MCC Insider Program and update project via the MCC Insider Hub here at Halo Waypoint! Learn more about the program including how to join and how to opt-out here: MCC Insider Program Details and FAQ. And follow along with updates and insights into the ongoing improvement and enhancement work via our recurring blog updates:

• MCC Development Update Blog #3 (April)
• MCC Development Update Blog #2 (March)
• MCC Development Update Blog #1 (January)