By ske7ch -
Public flights are an integral part of our MCC update plans as we roll out fixes, improvements, and features into work-in-progress builds so members of the MCC Insider Program can go hands-on to help validate all the great progress the team is making. Today, we’re looking back on the most recent flight to share some results and key takeaways from three action-packed days of MCC multiplayer mayhem. We’ll also hear from Dr. Jessie Thomas – our partner on the Xbox Research team – with a recap of some of the feedback she received via the official flight participant survey.
Before we dive in, we want to extend another huge THANK YOU to everyone who participated in flight #2! Your time and input truly are shaping the future of MCC and helping the team do the best work possible.
To everyone else who’s joined the MCC Insider program but hasn’t yet had an opportunity to participate in a flight – hang in there, we’ve got some news down below on what’s next and when you can likely expect to join in on the fun (soon!). And if you haven’t yet joined but would like to, we’d love to have you! Check out THIS PAGE for additional details and instructions on how to opt-in. And as always, if you’re interested in reading up on the work that’s been done to MCC so far, be sure to check out all of our prior MCC development update blogs.
Flight #2 Spec & Goals
The first public flight was all about validating foundational networking and matchmaking improvements at the core of MCC, which was measured via peer-to-peer playtesting of Halo 2: Anniversary multiplayer.
Following a successful test of that work, Flight #2 was designed to help validate continued updates and improvements to matchmaking but with a specific focus on dedicated server deployment and reliability across North America and Western Europe.
As you might recall from our prior blog updates, all multiplayer matchmaking games in MCC will utilize dedicated servers while customs games will rely on peer-to-peer connections.
To measure these matchmaking improvements and dedicated server operations the team focused on the following key data points (which were also the same metrics monitored during flight #1):
- Matchmaking success rate
- Time-to-match
The content offering for flight #2 was expanded with more variety across the three days of the flight (which was largely due to participant feedback from flight #1) and included the following:
- Thursday, 6/7 : Halo 3 4v4
- Friday, 6/8: Halo 3 Big Team Battle
- Saturday, 6/9: Halo 3 Action Sack & Halo 3 Team Doubles
In addition to unique playlist offerings across each of the three days, each day included TWO official play sessions to help better facilitate participation outside of North America.
To help test dedicated servers beyond North America, and support larger population playlists like Big Team Battle, the overall number of people invited to participate was nearly double that of flight #1. Everyone who was included in the first flight was invited back again in addition to a new set of players that were chosen from the MCC Insider pool and had a recent history of Halo 2 and/or Halo 3 multiplayer in MCC (among other factors).
Additional Goals
In addition to the specific metrics used to validate the overall matchmaking experience and dedicated server deployment and reliability, this flight had a few other goals tied to it:
- Continue to refine and hone the overall flighting program operations and build deployment process
- Expand and increase participant feedback and discussion via the removal of the confidentiality restrictions and creation of daily forum threads on Halo Waypoint
- Continue building best practices, expertise and tools to execute larger scale flights in the future
- Update the MCC build to clients and dedicated servers during the flight
- Experiment with additional methods to incentivize flight participation and player engagement
- Continue driving positive qualitative feedback and positive participant sentiment
Overall the team gathered some great additional insights and learnings in these areas. Read on for the results below.
Flight #2 Results
Overall, thanks to our MCC Insider participants, flight #2 was a success!
The primary goal was to validate dedicated server deployment and reliability and we saw the desired performance from the dedicated servers and were able to work through the new deployment pipelines and processes successfully.
On the data side, matchmaking success rates were in line with our targets though we did see some expected regression during some sessions that was a result of a combination of high-player playlists (Big Team Battle) and a lower than desired active player population. Overall, cases where matchmaking success rates did not hit the target were directly attributable to low population and regional distribution, not technical issues.
Time-to-match metrics ended up meeting the desired targets overall though, as with matchmaking success, we did see some players and sessions falling outside the ideal range on account of lower than expected concurrent player population and/or regional distribution. Additionally, the added step of connecting a party to the dedicated server (which wasn’t a factor in flight #1), also slightly increased everyone’s time-to-match. When we account for that, and look closer at the sessions that had the highest peak concurrency, the data was positive and much more in line with targets.
Now here’s Dr. Jessie Thomas to share some qualitative results based on MCC Insider survey feedback…