Forums / Support / Halo 5: Guardians Support

[Locked] Halo 5: Guardians - Investigating Server Issues

OP Unyshek

  1. 1
  2. ...
  3. 8
  4. 9
  5. 10
  6. 11
  7. 12
  8. ...
  9. 13
-
Please check the original post for the most up to date information. For all updates from March, please look below.
-
MARCH 29 UPDATE - 5:30PM PT

After jumping on the investigation again today with our partners, we believe we were able to identify the root cause. As a result of these findings, we are going to begin rolling back a change that was made at the impacted datacenters. To help set expectations, we anticipate this rollback may take approximately 24 hours to complete and may mean you'll still see some lingering issues as the transition occurs.

That said, please feel free to jump on this weekend and let us know if you're experience has improved – your feedback will be extremely valuable as we keep an eye on the situation.

Thank you for your patience and understanding throughout this process. We'll follow up with another update on Monday that will speak to whether or not this change has resolved the issue.

-
MARCH 28 UPDATE - 10:45PM PT
Our team rounded up partner teams for a call today and spent most of the day looking into potential root causes. While we weren't able to find the heart of the problem with everyone on the line, we were able to eliminate multiple theories. By doing so, we can continue to focus in on other options and move even closer towards finding the source of the issue.

We'll continue to provide updates as the investigation continues.

-
MARCH 27 UPDATE - 10PM PT
Hey all - checking in to let everyone know how everything went today. We've got good news and bad news. Unfortunately the changes that we rolled out this morning to help alleviate some of the server hitching issues were not effective. However, the good news is that our investigation is progressing well. We've identified a few symptoms that may be correlated with the hitching, which means we're closer towards finding the root cause of the issue.

We'll be moving forward with this new information, along with our partner teams, bright and early tomorrow morning.

Thank you for being so patient with us as we work through this, and please standby for further updates.

-
MARCH 26 UPDATE - 8PM PT
As we continue to investigate the root cause with partner teams, we'll be implementing a change that should help alleviate some of the hitching instances. Unfortunately we don't know if this change will completely solve the issue, but we hope it helps improve your in-game experience as we keep looking into everything.

We will be making this change tomorrow morning and monitoring the performance of our datacenters extremely closely.

We'll follow up with more details tomorrow to ensure you are up to date with the latest information.

-
MARCH 25 UPDATE - 7:30PM PT
Thank you for your continued reports as well as providing links to matches here on Waypoint. We're continuing our investigation and have begun to look even closer at the other datacenters that we use. Additionally, we've continued our escalation process and have more people looking into the issue.

Please know that we are actively on the case and will continue updates as soon as we get them.

-
MARCH 22 UPDATE - 7PM PT
While there isn't too much to update the thread on, we wanted to follow up before the weekend to let everyone know that we're still looking into it.

We'd like to ask that when reporting a match in the replies below, please include a link to it from your Game History here on Waypoint. It will help us identify your exact match, look into its details and telemetry, see the server you connected to, and investigate more thoroughly.

Apologies for any inconvenience that this may be causing.

-
MARCH 20 UPDATE - 6PM PT
We're actively monitoring our other datacenters so that we can identify instances like this one as soon as they happen. Our Services team is continuing to look into this issue with our external partner teams in order to find a root cause.

As the investigation continues, please stay tuned here for further updates.

-
MARCH 20 UPDATE - 3:25PM PT
Hey everyone,

We've been keeping a close eye on increasing player reports of server issues with Halo 5: Guardians. After some initial investigating, we were able to identify a datacenter in the North Central US that was causing players to experience hitches/warping and have temporary disabled it.

While this may increase the ping for people who would typically connect in the North Central US, we're confident that this will lead to a better experience for everyone while we further investigate the issue.

The team will continue to investigate and provide updates as soon as they are available.

Thank you for your patience.
I live in New Mexico so I’m taking it I’m mostly placed in the Western server. Well whatever the server I am being placed into at least every other game I’m throwing two names at once plus the throw is about 2 3 sec late. I’m also experiencing a lot of “skipping ahead” by players, what I mean is they will be on my screen running fine and all of a sudden they “jump” like 5 feet in front of where they were. It especially bad in Close Combat I’ve melee someone 3 4 times on my screen but all of a sudden I’m dead and they where behind me somehow? Plus the most infuriating thing is the melee system itself. Are there no melee lunges anymore? Like I’m saying just a slight lunge when ur 1 foot away from the guy holding forward you hit melee and nothing but the guy lunges 5 feet all of a sudden and kills you with 1 melee hit like what the??? I’ve been right next to people literally touching already hit the melee button and... nothing it doesn’t even register the beat down and my assassinations register as beat down every time now. Unless I hold the beat down button which I have turned off cause you always get killed when trying to do that cause it takes to long. Are these the problems 323 are looking to fix? I really hope so cause it’s infuriating playing sometimes when u land 4 shots on someone who is “1” shot(he really is 1 shot) and they don’t die but you have full health and get down to red turn the corner and 2 seconds later you fall to the ground and your dead from a pistol from the guy shooting at you before you turned the corner than he pulls so “Wanted” bending bullets around corners stuff and your dead. O man that’s a great feeling ain’t it fellow Halo players?!!?
I’m just curious as to whether you can share the root causes that were discovered within these data centers to be causing the poor performance?

And when you use the term “rollback” what exactly does that entail? And how’s it addressing the root issue(s)?
eLantern wrote:
I’m just curious as to whether you can share the root causes that were discovered within these data centers to be causing the poor performance?

And when you use the term “rollback” what exactly does that entail? And how’s it addressing the root issue(s)?
Good question I was thinking the same thing
eLantern wrote:
I’m just curious as to whether you can share the root causes that were discovered within these data centers to be causing the poor performance?

And when you use the term “rollback” what exactly does that entail? And how’s it addressing the root issue(s)?
This was related to the march 29th update:
Quote:
After jumping on the investigation again today with our partners, we believe we were able to identify the root cause. As a result of these findings, we are going to begin rolling back a change that was made at the impacted datacenters.
stckrboy wrote:
eLantern wrote:
I’m just curious as to whether you can share the root causes that were discovered within these data centers to be causing the poor performance?

And when you use the term “rollback” what exactly does that entail? And how’s it addressing the root issue(s)?
This was related to the march 29th update:
Quote:
After jumping on the investigation again today with our partners, we believe we were able to identify the root cause. As a result of these findings, we are going to begin rolling back a change that was made at the impacted datacenters.
Ah, so the root problem relates to a previous "change" that was made at the datacenters and they're rolling that back. Thanks for pointing that out.

Now this just makes me wonder what that change was and why it lead to the problems that it caused. Was it an important change that will need to be brought back, but with an update so that it doesn't cause the same issues? I don't anticipate answers to these questions, but I tend to let my curiosity get the best of me so I ask anyways.
US North East still affected

https://www.halowaypoint.com/en-us/games/halo-5-guardians/xbox-one/mode/warzone/matches/a4b92056-2b30-4deb-abf9-5ec5800ee40b/players/spartan0619?gameHistoryMatchIndex=0&gameHistoryGameModeFilter=Warzone
Northern California, still very laggy, makes it difficult to play.
In Colorado. Game is playing good. Played seven games of Slayer. I never had leg, it looked like one or two players were laggy on the other team.
Recent update post sounds like it might get better sometime this Saturday or next week. Guess I'll have to wait till then to try playing H5 again. Just hoping it doesn't end up being a month long problem.
While the hitching seems to be fixed. Hit registration has taken a nosedive. Small teleportation and acceleration is happening more often.
Quote:
APRIL 3 UPDATE - 7PM PT

Today comes with good news! Let's jump straight into it by location so you can see how things have progressed.
  • NorthCentral US: Rollback completed yesterday. We're seeing servers continue to perform exceptionally at this location.
  • SouthCentral US: Rollback completed today. While we anticipated this to take until Saturday, our partners found and expedited an alternative solution which has resolved the issue at this datacenter.
  • West US: Rollback completed today. This finished as planned today, but because it completed right before peak hours we opted to bring this location online tomorrow morning. In doing so, we ensure West US players can jump in without any issues tonight and allow the datacenter time for a more smooth return (and scaling) tomorrow.
We'll follow up with another update once the West US datacenter comes back online and we can look into its performance.
APRIL 4 UPDATE - 4:25PM PT
We've got even more good news today as well. After bringing the West US datacenter back online and monitoring it closely, we can confirm it is no longer seeing the issue. This means that our NorthCentral US, SouthCentral US, and West US databases are back online and functioning as expected

While we're considering this issue resolved, we will continue to monitor our all of our datacenters to ensure this does not impact any other locations or players. If you see any remaining cases of warping or hitching, please share a video of it happening and a include a link to the match here on Waypoint.

Thank you again for your patience as we worked through this over the past two weeks. It was a tough one to track down, solve, and apply in the live environment, so we truly appreciate you hanging in there with us.
Not sure if this is part of the server issues or not, but the hit registration and animations have not felt right today. I played two matches of warzone firefight about 40 minutes ago (after all datacenters came back online) and immediately noticed that my shots were not landing. I checked my internet speed twice through the xbox and both times it came out to roughly 190 mbps. Not sure if this is just on my end, or if it is a deeper problem within the game. Is anyone else experiencing this? I'm linking both games below as well as some dvr clips I recorded for those who are interested. The dvr clips were direct captures and not through the theater mode, so the reticle is lined up with where the shots should be going. Thanks in advance! Any responses would be helpful!

Game 1:
https://www.halowaypoint.com/en-us/games/halo-5-guardians/xbox-one/mode/warzone/matches/f0613c4d-a829-436c-817d-25c36f904c2f/players/spartan0619?gameHistoryMatchIndex=1&gameHistoryGameModeFilter=Warzone

Game 2:
https://www.halowaypoint.com/en-us/games/halo-5-guardians/xbox-one/mode/warzone/matches/8f4734d5-af5f-405c-a469-dc497f152bcb/players/spartan0619?gameHistoryMatchIndex=0&gameHistoryGameModeFilter=Warzone

DVR clip 1: (Double reload)
https://gamerdvr.com/gamer/spartan0619/video/72531463

DVR clip 2: (Delayed shot registration and blank shots)
https://gamerdvr.com/gamer/spartan0619/video/72531520
Not sure if this is part of the server issues or not, but the hit registration and animations have not felt right today. I played two matches of warzone firefight about 40 minutes ago (after all datacenters came back online) and immediately noticed that my shots were not landing. I checked my internet speed twice through the xbox and both times it came out to roughly 190 mbps. Not sure if this is just on my end, or if it is a deeper problem within the game. Is anyone else experiencing this?
You’re not alone in this. While it appears the hitching is gone the registration is certainly off. I don’t know if I’m shooting blanks or if something else is wacky.
You wanted to see warping? Check out this clip.... Although this was the only single instance of it happening this game. Dude was all over the place. I spawn, this dude appears out of thin air in front of me, I get him one shot, then he's behind me and runs past me, then behind me again and I finally finish the kill.

DVR clip:
https://gamerdvr.com/gamer/exo-bigg/video/72536537

Game link:
https://www.halowaypoint.com/en-us/games/halo-5-guardians/xbox-one/mode/arena/matches/a294e0ff-591c-4310-b63d-1bdfd3918a87/players/ExO%20BIGG?gameHistoryMatchIndex=4&gameHistoryGameModeFilter=arena
Agree hit registration seems a bit off
I played 2 nights ago for about 5 games and the lag was completely gone with the exception of a couple random players total in the entire night, which is seemingly normal.

Central Illinois: so either North Central US or South Central US Data center.

I'll try again Sunday night when I get home from my trip.
Thanks for the frequent updates!! All my “hitching” is gone now! Feels good to have an attentive post launch team that still provides support AND communication w the community :D yall are awesome, much love!
I don't know if it was a big coincidence or if posting helped, but I was lagging for almost 3 weeks. I made a post about it on 3/31 and what do you know, it stopped lagging the next day. Thank y'all for getting this fixed. I was about give up on Halo.
  1. 1
  2. ...
  3. 8
  4. 9
  5. 10
  6. 11
  7. 12
  8. ...
  9. 13