Timeouts

Created by Blacklite at 7th Nov 2011, last modified 8th Nov 2011

Our host says:

On November 4th around 3pm EST we began to see intermittent network drops on a segment of our space in the Bluemile Colombus Ohio datacenter.  After many hours of analysis and working with clients reports we determined on Saturday November 5th that there was an issue with OSPF routing on 2 of the VLANS that our dedicated server traffic passes through.  After many more hours of checking equipment and QOS logs we found a faulty fiber module in our DDoS box.  We have routed traffic around this module and DDoS protection remains in place.  We have been monitoring since traffic was routed around this module and have not seen any further traffic drops and we believe the maintenance that was performed has resolved the connectivity issues that you may have noticed.  We will continue to monitor throughout the day to ensure that this is resolved.

Update:

On 11/07/2011 we believe this issue has came back for certain clients, specifically camfrog traffic.  We have determined the issue lies with our RioRey DDoS appliance.  This issue is impacting whitelisted IPs as well as this traffic still passes through the DDoS appliance.  We performed a maintenance on 2 cards in the RioRey this past weekend. Riorey has the logs from our appliance and is analyzing the data to determine the disconnect issues. We expect to hear back from RioRey in the morning of 11/07/2011 EST.  Please know we are fully committed to resolving this issue and we will provide a RCA root cause analysis after this issue is confirmed resolved.

Comments

Add comment