Network connectivity issues affecting services in North America

Final update

This incident has been resolved.

1. Resolved · Oct 23, 18:43 UTC
   This incident has been resolved.
2. Monitoring · Oct 23, 18:18 UTC
   Our network team has rerouted traffic around the impacted paths and we are no longer seeing packet drops. Our connectivity provider reports that its service has been restored, but some links are still being brought back, so we are keeping our traffic management changes in place.
3. Identified · Oct 23, 17:33 UTC
   The team is continuing to perform traffic engineering actions to route around impacted paths. The team will continue performing traffic engineering actions until all packet drops are mitigated.
4. Identified · Oct 23, 17:03 UTC
   The infrastructure provider has experienced a recurrence of the failure. Cloudflare is continuing to apply traffic management actions to further remediate the issue.
5. Identified · Oct 23, 16:59 UTC
   We are continuing to work on a fix for the issue.
6. Monitoring · Oct 23, 16:27 UTC
   A fix has been implemented and we are monitoring the results.
7. Identified · Oct 23, 16:07 UTC
   We have identified a connectivity provider outage causing network congestion and degraded performance for some customers in North America. Affected customers may experience increased latency, packet loss, and intermittent errors across multiple services. Our network team is rerouting traffic away from affected paths, and we are working directly with our provider on restoration.
8. Investigating · Oct 23, 15:47 UTC
   We are investigating degraded network performance affecting some traffic between our Ashburn and Atlanta data centers. Some customers may see increased latency, packet loss, or intermittent errors on Workers and other services in the Eastern US. Our network team is actively working on mitigation.