Incident with Webhooks
Final update
On June 11, 2026, between 19:28 UTC and 21:06 UTC, GitHub webhook deliveries were delayed. Average delivery latency peaked at approximately 3.4 minutes, with some deliveries delayed by as much as 62 minutes at the 99th percentile. No events were lost , delayed events were queued and delivered once processing caught up. This was due to a change in how webhook traffic was distributed across regions: to relieve load on one region, a portion of processing was shifted to another, where higher latency prevented our delivery workers from keeping pace with incoming volume, creating a backlog. We mitigated the incident by rebalancing webhook traffic distribution; as load returned to normal levels, processing caught up and the delivery backlog fully drained. We are working on improving the accuracy of the utilization metrics used to scale our delivery worker pools, and reassess how we distribute webhook traffic across regions, to reduce our time to detection and mitigation of issues like this one in the future.
Timeline
- Resolved · Jun 11, 22:19 UTC
On June 11, 2026, between 19:28 UTC and 21:06 UTC, GitHub webhook deliveries were delayed. Average delivery latency peaked at approximately 3.4 minutes, with some deliveries delayed by as much as 62 minutes at the 99th percentile. No events were lost , delayed events were queued and delivered once processing caught up. This was due to a change in how webhook traffic was distributed across regions: to relieve load on one region, a portion of processing was shifted to another, where higher latency prevented our delivery workers from keeping pace with incoming volume, creating a backlog. We mitigated the incident by rebalancing webhook traffic distribution; as load returned to normal levels, processing caught up and the delivery backlog fully drained. We are working on improving the accuracy of the utilization metrics used to scale our delivery worker pools, and reassess how we distribute webhook traffic across regions, to reduce our time to detection and mitigation of issues like this one in the future.
More from GitHub
Full history| Started | Incident | Impact | Duration |
|---|---|---|---|
| Sep 2416:51 UTC | Disruption with billing information updates | minor | 3h 50m |
| Sep 2310:11 UTC | Incident across several services | minor | 18h 44m |
| Sep 2022:13 UTC | Incident with Pull Requests | minor | 1h 9m |
| Sep 1720:59 UTC | Elevated rate of errors for OpenAI models provided by Copilot | minor | 50m |
| Sep 1607:20 UTC | Degradation with Gemini 3.8 Flash | major | 10h 28m |
| Sep 1519:11 UTC | Disruption with some GitHub services | minor | 49m |
Also caused by storage
All| Started | Vendor | Incident | Impact | Duration |
|---|---|---|---|---|
| Sep 1715:50 UTC |  Elastic | AutoOps node metrics temporarily unavailable in some regionsElastic | major | 41m |
| Sep 1015:26 UTC |  Supabase | Unresponsive ProjectsSupabase | major | 27h 40m |
| Jul 1909:51 UTC |  DigitalOcean | Block Storage Volume NYC1, NYC3, SGP1, SYD1 and BLR1DigitalOcean | minor | 5h 41m |
| Mar 720:07 UTC |  Grafana Labs | Outage for prod-eu-central-0 due to AWS S3 outage.Grafana Labs | none | 36h 52m |
| Mar 719:53 UTC |  AWS | Increased Error RatesAWS | major | 1h 11m |
| Nov 2808:28 UTC |  Mailgun | Delayed Logs in EU RegionMailgun | minor | 10h 46m |
Sources: vendors' own status pages, published postmortems and SEC 8-K Item 1.05 filings, read daily. Times as reported. Logos via logo.dev; trademarks belong to their owners.