Actions Job Delays
Oct 1, 14:47 UTCOct 1, 17:56 UTC
Duration
3h 9m
Impact
Minor
Root cause
Capacity
GitHub, 90 days
68 incidents
Affected
Actions
Final update
This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available.
Timeline
- Resolved · Oct 1, 17:56 UTC
This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available.
- Investigating · Oct 1, 17:50 UTC
GitHub Actions experienced degraded performance for some hosted runners due to throttling within an upstream Azure dependency. Service capacity has recovered, and we are continuing to monitor while working with Azure on the underlying condition.
- Investigating · Oct 1, 16:48 UTC
We are currently applying a mitigation and anticipate recovery within thirty minutes.
- Investigating · Oct 1, 16:10 UTC
We have identified an issue with our upstream provider which is causing Actions requests to 429 which is creating the delays. We have escalated to the owning team and are investigating how to mitigate the 429s.
- Investigating · Oct 1, 15:29 UTC
We are seeing a reoccurrence in run-start delays, and are continuing to investigate to issue. Customers will potentially experience delays of up to ten minutes.
- Investigating · Oct 1, 15:20 UTC
Actions is experiencing degraded performance. We are continuing to investigate.
- Monitoring · Oct 1, 15:10 UTC
The degradation affecting Actions has been mitigated. We are monitoring to ensure stability.
- Investigating · Oct 1, 15:02 UTC
Run-start delays on Ubuntu runners have been resolved. We are investigating run-start delays on Windows runners and will share more information as it becomes available.
- Investigating · Oct 1, 14:54 UTC
We have identified the cause of increased Actions run start delays on Ubuntu runners and are actively deploying a fix. Customers may continue to experience intermittent delays while the mitigation rolls out and service metrics return to normal.
- Investigating · Oct 1, 14:47 UTC
We are investigating reports of degraded performance for Actions
More from GitHub
Full history| Started | Incident | Impact | Duration |
|---|---|---|---|
| Oct 113:37 UTCyesterday | Elevated request latency | minor | 20m |
| Oct 111:34 UTCyesterday | [Retroactive] Actions workflow run failures after deployment gate approvals | minor | 0m |
| Sep 2821:16 UTC4 days ago | Copilot Code Review is unable to complete reviews | critical | 52m |
| Sep 2416:51 UTC8 days ago | Disruption with billing information updates | minor | 3h 50m |
| Sep 2310:11 UTC9 days ago | Incident across several services | minor | 18h 44m |
| Sep 2022:13 UTC12 days ago | Incident with Pull Requests | minor | 1h 9m |
Also caused by capacity and load
All| Started | Vendor | Incident | Impact | Duration |
|---|---|---|---|---|
| Sep 2919:43 UTC3 days ago | Investigating service degradation - xAI models | major | 35m | |
| Sep 2318:42 UTC9 days ago | Network Performance Degradation , Asia-Pacific | minor | 6d 4h | |
| Sep 2213:19 UTC10 days ago | We are investigating an issue with CH servers in Azure germanywestcentral region | minor | 5h 6m | |
| Sep 221:44 UTC4 weeks ago | Elevated Linux worker queue times | minor | 21m | |
| Aug 2506:53 UTC5 weeks ago | [Critical] Issue with Downloads | critical | 44m | |
| Aug 2015:40 UTC6 weeks ago | We are investigating an issue where customers may experience timeouts, service degradations, errors, and elevated latencies across multiple products in the us-west1 region. | critical | 3h 40m |