Partial outage in AMS
Oct 3, 08:01 UTCOct 3, 08:57 UTC
Duration
57m
Impact
Critical
Root cause
Database
Fly.io, 90 days
71 incidents
Affected
Not listed by the vendor.
Final update
All impacted services are operational now.
Timeline
- Resolved · Oct 3, 08:57 UTC
All impacted services are operational now.
- Monitoring · Oct 3, 08:35 UTC
All MPG clusters have recovered. We are monitoring for additional impact.
- Monitoring · Oct 3, 08:19 UTC
All hosts are back online. MPGv2 control plane is fully operational now. Some MPGv1 clusters in AMS might be degraded while MPGv1's control plane recovers.
- Identified · Oct 3, 08:08 UTC
Those hosts are coming back online after a brief interruption. We are assessing the impact on MPG.
- Identified · Oct 3, 08:01 UTC
Some of our hosts are offline in AMS. Control plane for Managed Postgres v1 and v2 in AMS is unavailable as a result of this. MPG clusters should remain online but any control plane ops will fail. Some machines in AMS are also unavailable.
More from Fly.io
Full history| Started | Incident | Impact | Duration |
|---|---|---|---|
| Oct 101:11 UTC3 days ago | Delayed ingestion of app logs | minor | 1h 38m |
| Sep 2921:12 UTC5 days ago | GRU Networking Issues | none | 0m |
| Sep 2620:10 UTC8 days ago | Network outage in AMS region | critical | 0m |
| Sep 2619:31 UTC8 days ago | Network outage in AMS region | critical | 22m |
| Sep 2417:11 UTC10 days ago | State database issues in GRU | major | 2h |
| Sep 2416:31 UTC10 days ago | State database issues in GRU | major | 2h 40m |
Also caused by database
All| Started | Vendor | Incident | Impact | Duration |
|---|---|---|---|---|
| Sep 2310:11 UTC11 days ago | Incident across several services | minor | 18h 44m | |
| Sep 1309:16 UTC3 weeks ago | Incident with several GitHub Services | critical | 1h 29m | |
| Sep 921:53 UTC3 weeks ago | INC20000211 | critical | 1h 58m | |
| Aug 2616:17 UTC5 weeks ago | Issues performance is degraded in the US | minor | 4h 19m | |
| Aug 1216:16 UTC7 weeks ago | Incident with Pull Requests and Issues | minor | 25m | |
| Aug 818:57 UTC8 weeks ago | Account Registration, Droplets, and Related Services | minor | 6h 3m |