Project Lifecycle Issues in eu-west-1

Final update

This incident has been resolved.

1. Resolved · Sep 24, 17:38 UTC
   This incident has been resolved.
2. Monitoring · Sep 24, 17:12 UTC
   We are continuing to monitor to ensure ongoing stability.
3. Identified · Sep 24, 17:03 UTC
   The remaining projects have now been brought back up, and we have no projects remaining on the unhealthy underlying hardware in the upstream infrastructure. We are monitoring to ensure ongoing stability.
4. Identified · Sep 24, 16:56 UTC
   Most projects have been brought back up, and the team is working on the remaining projects, which should be completed soon. This issue was related to specific underlying hardware in the region, so this was isolated to only projects on that hardware. Project lifecycle operations (restarts, upgrades, etc) broadly across the region remain available.
5. Identified · Sep 24, 16:23 UTC
   All projects affected by the upstream infrastructure issues, and mitigation efforts are underway internally. These affected projects will be brought back up on new hardware as quickly as possible.
6. Identified · Sep 24, 15:50 UTC
   We have isolated the scope of the problem to a specific upstream incident. We are working to address the upstream incident and determine mitigations.
7. Investigating · Sep 24, 15:36 UTC
   We are investigating project lifecycle operation issues in eu-west-1.