Investigating connectivity issues across accounts and devices due to an AWS outage

Final update

**What Happened:** On October 20th, starting around 03:00 AM EST, our US server customers experienced approximately 250 minutes of disruption, including about 165 minutes of full system downtime followed by 522 minutes of degraded performance that affected automations and integrations.  The issue was caused by an outage of our third-party provider, AWS, which primarily impacted the US region, but also created downstream effects globally. As a result, some customers in our EU and AU regions experienced related degraded performance issues with API, automations, and integrations, lasting up to 150 minutes. ‌ **What caused it:** This incident was caused by a major outage at our third-party provider, AWS \(Amazon Web Services\), in one of their U.S. data center regions. This outage led to widespread disruptions across many platforms, including [monday.com](http://monday.com). This issue began with errors and delays within AWS’s core systems, which affected how our servers handled data and automations. As AWS implemented mitigation efforts to restore functionality, our team actively worked to maintain stability and recovery as services gradually returned to normal. ‌ **How we responded:*

1. Postmortem · Oct 21, 07:29 UTC
   **What Happened:** On October 20th, starting around 03:00 AM EST, our US server customers experienced approximately 250 minutes of disruption, including about 165 minutes of full system downtime followed by 522 minutes of degraded performance that affected automations and integrations.  The issue was caused by an outage of our third-party provider, AWS, which primarily impacted the US region, but also created downstream effects globally. As a result, some customers in our EU and AU regions experienced related degraded performance issues with API, automations, and integrations, lasting up to 150 minutes. ‌ **What caused it:** This incident was caused by a major outage at our third-party provider, AWS \(Amazon Web Services\), in one of their U.S. data center regions. This outage led to widespread disruptions across many platforms, including [monday.com](http://monday.com). This issue began with errors and delays within AWS’s core systems, which affected how our servers handled data and automations. As AWS implemented mitigation efforts to restore functionality, our team actively worked to maintain stability and recovery as services gradually returned to normal. ‌ **How we responded:** Our engineering and developers teams acted immediately to stabilize the platform once the issue was detected. To protect performance and further disruption, we temporarily paused automations and API activity connected to the affected region while we worked on restoring service. We kept customers i
2. Resolved · Oct 20, 21:18 UTC
   The platform is now back to regular service. Please refresh your browser to access the platform. Thank you for your patience
3. Monitoring · Oct 20, 21:03 UTC
   We are continuing to monitor for any further issues.
4. Monitoring · Oct 20, 20:23 UTC
   All platform components availability and latency have returned to normal. We are no longer observing issues coming from AWS infrastructure. Our teams will continue to monitor platform availability; Automations is back to work but we do expect delays in automation completion.
5. Monitoring · Oct 20, 18:17 UTC
   All system is working and being monitored beside automations and API in US which are still being affected.
6. Monitoring · Oct 20, 17:04 UTC
   A fix has been implemented, all system is working and being monitored beside automations in US which are still being affected.
7. Monitoring · Oct 20, 14:59 UTC
   We are monitoring results to restore full system stability on the platform, including automations on the US server and issues connecting customers with our Customer Support Team. We will continue to provide updates until the issue is fully resolved!
8. Monitoring · Oct 20, 14:45 UTC
   Automations are experiencing degraded performance on the US server. We are still experiencing issues connecting customers with our Customer Support Team. We will continue to provide updates until the issue is fully resolved!
9. Monitoring · Oct 20, 14:43 UTC
   Automations are experiencing degraded performance on the US server. We will continue to provide updates until the issue is fully resolved!
10. Monitoring · Oct 20, 13:52 UTC
    We are currently experiencing issues connecting customers with our Customer Support Team. We are continuing to monitor for any platform-related issues.
11. Monitoring · Oct 20, 13:00 UTC
    We are still monitoring the incident.
12. Monitoring · Oct 20, 11:19 UTC
    Automations are experiencing degraded performance on the US server. We're working to fully resolve the issue as soon as possible!
13. Monitoring · Oct 20, 11:12 UTC
    Automations and integrations are experiencing degraded performance on the US server. We will continue to provide updates until the issue is fully resolved!
14. Monitoring · Oct 20, 10:47 UTC
    We're still working to fully resolve the issue as soon as possible!
15. Monitoring · Oct 20, 10:18 UTC
    The EU and AU servers are fully functional, and the platform is partly functional in the US server. We're working to fully resolve the issue as soon as possible
16. Identified · Oct 20, 10:05 UTC
    The platform is starting to recover, we will update again soon.