Query failures in Dremio Cloud (US)

Final update

This incident has been resolved.

1. Resolved · Mar 21, 18:29 UTC
   This incident has been resolved.
2. Monitoring · Mar 21, 18:12 UTC
   We have completed manual mitigation for all customers and are continuing to monitor the environment for stability. We apologize for any inconvenience caused.
3. Identified · Mar 21, 17:39 UTC
   We have seen successful recovery for the majority of customers and are continuing to complete manual remediation for a small number of remaining customers. We will provide an update within the next 30 minutes.
4. Identified · Mar 21, 17:07 UTC
   We have seen successful recovery for the majority of customers and are continuing manual remediation for remaining impacted customers. We will provide an update within the next 30 minutes.
5. Identified · Mar 21, 16:40 UTC
   We have seen successful recovery for most customers and are continuing to manually mitigate for the remaining customers. We will provide another update within the next 30 minutes.
6. Identified · Mar 21, 16:05 UTC
   We have seen successful recovery for some customers and are continuing to mitigate for the remaining customers. We will provide another update within the next 30 minutes.
7. Identified · Mar 21, 15:40 UTC
   We are currently investigating an increased rate of query failures for some customers in Dremio Cloud (US). We have identified the issue and are working to mitigate this. We will provide another update within the next 30 minutes.