CDW - Potential Degrade Performance
Mar 1, 20:52 UTCMar 1, 21:38 UTC
Duration
46m
Impact
Informational
Root cause
Database
Cloudera, 90 days
1 incidents
Affected
Cloudera Data WarehouseCloudera Data Platform (US) - Data Warehouse
Final update
Our internal monitoring systems identified a decline in CDW control plane performance. Upon further investigation, we determined that a recent improvement to event handling resulted in an unexpected increase in database load. This issue has been rectified by successfully scaling up the database capacity. It is important to note that this behavior was not observed during our internal testing due to the broader variety of cluster configurations encountered in a production environment. Additionally, there was no impact on overall workloads.
Timeline
- Postmortem · Mar 27, 18:17 UTC
Our internal monitoring systems identified a decline in CDW control plane performance. Upon further investigation, we determined that a recent improvement to event handling resulted in an unexpected increase in database load. This issue has been rectified by successfully scaling up the database capacity. It is important to note that this behavior was not observed during our internal testing due to the broader variety of cluster configurations encountered in a production environment. Additionally, there was no impact on overall workloads.
- Resolved · Mar 1, 21:38 UTC
This incident has been resolved.
- Monitoring · Mar 1, 21:32 UTC
We are continuing to monitor for any further issues.
- Monitoring · Mar 1, 21:18 UTC
A fix has been implemented and we are monitoring the results.
- Identified · Mar 1, 20:52 UTC
We noted degraded application performance and are mitigating the issue right now .
More from Cloudera
Full history| Started | Incident | Impact | Duration |
|---|---|---|---|
| Sep 323:18 UTC4 weeks ago | Cloudera Management Console not accessible in US Control Plane | minor | 3h 29m |
| May 1118:12 UTC4 months ago | Intermittent Performance Issues - US Control Plane | none | 0m |
| Mar 1822:03 UTC6 months ago | Intermittent Management Console Access Issues Across US, EU, and AP Regions | minor | 0m |
| Sep 2510:23 UTC1 year ago | FreeIPA connectivity issues | minor | 6h 38m |
| Sep 2419:42 UTC1 year ago | Intermittent Performance and Access Issues with the Cloudera Management Console | minor | 21h 19m |
| Aug 1314:33 UTC1 year ago | DataHubs, DataLakes and FreeIPA are unreachable in US region | minor | 45m |
Also caused by database
All| Started | Vendor | Incident | Impact | Duration |
|---|---|---|---|---|
| Sep 2310:11 UTC10 days ago | Incident across several services | minor | 18h 44m | |
| Sep 1309:16 UTC2 weeks ago | Incident with several GitHub Services | critical | 1h 29m | |
| Sep 921:53 UTC3 weeks ago | INC20000211 | critical | 1h 58m | |
| Aug 2616:17 UTC5 weeks ago | Issues performance is degraded in the US | minor | 4h 19m | |
| Aug 1216:16 UTC7 weeks ago | Incident with Pull Requests and Issues | minor | 25m | |
| Aug 818:57 UTC8 weeks ago | Account Registration, Droplets, and Related Services | minor | 6h 3m |