Incidentsby IbisIncreased timeout errors in alert rule evaluations (AWS Germany, prod-eu-west-2)
Oct 21, 11:48 UTCOct 21, 13:26 UTC
Duration
1h 38m
Impact
Minor
Root cause
Not disclosed
Grafana Labs, 90 days
67 incidents
Affected
AWS Germany - prod-eu-west-2: Rule EvaluationAlerting and SLO - AWS Germany - prod-eu-west-2: Rule Evaluation
Final update
This incident has been resolved.
Timeline
- Resolved · Oct 21, 13:26 UTC
This incident has been resolved.
- Monitoring · Oct 21, 13:12 UTC
Remediation measures have been applied, and we are monitoring the recovery.
- Identified · Oct 21, 12:12 UTC
The root cause of the issue has been indentified and we are working on the remediation steps.
- Investigating · Oct 21, 11:48 UTC
Since approximately 20:30 UTC on October 20, we have observed an increase in alert rule evaluation errors affecting rules that query Grafana Cloud Prometheus (Mimir) data sources in Cluster AWS Germany. Our engineering team is actively investigating and working on remediation.
More from Grafana Labs
Full historySome GCK6 App Project Pages are Failing to Loadlasted 2h 8mPossible incorrect evaluation of Synthetic Monitoring alert rules in cluster prod-us-east-3lasted 0mDB issues in prod-sa-east-1lasted 2h 33mSome GCK6 App Project Pages are Failing to Loadlasted 1h 31mGrafana Cloud unavailibity in the prod-gb-south-0 regionlasted 3h 50mIRM outage in the region prod-gb-south-0 and prod-eu-west-0lasted 3h 5m