Incidentsby GlyphElevated Request Errors for Grafana Across Multiple Regions
Nov 3, 12:40 UTCNov 3, 13:42 UTC
Duration
1h 1m
Impact
Minor
Root cause
Not disclosed
Grafana Labs, 90 days
67 incidents
Affected
AWS Australia - prod-au-southeast-1AWS Australia - prod-au-southeast-2AWS Brazil - prod-sa-east-1AWS Canada prod-ca-central-1AWS Canada - prod-ca-east-0AWS Germany - prod-eu-west-2AWS Germany - prod-eu-west-4AWS Germany prod-eu-central-1AWS India - prod-ap-south-1AWS Indonesia - prod-ap-southeast-2AWS Indonesia - prod-ap-southeast-3AWS Ireland eu-west-1AWS Ireland - prod-eu-west-6AWS Japan - prod-ap-northeast-0
Final update
Between approximately 12:10 UTC and 13:22 UTC on 3 November (about 1 hour 12 minutes), Grafana across multiple Grafana Cloud regions experienced elevated request errors, causing slow-loading or failing dashboards and Explore queries. The issue has been resolved and service has been fully restored.
Timeline
- Resolved · Nov 3, 13:42 UTC
Between approximately 12:10 UTC and 13:22 UTC on 3 November (about 1 hour 12 minutes), Grafana across multiple Grafana Cloud regions experienced elevated request errors, causing slow-loading or failing dashboards and Explore queries. The issue has been resolved and service has been fully restored.
- Monitoring · Nov 3, 13:28 UTC
We have identified the cause of the elevated request errors affecting Grafana across multiple Grafana Cloud regions and have applied a mitigation. We are seeing signs of recovery and are continuing to monitor closely. Customers may still see slow-loading or failing dashboards and Explore queries while recovery continues. Next update by 14:00 UTC.
- Monitoring · Nov 3, 13:00 UTC
We have identified the cause of the elevated request errors affecting Grafana across multiple Grafana Cloud regions and have applied a mitigation. We are seeing early signs of recovery and are monitoring closely. Customers may still see slow-loading or failing dashboards and Explore queries while recovery continues. Next update by 13:30 UTC.
- Identified · Nov 3, 12:57 UTC
We have identified the cause of the elevated request errors affecting Grafana across multiple Grafana Cloud regions and are applying mitigation. Customers may still see slow-loading or failing dashboards and Explore queries while recovery continues. Next update by 13:30 UTC.
- Investigating · Nov 3, 12:40 UTC
We are investigating some failed requests affecting Grafana across multiple Grafana Cloud regions, beginning at approximately 12:10 UTC. Customers may see slow-loading or failing dashboards and Explore queries, and some requests may time out or be cancelled. Our engineers are actively investigating. Next update by 13:30 UTC.
More from Grafana Labs
Full historyGitHub returning 403 for all Git Sync repositorieslasted 1h 58mOctober usage report visualization issuestill openIncreased query latency for Logs in GCP US Central (prod-us-central-0)lasted 1h 31mGrafana Cloud: k6 timeoutslasted 0mMetrics Alert Rule Evaluation Failures in AWS Australia (prod-au-southeast-1)lasted 1h 48mMissing Metrics for Some Synthetic Monitoring Checks using the Montreal, North Virginia, and North California public probeslasted 0m