Mimir query unexpected response with status code 500
Nov 10, 03:38 UTCNov 10, 12:09 UTC
Duration
8h 32m
Impact
Minor
Root cause
Not disclosed
Grafana Labs, 90 days
68 incidents
Affected
AWS US East - prod-us-east-0: QueryingGCP US Central - prod-us-central-0: QueryingMetrics - AWS US East - prod-us-east-0: QueryingMetrics - GCP US Central - prod-us-central-0: Querying
Final update
From approximately 03:20 UTC on 10 November, some Grafana Cloud Metrics customers in AWS US East (OH) and GCP US Central saw errors, or failed queries, when reading older data. Service has fully recovered and this incident is now resolved.
Timeline
- Resolved · Nov 10, 12:09 UTC
From approximately 03:20 UTC on 10 November, some Grafana Cloud Metrics customers in AWS US East (OH) and GCP US Central saw errors, or failed queries, when reading older data. Service has fully recovered and this incident is now resolved.
- Monitoring · Nov 10, 11:18 UTC
Our mitigation remains in place and query behavior in AWS US East (OH) and GCP US Central continues to improve. Some customers may still see occasional, brief query errors in the meantime.
- Monitoring · Nov 10, 09:53 UTC
Our mitigation remains in place and query behavior in AWS US East (OH) and GCP US Central continues to improve. Some customers may still see occasional, brief query errors in the meantime.
- Monitoring · Nov 10, 04:42 UTC
A fix has been implemented and we are monitoring the results.
- Identified · Nov 10, 03:39 UTC
We’ve identified an issue querying older data, and are already seeing recovery.
- Identified · Nov 10, 03:38 UTC
We’ve identified an issue querying older data, and are already seeing recovery
More from Grafana Labs
Full historyCloud Logs writes - Elevated latency and error rates in prod-eu-central-0lasted 17mSynthetic Monitoring , Jakarta Public Probe Degradationlasted 0mGitHub returning 403 for all Git Sync repositorieslasted 1h 58mOctober usage report visualization issuelasted 3d 3hIncreased query latency for Logs in GCP US Central (prod-us-central-0)lasted 1h 31mElevated Request Errors for Grafana Across Multiple Regionslasted 1h 1m