Incidentsby GlyphTemporary Trace Ingestion Errors
Oct 31, 15:30 UTCOct 31, 15:30 UTC
Duration
0m
Impact
Informational
Root cause
Not disclosed
Grafana Labs, 90 days
66 incidents
Affected
Not listed by the vendor.
Final update
From approximately 16:30-8:15 UTC, a configuration change inadvertently removed a required headless service for hosted traces in one of our production regions. This caused elevated error rates and increased service-level objective (SLO) burn for the trace ingestion pathway. The underlying issue was a mismatch in internal configuration references following a prior migration. Re-enabling the headless service restored normal operation.
Timeline
- Resolved · Oct 31, 15:30 UTC
From approximately 16:30-8:15 UTC, a configuration change inadvertently removed a required headless service for hosted traces in one of our production regions. This caused elevated error rates and increased service-level objective (SLO) burn for the trace ingestion pathway. The underlying issue was a mismatch in internal configuration references following a prior migration. Re-enabling the headless service restored normal operation.
More from Grafana Labs
Full historySynthetic Monitoring , Jakarta Public Probe Degradationlasted 0mGitHub returning 403 for all Git Sync repositorieslasted 1h 58mOctober usage report visualization issuelasted 3d 3hIncreased query latency for Logs in GCP US Central (prod-us-central-0)lasted 1h 31mElevated Request Errors for Grafana Across Multiple Regionslasted 1h 1mGrafana Cloud: k6 timeoutslasted 0m