Error ingestion degraded in US region
Jul 10, 22:43 UTCJul 11, 03:34 UTC
Duration
4h 51m
Impact
Critical
Root cause
Not disclosed
Sentry, 90 days
25 incidents
Affected
US Ingestion - US Error IngestionAlerting - US Errors Alerting
Final update
We have processed the backlog.
Timeline
- Resolved · Jul 11, 03:34 UTC
We have processed the backlog.
- Monitoring · Jul 11, 03:28 UTC
We have processed the majority of our backlog. We are now recovering the tail of the backlog.
- Identified · Jul 11, 02:37 UTC
We have processed the majority of our backlog. We are now recovering an additional 25M error events ranging from 22:20 to 23:20 UTC, which should complete in the next 2 hours
- Identified · Jul 11, 01:46 UTC
We expect recovery of historical data to be complete within the next 2 hours.
- Identified · Jul 11, 00:56 UTC
We are still working on recovery. New errors should be processed in real time, and we are working on recovering historical data.
- Identified · Jul 11, 00:04 UTC
We have identified the problem and are working on recovery. New errors are being processed in real time and we are in the process of consuming backlogged errors.
- Identified · Jul 10, 23:02 UTC
We have identified the problem and are working on a fix, ingested errors are delayed and will be processed once the fix is in place.
- Investigating · Jul 10, 22:43 UTC
We are currently investigating this issue.