Incidentsby GlyphITDR Ingestion Delay
Aug 31, 16:53 UTCSep 20, 21:18 UTC
Duration
20d 4h
Impact
Minor
Root cause
Not disclosed
Huntress, 90 days
10 incidents
Affected
ITDR
Final update
Microsoft has deployed additional fixes to their infrastructure, and our internal telemetry is back to showing normal ingest timing for events. This incident is now resolved.
Timeline
- Resolved · Sep 20, 21:18 UTC
Microsoft has deployed additional fixes to their infrastructure, and our internal telemetry is back to showing normal ingest timing for events. This incident is now resolved.
- Monitoring · Sep 17, 15:18 UTC
Our partners at Microsoft have deployed a fix for the affected customers which will roll out over the remainder of this week and into the weekend. We will continue working with them to verify the fix is working as intended.
- Monitoring · Sep 16, 14:53 UTC
Our partners at Microsoft are in the process of rolling out a fix across the affected tenants. We are monitoring the effect and providing information to them on the effectiveness of this solution.
- Monitoring · Sep 14, 18:49 UTC
We are continuing to work with Microsoft to resolve the intermittent delays for a small set of affected customers.
- Monitoring · Sep 11, 15:55 UTC
We are continuing to provide information about any affected customers to our partners at Microsoft. Intermittent delays continue to show up for a small set of affected customers.
- Monitoring · Sep 1, 22:09 UTC
We're continuing to work with our partners at Microsoft, who have identified the cause of the delay and are in the process of implementing a fix. We are still seeing intermittent delays of up to 4 hours for some customers. We will continue to provide updates as soon as more information becomes available.
- Identified · Aug 31, 17:37 UTC
We've identified that the cause of our ingestion delay is from a degradation in Microsoft's M365 APIs. No GWS customer are affected. We're working through our partner channels to get this resolved, and will update this page as soon as we know more information.
- Investigating · Aug 31, 16:53 UTC
We have noticed a slight delay in our ITDR ingestion pipeline, and are investigating the cause.