Event Streams Service Disruption

Final update

Our engineers have monitored the fix and confirmed the issue with event streams has been resolved. All services are now operating normally at this time.

1. Resolved · Sep 25, 02:24 UTC
   Our engineers have monitored the fix and confirmed the issue with event streams has been resolved. All services are now operating normally at this time.
2. Identified · Sep 25, 02:05 UTC
   Our team is continuing to work on restoring a small subset of events that are delayed from 12:12 PM PT and customers may see some duplicate events happened for a few minutes around 12:13 PM PT. We expect to provide another update in 1 hour or as soon as more information becomes available.
3. Identified · Sep 25, 01:03 UTC
   Our team is continuing to work on restoring a small subset of events that are delayed from 12:12 PM PT and customers may see some duplicate events happened for a few minutes around 12:13 PM PT. We expect to provide another update in 1 hour or as soon as more information becomes available.
4. Identified · Sep 24, 23:45 UTC
   Our team is continuing to work on restoring a small subset of events that are delayed from 12:12 PM PT and customers may see some duplicate events happened for a few minutes around 12:13 PM PT. We expect to provide another update in 1 hour or as soon as more information becomes available.
5. Identified · Sep 24, 22:13 UTC
   Our engineers have identified the issue and are working on restoring delayed events. A subset of mail-send related events (such as Delivered, Deferred, Bounces, Spam Reports) for Email Activity, Marketing Campaigns, Group Unsubscribes, and Event Webhook are delayed for a specific time window (TBD). Email stats for new mail recieved after 12:15 PM PT is not impacted and no event data was lost. We will provide an update in 1 hour or as soon as more information becomes available.
6. Investigating · Sep 24, 20:44 UTC
   Our engineers are continuing to investigate the issue causing inconsistencies with Event Streams. We expect to provide another update in 2 hours or as soon as more information becomes available.
7. Investigating · Sep 24, 20:40 UTC
   We are investigating a service interruption with Event Streams. Customers may experience event data loss or delays in event delivery. Our team is actively investigating the scope of impact and data recovery options. We expect to provide another update in 2 hours or as soon as more information becomes available.