Transaction Updates Delayed
Jul 22, 14:58 UTCJul 24, 10:02 UTC
Duration
43h 4m
Impact
Minor
Root cause
Dependency
Plaid, 90 days
16 incidents
Affected
Not listed by the vendor.
Final update
This incident is now resolved. The underlying issue has been fully mitigated, and all delayed default update webhooks have now been delivered. New item connections and their initial and historical transaction webhooks are firing normally, and new transaction updates, including default update webhooks, are being processed normally.
Timeline
- Resolved · Jul 24, 10:02 UTC
This incident is now resolved. The underlying issue has been fully mitigated, and all delayed default update webhooks have now been delivered. New item connections and their initial and historical transaction webhooks are firing normally, and new transaction updates, including default update webhooks, are being processed normally.
- Monitoring · Jul 24, 04:26 UTC
The third-party provider issue has been mitigated and recovery is progressing. New item connections and their initial and historical transaction webhooks are healthy and firing normally again, and new transaction updates, including default update webhooks, are being processed normally. We have resumed working through the remaining backlog of older default update webhooks and will continue to monitor closely as it clears.
- Investigating · Jul 24, 02:22 UTC
We are again experiencing delays in Transactions update webhooks due to a recurrence of the third-party service provider issue that caused the earlier degradation. We are working closely with the provider to mitigate the issue and restore normal processing as quickly as possible.
- Monitoring · Jul 23, 23:50 UTC
The underlying issue has been resolved, and recovery is progressing. Initial and historical transaction webhooks are now caught up, and new webhooks, including default webhooks, are being processed normally. We are continuing to work through the remaining backlog of older default update webhooks and expect all missing webhooks to be processed by the end of the day Pacific Time.
- Monitoring · Jul 23, 19:43 UTC
The underlying issue has been resolved, and recovery is progressing. Initial and historical transaction webhooks are now caught up, and new webhooks, including default webhooks, are being processed normally. We are continuing to work through the remaining backlog of older default update webhooks and expect all missing webhooks to be processed by the end of the day.
- Monitoring · Jul 23, 15:23 UTC
The underlying issue has been resolved, and recovery is progressing. Initial and historical transaction webhooks are now caught up, and new webhooks, including default webhooks, are being processed normally. We are continuing to work through the remaining backlog of older default update webhooks and expect all missing webhooks to be processed by the end of the day.
- Monitoring · Jul 22, 23:53 UTC
The underlying issue has been resolved, and recovery is progressing. Initial and historical transaction webhooks are now caught up, and new webhooks, including default webhooks, are being processed normally. We are continuing to work through the remaining backlog of older default webhook updates and are monitoring recovery closely.
- Investigating · Jul 22, 21:25 UTC
The underlying issue is resolved and the system continues to recover. Initial and Historical webhook queues are processing and Default webhooks will process when that completes.
- Investigating · Jul 22, 17:39 UTC
We are currently experiencing delays in Transactions updates. We are actively investigating and working to mitigate this issue. We are prioritizing recovery of initial and historical Transaction webhooks and then will work through the Default webhook queue. We expect the first group to complete in the next 2-3 hours. Webhooks are being sent on a FIFO basis.
- Investigating · Jul 22, 17:32 UTC
We are currently experiencing delays in Transactions updates. We are actively investigating and working to mitigate this issue. 1:21pm EST - We are prioritizing initial and full pull recovery. We will work on recovering default pulls next. We don't yet have an estimate to when these delays will be fully resolved.
- Investigating · Jul 22, 17:22 UTC
We are currently experiencing delays in Transactions updates. We are actively investigating and working to mitigate this issue. 12:41pm EST - We're currently monitoring initial/full pull recovery over the next five to eight hours, will provide more information soon 1:21pm EST - To clarify, we are prioritizing initial and full pull recovery. We will work on recovering default pulls next. We don't yet have an estimate to when these delays will be fully resolved.
- Investigating · Jul 22, 16:43 UTC
We are currently experiencing delays in Transactions updates. We are actively investigating and working to mitigate this issue. 12:41pm EST - We're currently monitoring initial/full pull recovery over the next five to eight hours, will provide more information soon
- Investigating · Jul 22, 14:58 UTC
We are currently experiencing delays in Transactions updates. We are actively investigating and working to mitigate this issue.
More from Plaid
Full historyService Disruption Impacting Chase connectionslasted 9h 23mElevated /signal/evaluate errorslasted 39mTransfer RTP Requests are Temporarily Disabled for Some Customerslasted 1h 9mService Disruption Impacting Wells Fargo connectionslasted 12h 25mService Disruption Impacting Chase connectionslasted 18mService Disruption Impacting Chase connectionslasted 1h 19m