Increased Errors across Affirm Services
Nov 25, 20:28 UTCNov 26, 04:52 UTC
Duration
8h 24m
Impact
Critical
Root cause
Database
Affirm, 90 days
1 incidents
Affected
USUS - Shop Pay InstallmentsUS - Platforms / Payment Gateways - US - Shop Pay InstallmentsAffirm Virtual Card Network - USAffirm Checkout - US
Final update
The issue has been resolved and all services have been restored to full functionality.
Timeline
- Resolved · Nov 26, 04:52 UTC
The issue has been resolved and all services have been restored to full functionality.
- Monitoring · Nov 26, 04:03 UTC
A fix has been implemented and we are monitoring the results.
- Identified · Nov 26, 04:01 UTC
We are continuing to work on a fix for this issue.
- Identified · Nov 26, 02:53 UTC
We are performing a controlled recovery and expect resolution in about 1 hour.
- Identified · Nov 26, 01:37 UTC
Affirm is still experiencing degraded service and high error rates. We are continuing to work on identifying a solution to mitigate this issue.
- Identified · Nov 26, 00:40 UTC
Affirm's end to end conversion rate has dropped sharply due to increased error rates. We are continuing to work on identifying a solution to mitigate this issue.
- Identified · Nov 25, 23:27 UTC
We are continuing to work on a fix for this issue.
- Identified · Nov 25, 23:20 UTC
The recent update has resulted in Shop Pay Installments availability to drop substantially. We will update as soon as possible.
- Identified · Nov 25, 23:13 UTC
We're monitoring the effects of a change our teams just implemented. We will update on next steps as soon as possible.
- Identified · Nov 25, 22:35 UTC
The root cause is related to excessive traffic on our backend database. Our teams are working on relieving the load on our database. We are implementing a fix to cause temporary relief that will allow recovery to occur.
- Identified · Nov 25, 21:59 UTC
Our services are still impacted. We are working on additional mitigation methods to restore service.
- Identified · Nov 25, 21:38 UTC
Our teams have implemented a fix and are monitoring to see if services recover.
- Identified · Nov 25, 21:30 UTC
Our teams have identified the issue and are working on implementing fixes to mitigate the issue.
- Investigating · Nov 25, 20:59 UTC
We are still working on identifying the root cause of the issue and mitigating the issue. We will update as soon as possible.
- Investigating · Nov 25, 20:28 UTC
Our teams have identified an increase in errors across Affirm services that are impacting checkouts. We are working on identifying the root cause of the issue to mitigate the impact. We will update as soon as possible.
More from Affirm
Full history| Started | Incident | Impact | Duration |
|---|---|---|---|
| Aug 1221:12 UTC | Affirm Services Degradation | minor | 48m |
| Jun 2214:36 UTC | Checkout Degradation | minor | 29m |
| Jun 1723:22 UTC | Sandbox checkout confirm_loan failures | minor | 1h 22m |
| Jun 1015:06 UTC | CA Checkout Degradation | major | 52m |
| Jun 408:01 UTC | Checkout Degradation | minor | 1h 35m |
| Jun 319:00 UTC | Degradation on Sandbox APIs | major | 1h 49m |
Also caused by database
All| Started | Vendor | Incident | Impact | Duration |
|---|---|---|---|---|
| Sep 1309:16 UTC | Incident with several GitHub Services | critical | 1h 29m | |
| Sep 921:53 UTC | INC20000211 | critical | 1h 58m | |
| Aug 2616:17 UTC | Issues performance is degraded in the US | minor | 4h 19m | |
| Aug 1216:16 UTC | Incident with Pull Requests and Issues | minor | 25m | |
| Aug 818:57 UTC | Account Registration, Droplets, and Related Services | minor | 6h 3m | |
| Jul 2817:41 UTC | Event ingestion and retrieval delays for some North America customers | none | 2h |
Sources: vendors' own status pages, published postmortems and SEC 8-K Item 1.05 filings, read daily. Times as reported. Logos via logo.dev; trademarks belong to their owners.