elevated latency/errors for a subset of customers
Oct 28, 21:38 UTCOct 28, 23:50 UTC
Duration
2h 12m
Impact
Informational
Root cause
Database
Convex, 90 days
6 incidents
Affected
Free & StarterLive Traffic - Database Services
Final update
This incident has been resolved. The specific cause was due to a maintenance operation from a downstream provider which resulted in a database cluster switching to using an unoptimized table format. We'll be working to ensure the underlying cause is resolved.
Timeline
- Resolved · Oct 28, 23:50 UTC
This incident has been resolved. The specific cause was due to a maintenance operation from a downstream provider which resulted in a database cluster switching to using an unoptimized table format. We'll be working to ensure the underlying cause is resolved.
- Identified · Oct 28, 21:50 UTC
Service has been restored to Pro customers and we're working on restoring free deployments.
- Investigating · Oct 28, 21:38 UTC
We're investigating an issue with a database cluster leading to degraded performance and higher error rates for affected deployments.
More from Convex
Full history| Started | Incident | Impact | Duration |
|---|---|---|---|
| Sep 2821:09 UTC5 days ago | AI gateway unavailable | critical | 1h 4m |
| Sep 1823:01 UTC2 weeks ago | Dashboard logins degraded | none | 1h 7m |
| Aug 1623:33 UTC6 weeks ago | Account verification/team invite emails not working | major | 1h 53m |
| Aug 1203:41 UTC7 weeks ago | Some HTTP Actions returning 404 | minor | 20m |
| Aug 501:42 UTC8 weeks ago | Action failures for business deployments that use createFunctionHandle | major | 15m |
| Jul 1505:35 UTC2 months ago | New deployments unable to push Node Actions to Convex | minor | 57m |
Also caused by database
All| Started | Vendor | Incident | Impact | Duration |
|---|---|---|---|---|
| Sep 2310:11 UTC10 days ago | Incident across several services | minor | 18h 44m | |
| Sep 1309:16 UTC2 weeks ago | Incident with several GitHub Services | critical | 1h 29m | |
| Sep 921:53 UTC3 weeks ago | INC20000211 | critical | 1h 58m | |
| Aug 2616:17 UTC5 weeks ago | Issues performance is degraded in the US | minor | 4h 19m | |
| Aug 1216:16 UTC7 weeks ago | Incident with Pull Requests and Issues | minor | 25m | |
| Aug 818:57 UTC8 weeks ago | Account Registration, Droplets, and Related Services | minor | 6h 3m |