Who is down right now

26 down now · 292 this week

Convex · Data and observabilityMar 12, 2026, 13:17 UTC 8 months ago

elevated error rates on convex for pro instances

InformationalCapacityUpdated 49d ago
Mar 12, 13:17 UTCMar 12, 13:55 UTC
Duration
38m
Impact
Informational
Root cause
Capacity
Convex, 90 days
4 incidents
Affected
Free & StarterLive Traffic - Node.js RuntimeLive Traffic - Database ServicesLive Traffic - Function Runtime
Status page

Final update

At approximately 6am Pacific we had a high rate of server errors for approximately 30 minutes for a subset of customer backends. This was caused by a 20x increase in traffic that saturated one of our databases and led to resource exhaustion on one of our function execution clusters, which is why the impact spilled beyond just that specific database. We’ve scaled up our infrastructure further and are deploying some changes to load shedding and retries to minimize the impact of such an event in future. While we aim to have no cascading impact on other customers from a load spike we’re also deploying dedicated isolated hardware for customers on our upcoming Business plan.

Timeline

  1. Postmortem · Mar 13, 03:36 UTC
    At approximately 6am Pacific we had a high rate of server errors for approximately 30 minutes for a subset of customer backends. This was caused by a 20x increase in traffic that saturated one of our databases and led to resource exhaustion on one of our function execution clusters, which is why the impact spilled beyond just that specific database. We’ve scaled up our infrastructure further and are deploying some changes to load shedding and retries to minimize the impact of such an event in future. While we aim to have no cascading impact on other customers from a load spike we’re also deploying dedicated isolated hardware for customers on our upcoming Business plan.
  2. Resolved · Mar 12, 13:55 UTC
    This incident has been resolved.
  3. Monitoring · Mar 12, 13:40 UTC
    A fix has been implemented and we are monitoring the results.
  4. Investigating · Mar 12, 13:17 UTC
    We are currently investigating an issue leading to higher error rates

More from Convex

Full history

Also caused by capacity and load

All