Who is down right now

20 down now · 248 this week

Upstash · Data and observabilityJun 23, 2026, 16:22 UTC 4 months ago

QStash EU Region , Degraded Performance

MinorCapacityUpdated 47d ago
Jun 23, 16:22 UTCJun 23, 17:14 UTC
Duration
52m
Impact
Minor
Root cause
Capacity
Upstash, 90 days
4 incidents
Affected
EU-CENTRAL-1QStash - EU-CENTRAL-1
Status page

Final update

### **Summary** On June 23, 2026, QStash users in the EU-CENTRAL-1 region experienced elevated latency and degraded performance. The issue was caused by a regression introduced in a recent enhancement to Flow Control scheduling logic. The change increased resource consumption under load, leading to reduced performance on affected shards. The incident was resolved by rolling back to the previous stable version. ### **Impact** * Service: QStash \(EU-CENTRAL-1\) * Start: 2026-06-23 16:22 UTC * Resolved: 2026-06-23 17:14 UTC * Duration: ~52 minutes * Impact: Increased latency and degraded performance for workloads routed to the affected instance in the EU. Multiple customers in the region experienced slower request processing during the incident window. ### **Root Cause** We recently deployed an enhancement to our Flow Control feature, designed to improve fairness between independent flow controls when unused global parallelism capacity was available. Under production load, the new behavior introduced a regression that significantly increased resource utilization. The elevated resource consumption caused performance degradation on the affected QStash instamce, impacting all workloads s

Timeline

  1. Postmortem · Jun 24, 12:28 UTC
    ### **Summary** On June 23, 2026, QStash users in the EU-CENTRAL-1 region experienced elevated latency and degraded performance. The issue was caused by a regression introduced in a recent enhancement to Flow Control scheduling logic. The change increased resource consumption under load, leading to reduced performance on affected shards. The incident was resolved by rolling back to the previous stable version. ### **Impact** * Service: QStash \(EU-CENTRAL-1\) * Start: 2026-06-23 16:22 UTC * Resolved: 2026-06-23 17:14 UTC * Duration: ~52 minutes * Impact: Increased latency and degraded performance for workloads routed to the affected instance in the EU. Multiple customers in the region experienced slower request processing during the incident window. ### **Root Cause** We recently deployed an enhancement to our Flow Control feature, designed to improve fairness between independent flow controls when unused global parallelism capacity was available. Under production load, the new behavior introduced a regression that significantly increased resource utilization. The elevated resource consumption caused performance degradation on the affected QStash instamce, impacting all workloads sharing that instance. ### **Resolution** After identifying the regression as the source of the slowdown, we rolled back the deployment to the last known stable version. * Rollback completed: 2026-06-23 19:18:46.84 UTC Following the rollback, system performance returned to normal levels and service s
  2. Resolved · Jun 23, 17:14 UTC
    This incident has been resolved.
  3. Monitoring · Jun 23, 16:32 UTC
    A fix has been implemented and we are monitoring the results.
  4. Investigating · Jun 23, 16:22 UTC
    We are currently investigating this issue.

More from Upstash

Full history

Also caused by capacity and load

All