Incidentsby IbisSome customers in us-west-2 may experience elevated rate of API errors & higher latencies
May 29, 13:31 UTCMay 29, 15:49 UTC
Duration
2h 18m
Impact
Minor
Root cause
Database
Temporal, 90 days
27 incidents
Affected
us-west-2Amazon Web Services (AWS) - us-west-2
Final update
Remediation has been completed at this time and we have confirmed the recovery of all affected namespaces.
Timeline
- Resolved · May 29, 15:49 UTC
Remediation has been completed at this time and we have confirmed the recovery of all affected namespaces.
- Monitoring · May 29, 15:00 UTC
Latencies and error rates are stable after remediation and we are continuing to monitoring it.
- Identified · May 29, 14:50 UTC
Error rates remain stable, and we continue to implement remediation measures while monitoring service performance closely.
- Identified · May 29, 14:03 UTC
We continue to investigate the root cause while implementing remediation measures. Error rates have recovered.
- Identified · May 29, 13:43 UTC
Failover was initiated for the affected HA namespaces while we keep investigating the root cause.
- Identified · May 29, 13:31 UTC
Some customers in us-west-2 may experience elevated rate of API errors & higher latencies
More from Temporal
Full historyNew namespace creation restricted across all GCP regionsstill openElevated latency and errors in GCP us-west1lasted 17mElevated latency and errors in AWS us-east-1lasted 2h 17mElevated error rates for cross-region traffic between US AWS regionslasted 0mElevated GCP storage latency in us-central1 – precautionary namespace failoverlasted 4h 25mIncreased errors on Temporal Cloud control plane, Cloud Ops API, and Temporal UIlasted 29m