Who is down right now

26 down now · 292 this week

Humanloop · AIMar 11, 2025, 06:00 UTC 1 year ago

Partial service outage

MajorNot disclosedUpdated 4d ago
Mar 11, 06:00 UTCMar 11, 07:50 UTC
Duration
1h 50m
Impact
Major
Root cause
Not disclosed
Humanloop, 90 days
0 incidents
Affected
Not listed by the vendor.
Status page

Final update

Services are all now back to normal

Timeline

  1. Resolved · Mar 11, 07:50 UTC
    Services are all now back to normal
  2. Monitoring · Mar 11, 07:00 UTC
    Monitoring services after fix deployed and failover to read replica completed.
  3. Identified · Mar 11, 06:50 UTC
    Post deployment database backfill activities caused anomalous database load leading to severe contention and downtime. Contention was resolved by promoting read replica to master writer instance.
  4. Investigating · Mar 11, 06:00 UTC
    Early warning alarms around master database load triggered. On call engineer pinged, investigation started.

More from Humanloop

Full history