Incidentsby GlyphOhio Services with Disks, Data Persistence failing
Mar 3, 00:41 UTCMar 3, 01:00 UTC
Duration
20m
Impact
Major
Root cause
Database
Render, 90 days
15 incidents
Affected
Web ServicesBackground WorkersPostgreSQLRedisOhio - Web ServicesOhio - Background WorkersOhio - PostgreSQLOhio - Redis
Final update
All affected services have been migrated to new hosts and are running at this time. This incident has been resolved.
Timeline
- Resolved · Mar 3, 01:00 UTC
All affected services have been migrated to new hosts and are running at this time. This incident has been resolved.
- Identified · Mar 3, 00:59 UTC
Many services have recovered with healthy disks on new hosts, however this issue remains open as some services are still impaired.
- Identified · Mar 3, 00:41 UTC
Services located in Ohio, both application types (Web Service, Private Service, etc.) and data types (Postgres, Key Value with persistence) are failing due to disk errors. We are working on mitigations.
More from Render
Full historyIssues affecting connections to a subset of Postgres Databaseslasted 3h 44mPlatform emails not being sentlasted 30mDegraded network performancelasted 1h 28mDeploying from GitLab may intermittently faillasted 3h 41mDashboard not updating in real time; Some Workflows SDK requests timing outlasted 1h 35mBuilds, deploys, and queued deploys are not starting in all regionslasted 2h 45m