Batch job processing delayed
Jul 27, 02:53 UTCJul 27, 21:28 UTC
Duration
18h 35m
Impact
Minor
Root cause
Storage
Databento, 90 days
13 incidents
Affected
Historical
Final update
This incident has been resolved.
Timeline
- Resolved · Jul 27, 21:28 UTC
This incident has been resolved.
- Monitoring · Jul 27, 17:12 UTC
We have taken additional internal steps to remediate this issue, and we're seeing improved batch job processing times (as of 10 minutes ago).
- Monitoring · Jul 27, 04:18 UTC
The batch job processing system has caught up on the backlog of jobs, we're continuing to monitor the system.
- Identified · Jul 27, 02:53 UTC
Our batch job processing system fell behind due to internal storage degradation. Processing is resuming with the backlog of jobs. (There is no need to re-submit any jobs.)
More from Databento
Full history| Started | Incident | Impact | Duration |
|---|---|---|---|
| Sep 415:12 UTC4 weeks ago | OPRA historical data for 2026-09-04 delayed | none | 2d 23h |
| Aug 2807:59 UTC5 weeks ago | Live authentication timeouts in NY4 | critical | 6h 50m |
| Aug 2109:56 UTC6 weeks ago | Batch jobs being queued indefinitely | major | 3h 34m |
| Aug 1319:04 UTC7 weeks ago | GLBX.MDP3 intraday data is delayed | none | 5h 5m |
| Aug 1022:55 UTC7 weeks ago | GLBX.MDP3 delays in the Historical API | minor | 4h 3m |
| Aug 615:05 UTC8 weeks ago | Databento portal and historical API down | critical | 4h 26m |
Also caused by storage
All| Started | Vendor | Incident | Impact | Duration |
|---|---|---|---|---|
| Sep 2022:13 UTC2 weeks ago | Incident with Pull Requests | minor | 1h 9m | |
| Sep 1715:50 UTC2 weeks ago | AutoOps node metrics temporarily unavailable in some regions | major | 41m | |
| Sep 1015:26 UTC3 weeks ago | Unresponsive Projects | major | 27h 40m | |
| Aug 2623:37 UTC5 weeks ago | Disruption with GitHub Billing | minor | 20h 7m | |
| Aug 2413:56 UTC5 weeks ago | Actions delays in starting runs | minor | 38m | |
| Aug 1316:21 UTC7 weeks ago | Disruption with GHEC Team Sync | minor | 2h 6m |