Platform Log ingestion issue
Jul 30, 16:02 UTCJul 30, 22:38 UTC
Duration
6h 35m
Impact
Minor
Root cause
Not disclosed
Astronomer, 90 days
17 incidents
Affected
Not listed by the vendor.
Final update
Ingestion lag is back at baseline
Timeline
- Resolved · Jul 30, 22:38 UTC
Ingestion lag is back at baseline
- Monitoring · Jul 30, 21:47 UTC
We are continuing to see progress as the log injestions works through the backlog. Available logs are now about ~20mins old, and we expect to see that latency return to normal baseline in the next 2 hours.
- Monitoring · Jul 30, 20:05 UTC
We continue to see log ingestion improving on the platform, and we're continuing to monitor the situation.
- Monitoring · Jul 30, 19:06 UTC
We are beginning to see ingestion catch up with backlog log load. Customers may begin to see some logs becoming available in the Cloud UI
- Identified · Jul 30, 18:43 UTC
We are continuing to investigate and monitor this issue.
- Identified · Jul 30, 18:03 UTC
We have identified the source of increased log generation volume, which is causing issues on the ingestion side. We're implementing mitigations now and are monitoring the results.
- Investigating · Jul 30, 16:33 UTC
We are continuing to investigate this issue, and will provide another update in 30mins or as new information is available.
- Investigating · Jul 30, 16:02 UTC
We are currently investigating an issue with platform log ingestion for Astro. Logs may not be available in the Cloud UI.
More from Astronomer
Full historyPHP Security Advisory: Astro and Astro Private Cloud Not Affected; Review Custom Imagesstill openSecurity Update: Runtime Releases and Guidance for the eMudhra/emSign Certificate Issuelasted 45h 27mDelayed delivery of Astro Alertslasted 26h 9mOrganization Dashboards are failing to load with Invalid kid parameter errorlasted 1h 23mAstro UI and API are appearing offline for some userslasted 1h 51mCapacity constraints in GCP us-central1 may delay or fail some task executionslasted 7d 21h