Flag delivery errors in Streaming API
Final update
This incident has been resolved. Between 12:00 and 12:12 PM PDT on August 26, server-side SDKs establishing new streaming connections experienced elevated error rates. Existing streaming connections were not affected, and polling requests continued to be served with error rates below 1%. Some requests to the LaunchDarkly web application also experienced elevated errors between approximately 11:29 AM and 12:03 PM PDT. The issue was caused by an internal configuration change, which has been reverted. All systems have recovered and are operating normally. No customer data was lost or compromised.
Timeline
- Resolved · Aug 26, 20:00 UTC
This incident has been resolved. Between 12:00 and 12:12 PM PDT on August 26, server-side SDKs establishing new streaming connections experienced elevated error rates. Existing streaming connections were not affected, and polling requests continued to be served with error rates below 1%. Some requests to the LaunchDarkly web application also experienced elevated errors between approximately 11:29 AM and 12:03 PM PDT. The issue was caused by an internal configuration change, which has been reverted. All systems have recovered and are operating normally. No customer data was lost or compromised.
- Monitoring · Aug 26, 19:45 UTC
Server-side streaming requests experienced elevated error rates between approximately 12:00 p.m. and 12:12 p.m. PDT, and some SDKs establishing new streaming connections may have been unable to connect. Polling request error rates remained below 1% throughout the incident. No customer data was lost or compromised. We apologize for the disruption.
More from LaunchDarkly
Full history| Started | Incident | Impact | Duration |
|---|---|---|---|
| Aug 2412:20 UTC | Email delivery failure affecting sign-up verification and notification emails | minor | 1h 15m |
| Aug 1710:36 UTC | Flag Updates via Polling API delayed | minor | 5h 26m |
| Aug 1414:47 UTC | Flag Updates delayed in US Region | minor | 2h |
| Aug 719:29 UTC | Elevated errors on server-side SDK streaming connections | minor | 46m |
| Jul 3117:44 UTC | Investigating - Known Impact | minor | 5h 10m |
| Jul 2819:24 UTC | Delay in Session Replay and Error ingest | minor | 41m |
Also caused by configuration change
All| Started | Vendor | Incident | Impact | Duration |
|---|---|---|---|---|
| Sep 2423:36 UTC |  Render | Key Value service restartsRender | minor | 0m |
| Sep 2414:08 UTC |  HubSpot | The Salesforce integration is unavailable for some customersHubSpot | none | 7h 18m |
| Sep 2218:20 UTC |  Twilio | Phone Number APIs and Console Were Returning Incorrect 404 ResponsesTwilio | none | 0m |
| Sep 1622:30 UTC |  Cloudflare | Hyperdrive Elevated Origin Connection Failure RatesCloudflare | none | 0m |
| Sep 1211:04 UTC |  Slack | Some customers experiencing blurry image previews and download issuesSlack | minor | 8h 56m |
| Sep 1107:18 UTC |  Snowflake | INC20000213Snowflake | critical | 6h 1m |
Sources: vendors' own status pages, published postmortems and SEC 8-K Item 1.05 filings, read daily. Times as reported. Logos via logo.dev; trademarks belong to their owners.