Oversized Bot Management feature file breaks the core proxy
Nov 18, 11:20 UTCNov 18, 17:06 UTC
Duration
5h 46m
Impact
Critical
Root cause
Config change
Cloudflare, 90 days
56 incidents
Affected
CDNBot ManagementWorkers KVAccessTurnstileDashboardGlobal
Lesson: Treat internally generated config files like user input: validate size and shape before they propagate, and keep global kill switches for each feature.
What happened
A change to database permissions made a ClickHouse query return duplicate rows, doubling the Bot Management feature file. The larger file exceeded a limit in the core proxy, which returned HTTP 5xx errors across the network. Core traffic was largely normal by 14:30 UTC and all systems by 17:06 UTC.
More from Cloudflare
Full history| Started | Incident | Impact | Duration |
|---|---|---|---|
| Sep 2500:03 UTC | Issues with Durable Objects | minor | Ongoing |
| Sep 2420:40 UTC | Intermittent 500 errors for backend services on Replicate | minor | 1h 54m |
| Sep 2417:33 UTC | Delays Starting Cloudflare Workers Builds | minor | 3h 19m |
| Sep 2417:26 UTC | Browser isolation session data sync issues | minor | 5h 11m |
| Sep 2415:20 UTC | Cloudflare Access updates delayed | minor | 3h 5m |
| Sep 2410:22 UTC | Customers using BYOIP can have issues updating their BGP prefixes, including advertising or withdrawing prefixes. | minor | 3m |
Also caused by configuration change
All| Started | Vendor | Incident | Impact | Duration |
|---|---|---|---|---|
| Sep 2423:36 UTC |  Render | Key Value service restartsRender | minor | 0m |
| Sep 2414:08 UTC |  HubSpot | The Salesforce integration is unavailable for some customersHubSpot | none | 7h 18m |
| Sep 2218:20 UTC |  Twilio | Phone Number APIs and Console Were Returning Incorrect 404 ResponsesTwilio | none | 0m |
| Sep 1211:04 UTC |  Slack | Some customers experiencing blurry image previews and download issuesSlack | minor | 8h 56m |
| Sep 1107:18 UTC |  Snowflake | INC20000213Snowflake | critical | 6h 1m |
| Sep 407:34 UTC | Snowflake | INC20000199Snowflake | critical | 2h 36m |
Sources: vendors' own status pages, published postmortems and SEC 8-K Item 1.05 filings, read daily. Times as reported. Logos via logo.dev; trademarks belong to their owners.