Incidentsby GlyphOversized Bot Management feature file breaks the core proxy
Nov 18, 11:20 UTCNov 18, 17:06 UTC
Duration
5h 46m
Impact
Critical
Root cause
Config change
Cloudflare, 90 days
187 incidents
Affected
CDNBot ManagementWorkers KVAccessTurnstileDashboardGlobal
Lesson: Treat internally generated config files like user input: validate size and shape before they propagate, and keep global kill switches for each feature.
What happened
A change to database permissions made a ClickHouse query return duplicate rows, doubling the Bot Management feature file. The larger file exceeded a limit in the core proxy, which returned HTTP 5xx errors across the network. Core traffic was largely normal by 14:30 UTC and all systems by 17:06 UTC.