Cloudflare · Cloud and hostingJul 2, 2019, 13:42 UTC

WAF regular expression exhausts CPU worldwide for 27 minutes

CriticalDeploymentUpdated 47d ago

Jul 2, 13:42 UTCJul 2, 14:09 UTC

Duration

27m

Impact

Critical

Root cause

Deployment

Cloudflare, 90 days

187 incidents

Affected

CDNWAFGlobal

Read the postmortem Status page

Lesson: Rules and regexes are code; stage them, and use an engine with guaranteed linear time for untrusted input.

What happened

A new WAF rule for XSS detection contained a regular expression with catastrophic backtracking. Deployed globally through an automatic process, it pushed CPU to 100% on every HTTP server and the network dropped traffic for 27 minutes.

More from Cloudflare

Full history

MinorNov 7, 04:55 UTC

Network Performance Issues in Chicagolasted 17m

MinorNov 7, 00:59 UTC

Spectrum Availability Issue in Phoenixlasted 2d 13h

MinorOngoingNov 6, 17:47 UTC

Fedramp Customers have Delayed Purgesstill open

MinorNov 6, 09:47 UTC

Wrangler Deployments, Pages, and Workers Builds failureslasted 21m

MinorNov 6, 08:54 UTC

Network Performance Issues in Singaporelasted 1h 50m

MinorNov 5, 19:51 UTC

Workers Builds Issueslasted 29m

Also caused by deployment or rollout

All

MinorSep 11, 19:47 UTC

UiPath: Some customers will not be able to deploy solution packageslasted 1h 46m

NoticeSep 3, 15:34 UTC

UiPath: Degraded-performance in MLS Observer thruough DU - USlasted 43m

CriticalAug 20, 14:43 UTC

GitHub: Intermittent failures creating agent tasksBetween 13:57 UTC on August 20 and 00:37 UTC on August 21, 2026, some users of the Copilot Cloud Agent experienced delays of up to 60 to 90 minutes in seeing the status and results of their agent tasks.lasted 9h 54mPostmortem

MajorAug 20, 00:31 UTC

Box: [Medium] Issue with Microsoft Office Integration and Box Editlasted 1h 13m

MinorAug 6, 22:22 UTC

Slack: Trouble Using Search Bar For Some Adminslasted 2h 44m

NoticeJul 22, 20:20 UTC

Amplitude: Some users may have experienced errors when accessing Amplitude.lasted 0m