Free Plan Customers Are Experiencing Message Failures
Oct 1, 12:39 UTCOct 1, 18:59 UTC
Duration
6h 20m
Impact
Minor
Root cause
Dependency
Slack, 90 days
8 incidents
Affected
Messaging
Final update
A messaging integration briefly caused some free-plan workspaces to exceed their daily messaging limit, blocking them from sending messages. We took action to lift the sending block for all affected free-plan workspaces, and have confirmed messaging is now restored. We apologize for how this incident affected you and your business. We will undertake a full investigation of the incident, establishing the technical trigger, the underlying cause, and preventive action to avoid a repeat in the future.
Timeline
- Resolved · Oct 1, 18:59 UTC
A messaging integration briefly caused some free-plan workspaces to exceed their daily messaging limit, blocking them from sending messages. We took action to lift the sending block for all affected free-plan workspaces, and have confirmed messaging is now restored. We apologize for how this incident affected you and your business. We will undertake a full investigation of the incident, establishing the technical trigger, the underlying cause, and preventive action to avoid a repeat in the future.
- Update · Oct 1, 18:52 UTC
Deployment of the fix is underway to restore access for impacted free plan workspaces, and we are actively monitoring the environment. More information will follow as we progress through this stage of remediation.
- Update · Oct 1, 14:31 UTC
We’ve identified a potential fix for this issue and we’re working on getting this out to users as quickly as possible. We’ll be back once we have another update to share. Thank you for your continued patience as we work towards a solution.
- Update · Oct 1, 13:58 UTC
A feature toggle ramp-up caused an app integration to repeatedly post messages in a processing loop. This pushed free plan customers over the message volume threshold, triggering the sending block. We're working to restore access and are actively working on remediation options for impacted Workspaces. We apologize for the inconvenience and will provide another update as soon as we have more details to share.
- Update · Oct 1, 12:56 UTC
We identified that a third-party app is stuck in a loop that has triggered billions of messages. This is causing free plan users to exceed messaging limits. We apologize for the inconvenience and will provide another update as soon as we have more details to share.
- Update · Oct 1, 12:39 UTC
We're aware that some customers on our free plan are experiencing message-sending limit blocks in an unusual spike pattern, which is preventing them from sending messages. We're investigating why this message limit was triggered unexpectedly for some teams. If you're affected, you may be temporarily unable to send messages until this is resolved. We'll provide an update in 30 minutes or sooner if additional information becomes available.
More from Slack
Full history| Started | Incident | Impact | Duration |
|---|---|---|---|
| Sep 1211:04 UTC2 weeks ago | Some customers experiencing blurry image previews and download issues | minor | 8h 56m |
| Sep 1000:34 UTC3 weeks ago | Trouble loading Workspace-level Apps & Workflows settings page is returning blank pages | minor | 2h 40m |
| Aug 1321:21 UTC7 weeks ago | Trouble Accessing Historical Messages With Custom Data Retention Policies Enabled | minor | 32d 1h |
| Aug 622:22 UTC8 weeks ago | Trouble Using Search Bar For Some Admins | minor | 2h 44m |
| Jul 2802:13 UTC2 months ago | Feature Degradation Affecting the Receipt of Emails in Slack | minor | 1h 30m |
| Jul 2408:22 UTC2 months ago | Issue sending messages to channels via certain Workflows | minor | 2h 24m |
Also caused by third-party dependency
All| Started | Vendor | Incident | Impact | Duration |
|---|---|---|---|---|
| Sep 2702:15 UTC5 days ago | INC20000239 | critical | 1h 1m | |
| Sep 1609:48 UTC2 weeks ago | Support Portal Service Disruption | minor | 11h 21m | |
| Sep 1608:31 UTC2 weeks ago | Support Helpdesk Availability Issues | minor | 6h 27m | |
| Sep 1100:33 UTC3 weeks ago | Change to Outbound IP for Dev Platform | major | 51m | |
| Sep 417:03 UTC4 weeks ago | Degradation of Hosted Grafana in US Central Region | major | 3h 39m | |
| Sep 416:05 UTC4 weeks ago | Elevated Linux worker queue times | major | 15h 4m |