Duplicate Notifications

Final update

Between Dec 2, 2025 16:20 UTC and Dec 3, 2025 14:27 UTC, a change deployed to our workflow execution system caused duplicate notifications to be sent to some customers and triggered unexpected auto-reruns for a small number of projects. Impact: - Some customers received multiple failure notification emails for the same workflow. - The number of duplicate notifications varied based on how many jobs in the workflow were affected (e.g., marked skipped or cancelled). - A subset of customers may have received duplicate Slack notifications. - A small number of projects experienced duplicated auto-reruns. The issue has been fully mitigated, and notification behavior has returned to normal. We appreciate your patience while our team resolved this issue.

1. Resolved · Dec 2, 02:00 UTC
   Between Dec 2, 2025 16:20 UTC and Dec 3, 2025 14:27 UTC, a change deployed to our workflow execution system caused duplicate notifications to be sent to some customers and triggered unexpected auto-reruns for a small number of projects. Impact: - Some customers received multiple failure notification emails for the same workflow. - The number of duplicate notifications varied based on how many jobs in the workflow were affected (e.g., marked skipped or cancelled). - A subset of customers may have received duplicate Slack notifications. - A small number of projects experienced duplicated auto-reruns. The issue has been fully mitigated, and notification behavior has returned to normal. We appreciate your patience while our team resolved this issue.