Who is down right now

20 down now · 248 this week

GitHub · Developer toolsApr 27, 2026, 16:48 UTC 6 months ago

Disruption with some GitHub services

MinorNot disclosedUpdated 43d ago
Apr 27, 16:48 UTCApr 27, 19:02 UTC
Duration
2h 14m
Impact
Minor
Root cause
Not disclosed
GitHub, 90 days
69 incidents
Affected
Not listed by the vendor.

Lesson: Engineering teams must document root cause investigations completely to understand broad service disruptions.

What happened

On April 22, 2026 from 18:49 to 19:32 UTC , the Copilot Cloud Agent service began failing during session execution for users running the Agent HQ Codex agent. Codex agent sessions failed to start for all entry points (issue assignment, @copilot comment mentions). 0.5% of total Copilot Cloud Agent jobs were impacted (~2,000 failed jobs). Copilot and other agent sessions were unaffected. This was caused by a model resolution mismatch in Codex agent sessions, resulting in an incompatible model being used at runtime. A mitigation was deployed to select a stable default model for Codex agent sessions. We are working to harden the underlying model-resolution path so it correctly scopes to the requesting agent's supported models to prevent similar failure mode in the future.

Timeline

  1. Resolved · Apr 27, 19:02 UTC
    On April 22, 2026 from 18:49 to 19:32 UTC , the Copilot Cloud Agent service began failing during session execution for users running the Agent HQ Codex agent. Codex agent sessions failed to start for all entry points (issue assignment, @copilot comment mentions). 0.5% of total Copilot Cloud Agent jobs were impacted (~2,000 failed jobs). Copilot and other agent sessions were unaffected. This was caused by a model resolution mismatch in Codex agent sessions, resulting in an incompatible model being used at runtime. A mitigation was deployed to select a stable default model for Codex agent sessions. We are working to harden the underlying model-resolution path so it correctly scopes to the requesting agent's supported models to prevent similar failure mode in the future.

More from GitHub

Full history