Increased app-not-found errors

Final update

This incident has been resolved.

1. Resolved · Aug 9, 07:10 UTC
   This incident has been resolved.
2. Monitoring · Aug 9, 06:42 UTC
   A fix has been implemented and we are monitoring the results
3. Identified · Aug 9, 06:03 UTC
   We’ve deployed an additional mitigation to further reduce Corrosion retry pressure and are seeing improvement; we’re continuing to monitor while remaining affected nodes catch up.
4. Identified · Aug 9, 04:58 UTC
   We’ve applied a mitigation to reduce the impact from Corrosion batch insertion retries and are continuing to monitor while affected nodes catch up.
5. Identified · Aug 9, 03:50 UTC
   We have identified the issue as failed insertions in a subset of Corrosion batches. These failures trigger retries, which can cause timeouts for other batches.
6. Investigating · Aug 9, 02:50 UTC
   We are currently investigating app-not-found errors returned by Machines API calls made shortly after creating new applications.