Amp

Failure mode: Exhausted memory, disk, CPU, queues, connection pools or quotas. Root cause: Not enough capacity for the load. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

1. investigating 2026-10-21 16:38 UTC

   Inference requests are failing intermittently. We're investigating.

2. resolved 2026-10-21 17:06 UTC

   One our storage nodes saturated and was not replaced fast enough to keep up with ongoing traffic. We have identified the issue and mitigated it. We are working on a fix to prevent this in the future.

Excerpts are shortened. Read the full incident on Amp's status page.

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

1. investigating 2026-10-21 12:07 UTC

   We're investigating an issue creating new threads or processing thread messages

2. identified 2026-10-21 12:13 UTC

   We've identified the issue and are deploying a fix. Creating new threads and sending thread messages are returning errors.

3. resolved 2026-10-21 12:29 UTC

   Creating threads and running inference is now working as expected

Excerpts are shortened. Read the full incident on Amp's status page.

Failure mode: Traffic surge or denial-of-service attack. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

1. investigating 2026-10-06 15:55 UTC

   We're seeing some degraded performance on [ampcode.com](http://ampcode.com "ampcode.com"). We are seeing intermittent timeouts when loading pages and threads.

2. resolved 2026-10-06 20:02 UTC

   We've resolved the increased load, and latencies are back down to normal levels. Apologies if you saw some slowdowns!

Excerpts are shortened. Read the full incident on Amp's status page.

Failure mode: A code release introduced the failure. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

1. investigating 2026-10-01 03:27 UTC

   We're seeing some agents fail to start or reconnect for some threads. Thread data is still loading. Only a subset of threads are affected. We are seeing high CPU usage on some nodes during a deploy that we believe to be causing this.

2. monitoring 2026-10-01 03:49 UTC

   Some CPU metrics remain above the ideal range, but we are seeing a broad recovery. The impact of this incident was limited to a small number of threads and caused agent unavailability of up to a few minutes. We are still monitoring the metrics.

3. resolved 2026-10-01 04:00 UTC

   This issue is resolved.

Excerpts are shortened. Read the full incident on Amp's status page.

Failure mode: A provider or third-party service this vendor depends on failed. Root cause: A failure at a provider or vendor they depend on. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

Names an upstream provider: OpenAI.

1. identified 2026-09-25 23:00 UTC

   Using OpenAI models through a ChatGPT subscription is failing with websocket errors. This appears to be an upstream issue. We will update once it's confirmed by OpenAI

2. identified 2026-09-25 23:04 UTC

   Small update: it appears that API traffic is unaffected, so if you disable your ChatGPT sub, you can fall back to using Amp credits to continue your orbin'

3. identified 2026-09-25 23:07 UTC

   OpenAI has updated their status page now, so you can find the most up-to-date info there: [https://status.openai.com/incidents/01M3DCNWMW57HYK8FJ5FBFPA39](https://status.openai.com/incidents/01M3DCNWMW57HYK8FJ5FBFPA39...

4. resolved 2026-09-26 00:26 UTC

   OpenAI has resolved the Codex outage

Excerpts are shortened. Read the full incident on Amp's status page.

Failure mode: A code release introduced the failure. Root cause: A bug in the vendor's own code. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

1. investigating 2026-09-24 07:59 UTC

   We are seeing a higher failure rate for orbs when starting up. Existing orbs appear to be unaffected.

2. monitoring 2026-09-24 08:04 UTC

   We have identified the problem and are rolling back the problematic deployment that is preventing new orbs from being created.

3. monitoring 2026-09-24 08:38 UTC

   During the rollback, some other non-orb threads started to experience higher error rates due to high load in the rollback.

4. resolved 2026-09-24 08:44 UTC

   Orbs and threads are working again.

Excerpts are shortened. Read the full incident on Amp's status page.

Failure mode: A provider or third-party service this vendor depends on failed. Root cause: A failure at a provider or vendor they depend on. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

1. identified 2026-09-18 22:54 UTC

   Our authentication provider (WorkOS) is down which is preventing logged in sessions from connecting.

2. monitoring 2026-09-18 22:59 UTC

   Authentication is working again, we are continuing to monitor availability

3. monitoring 2026-09-18 23:09 UTC

   Authentication is working again, we are continuing to monitor availability

4. resolved 2026-09-18 23:12 UTC

   Our authentication provider has recovered service.

Excerpts are shortened. Read the full incident on Amp's status page.

Failure mode: A code release introduced the failure. Root cause: A bug in the vendor's own code. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

1. investigating 2026-09-09 21:37 UTC

   During a deployment, there is an issue where some threads are temporarily unable to be interacted with.

2. identified 2026-09-09 21:51 UTC

   Many threads are unavailable on the web. The Amp CLI is still working.

3. monitoring 2026-09-09 21:56 UTC

   We have rolled back the problematic web change, and threads on the web are loading again.

4. resolved 2026-09-09 22:20 UTC

   We've confirmed the issue is resolved and has been for the last 25 minutes.

Excerpts are shortened. Read the full incident on Amp's status page.

Failure mode: A configuration change caused the failure. Root cause: A configuration change. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

1. investigating 2026-09-08 11:18 UTC

   We're seeing elevated error rates and delays when creating and connecting to threads.

2. monitoring 2026-09-08 11:30 UTC

   Thread error rates are back to normal, and we'll continue to monitor.

3. monitoring 2026-09-08 11:41 UTC

   **Prometheus autoscaling caused database pod eviction** A Google managed Prometheus collector autoscaler increased the collectors’ resource requests and restarted collectors across the cluster. The collector pods had a wrongly configured too-high priority, higher than our...

4. resolved 2026-09-08 12:01 UTC

   Database recovered, limits adjusted. What we're going to do: • Audit all of our resource priority classes • Introduce more spare capacity for storage nodes so that we don't have to wait for new nodes to come online • Revise alerts so it's faster than 5min for us to pinpoint...

5. investigating 2026-09-08 12:05 UTC

   Some threads fail to connect sporadically

6. monitoring 2026-09-08 12:12 UTC

   Updating the priority classes for our storage nodes lead to a re-deployment at the same time as another deployment was going on. We're disabling auto-deployments now while incidents are going on.

7. resolved 2026-09-08 12:33 UTC

   Fully recovered, monitoring, and rolling out fixes for the problems we found

8. investigating 2026-09-08 14:31 UTC

   We are experiencing database issues again and are investigating.

9. resolved 2026-09-08 14:55 UTC

   This morning's incident left 369 user connections to threads in a degraded state. They are now restored and working.

Excerpts are shortened. Read the full incident on Amp's status page.

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

1. investigating 2026-09-06 16:22 UTC

   We're seeing elevated error rates and delays when creating and connecting to threads.

2. monitoring 2026-09-06 19:48 UTC

   We've found the issue and failures are down to zero right now, but we are still working on a more permanent fix. For the last few hours roughly 5-10% of thread creations and reconnects were failing.

3. monitoring 2026-09-06 20:16 UTC

   The mitigation is working and errors/delays are still at zero. We are continuing to monitor it and will deploy the longer-term fix soon.

4. resolved 2026-09-06 22:32 UTC

   We have been at zero errors/failures for 3 hours now. We will continue to monitor it and will be shipping longer-term improvements over the next few hours to prevent this from happening again.

Excerpts are shortened. Read the full incident on Amp's status page.

Failure mode: A code release introduced the failure. Root cause: A bug in the vendor's own code. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

1. investigating 2026-09-05 17:24 UTC

   We're seeing general performance degradation across most thread operations.

2. resolved 2026-09-05 17:55 UTC

   A confluence of a few different deploys caused our load balancers started misbehaving and overloading a few of our servers. We've adjusted the balancing so if you saw threads failing, they should all be healthy now.

Excerpts are shortened. Read the full incident on Amp's status page.

Failure mode: A code release introduced the failure. Root cause: A bug in the vendor's own code. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

1. investigating 2026-09-04 20:44 UTC

   We're seeing elevated error rates when creating and resuming threads. Will update as soon as we have more information.

2. resolved 2026-09-04 21:05 UTC

   We had a deployment out with a bug that caused our server to crash in some situations, causing intermittent errors while using amp. We've rolled back and the errors have dropped to zero. Sorry for the noise!

Excerpts are shortened. Read the full incident on Amp's status page.

Failure mode: A provider or third-party service this vendor depends on failed. Root cause: A failure at a provider or vendor they depend on. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

Names an upstream provider: OpenAI.

1. identified 2026-09-03 15:02 UTC

   Using ChatGPT subscriptions in Amp is failing due to an upstream incident. Follow for updates

2. resolved 2026-09-03 16:58 UTC

   OpenAI has resolved their incident, and ChatGPT subs appear to be working well again.

Excerpts are shortened. Read the full incident on Amp's status page.

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

1. investigating 2026-09-03 01:35 UTC

   Orbs are seeing a higher error rate when starting.

2. monitoring 2026-09-03 02:17 UTC

   Orb start and resume rates have returned to normal, and we're continuing to monitor.

3. resolved 2026-09-03 02:34 UTC

   Orb creation rates have fully stabilised, and all systems are normal. We're sorry for the interruption, and we'll continue to investigate ways to make Amp more resilient to this going forward.

Excerpts are shortened. Read the full incident on Amp's status page.

Failure mode: A provider or third-party service this vendor depends on failed. Root cause: A failure at a provider or vendor they depend on. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

Names an upstream provider: Google Cloud.

Started during Google Cloud: Multiple products in us-central1-b are experiencing network service degradation..

1. investigating 2026-09-01 14:49 UTC

   We're seeing elevated error rates when creating to new threads. We're looking into it

2. investigating 2026-09-01 15:26 UTC

   Google Cloud reports connectivity issues between VMs, which results in us not being able to scale resources up/down as necessary.

3. investigating 2026-09-01 16:13 UTC

   Google Cloud has now officially opened an incident: [https://status.cloud.google.com/incidents/J5ia5t9p3g9Q5Wi7r8Ev](https://status.cloud.google.com/incidents/J5ia5t9p3g9Q5Wi7r8Ev "https://status.cloud.google.com/incidents/J5ia5t9p3g9Q5Wi7r8Ev") We're in the process of...

4. identified 2026-09-01 17:12 UTC

   The issues we observed related to the GCP outage appear to be resolved, but our services are still recovering. We're currently working on restoring full data replication, which we estimate will complete around 11am PT, at which point service should be fully restored.

5. monitoring 2026-09-01 18:05 UTC

   Availability is restored. We're adding back capacity and monitoring.

6. resolved 2026-09-01 18:27 UTC

   Service is fully restored, and all our systems have stabilized. We know outages like this hurt, and we have work in progress that will mean we're more resilient when faced with infrastructure instability like this. Unfortunately, this incident hit before that work was complete....

Excerpts are shortened. Read the full incident on Amp's status page.

Failure mode: A code release introduced the failure. Root cause: A bug in the vendor's own code. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

1. investigating 2026-08-26 20:37 UTC

   [ampcode.com](http://ampcode.com "ampcode.com") and the native app are returning 500 errors for some users. Still investigating, but will update soon

2. resolved 2026-08-26 20:54 UTC

   The browser application was crashing with a 500 error after page load. We've rolled back the change and a hard refresh should fix any remaining issues.

Excerpts are shortened. Read the full incident on Amp's status page.

Failure mode: A code release introduced the failure. Root cause: A bug in the vendor's own code. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

1. investigating 2026-08-25 01:05 UTC

   The Amp application and API are unreachable.

2. identified 2026-08-25 01:15 UTC

   We have found the cause and are rolling back to a good version. Amp is partially available again.

3. resolved 2026-08-25 01:22 UTC

   We found the source of the issue (a very inefficient PostgreSQL query) and have rolled back and have deployed a fix for future deploys.

Excerpts are shortened. Read the full incident on Amp's status page.

Failure mode: A provider or third-party service this vendor depends on failed. Root cause: A failure at a provider or vendor they depend on. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

Names an upstream provider: OpenAI.

1. investigating 2026-08-20 17:04 UTC

   We are seeing elevated error rates from OpenAI for requests using OpenAI-backed models (e.g. GPT-5.x). Affected requests fail with "Invalid prompt: your prompt was flagged as potentially violating our usage policy", even for normal coding prompts. This is an upstream issue with...

2. monitoring 2026-08-20 17:16 UTC

   Error rates from OpenAI have started recovering and requests are succeeding again. We are monitoring error rates to confirm full recovery before resolving this incident.

3. resolved 2026-08-20 17:35 UTC

   OpenAI requests are completing successfully again. We're still working with OpenAI to identify the underlying cause of the prompt-flagging errors.

Excerpts are shortened. Read the full incident on Amp's status page.

Failure mode: A code release introduced the failure. Root cause: A configuration change. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

1. identified 2026-08-20 12:46 UTC

   A recent change to the Orb base image lead to Orbs not starting up for fresh instances. We're rolling it back.

2. identified 2026-08-20 13:13 UTC

   We've identified the issue, rolled out a fix, and are now restarting all Orbs that previously failed to start.

3. monitoring 2026-08-20 13:41 UTC

   Restarting failed orbs and watching them restart.

4. resolved 2026-08-20 13:51 UTC

   Orbs that were created in a ~1hr window failed to start up. We restarted them. No data was lost. After a fix rolled out, new orbs would start up without problems.

Excerpts are shortened. Read the full incident on Amp's status page.

Failure mode: Exhausted memory, disk, CPU, queues, connection pools or quotas. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

1. investigating 2026-08-16 10:14 UTC

   [ampcode.com](http://ampcode.com "ampcode.com"), the CLI, and the API are all unreachable.

2. investigating 2026-08-16 10:18 UTC

   Creating and using threads is working again but is very slow. The API is available. We are investigating.

3. investigating 2026-08-16 10:24 UTC

   We are seeing the error rate increase even from the Amp CLI now.

4. resolved 2026-08-16 10:30 UTC

   We found the source of the high load on our database and have resolved the issue.

Excerpts are shortened. Read the full incident on Amp's status page.

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

1. investigating 2026-08-16 04:00 UTC

   Thread creation is taking up to 60 seconds and archiving and other operations are slow.

2. monitoring 2026-08-16 05:18 UTC

   We have rolled out a fix and error rates are declining rapidly.

3. resolved 2026-08-16 05:36 UTC

   This issue has been resolved.

Excerpts are shortened. Read the full incident on Amp's status page.

Failure mode: A code release introduced the failure. Root cause: A bug in the vendor's own code. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

1. identified 2026-08-14 16:06 UTC

   A deploy went out with a poorly shaped query plan for listing threads, which is adding large amounts of load to the DB and slowing down many operations. We've rolled back that deploy and should be back soon.

2. resolved 2026-08-14 16:13 UTC

   We've rolled back the bad change and everything is healthy again. Sorry for the blip.

Excerpts are shortened. Read the full incident on Amp's status page.

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

1. investigating 2026-08-13 19:07 UTC

   Executors are failing to start for new orbs

2. monitoring 2026-08-13 20:12 UTC

   The underlying issue has ben resolved and new orbs are starting successfully again. Some existing orbs affected by this incident are still stuck, and we are working on resolution for these.

3. monitoring 2026-08-13 21:05 UTC

   New and existing orbs are in a healthy state. There is a small set of existing threads still in an unhealthy state.

4. resolved 2026-08-13 21:54 UTC

   All new orbs are operational.

Excerpts are shortened. Read the full incident on Amp's status page.

Failure mode: Data delayed, stale, lost or corrupted, or needing replay after recovery. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

1. investigating 2026-08-11 12:24 UTC

   A fraction of Amp runners are not able to connect to our servers. We are investigating the issue.

2. monitoring 2026-08-11 13:49 UTC

   We've run our repair for all impacted users, so creating threads on runners is now working for everyone. There are still a few threads from the incident that are left in bad state, so we are shifting our focus to repairing those.

3. resolved 2026-08-11 14:28 UTC

   We've repaired all the threads which were unable to connect because of this incident, so all known issues have been resolved. Thanks for your patience!

Excerpts are shortened. Read the full incident on Amp's status page.

Failure mode: A provider or third-party service this vendor depends on failed. Root cause: A failure at a provider or vendor they depend on. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

1. investigating 2026-08-10 23:48 UTC

   Thread creation and connection is failing

2. monitoring 2026-08-11 00:05 UTC

   We ran into some GCE capacity issues which meant that a routine node update was suddenly not routine. Thread creation is working again, but you may still notice some slowness as we fully recover.

3. resolved 2026-08-11 02:36 UTC

   Service has stabilized for new threads. We are still investigating some reports of threads which experienced errors during the incident staying stuck.

Excerpts are shortened. Read the full incident on Amp's status page.

Failure mode: A provider or third-party service this vendor depends on failed. Root cause: A failure at a provider or vendor they depend on. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

1. investigating 2026-08-07 06:17 UTC

   We are currently investigating an outage in our infrastructure, which makes thread creation slow or causes time out.

2. monitoring 2026-08-07 09:24 UTC

   We have identified the cause. A bug in undici, the HTTP library bundled with Node.js, prevented new outgoing requests from being dispatched while a long-lived streaming request was open on the same HTTP/2 connection. Under high load, new requests queued behind those streams and...

3. resolved 2026-08-07 09:38 UTC

   A bug in undici, the HTTP library bundled with Node.js, prevented new outgoing requests from being dispatched while a long-lived streaming request was open on the same HTTP/2 connection. Under high load, new requests queued behind those streams and timed out. We deployed Node.js...

Excerpts are shortened. Read the full incident on Amp's status page.