Axiom

Failure mode: Orchestration, control plane, service discovery or load balancing failed. Root cause: A configuration change. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

1. investigating 2026-10-21 11:35 UTC

   We are investigating elevated query error rates and increased latency affecting some customers. Our engineering team is actively working to identify the cause and restore normal operation. We will provide further updates as more information becomes available.

2. resolved 2026-10-21 12:08 UTC

   A portion of queries experienced increased latency and elevated error rates which was caused by an interaction between our service mesh and internal cache services. Following a deployment, some query workers continued attempting to connect to outdated cache instance addresses,...

Excerpts are shortened. Read the full incident on Axiom's status page.

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

1. identified 2026-10-15 12:21 UTC

   We are experiencing an issue affecting Metrics (MPL) queries in the US region. Customers may encounter failed queries or be unable to retrieve metric data through the Axiom Console and API. We have identified the cause and are deploying a fix to restore normal operation.

2. resolved 2026-10-15 13:15 UTC

   We've resolved the issue, we will improve the operational process to prevent the issue

Excerpts are shortened. Read the full incident on Axiom's status page.

Failure mode: Orchestration, control plane, service discovery or load balancing failed. Root cause: A configuration change. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

1. identified 2026-10-15 10:55 UTC

   We are investigating an issue affecting Metrics ingestion (using the `/v1/metrics` endpoint) and querying following an infrastructure change. Affected customers may experience failed or delayed metric ingestion, and recently ingested metrics may not appear in query results. We...

2. resolved 2026-10-15 11:07 UTC

   We've fixed the problem and Metrics ingest is back to normal.

3. resolved 2026-10-15 11:54 UTC

   The query service was also affected during the incident, it failed to discover recent metric data during this time window, and this is fixed now

Excerpts are shortened. Read the full incident on Axiom's status page.

Failure mode: A code release introduced the failure. Root cause: A bug in the vendor's own code. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

1. investigating 2026-10-14 19:12 UTC

   We've observed elevated error rates and slower responses for some queries in the US region. You may see failed or partially loaded dashboards, query errors (HTTP 500), or failed queries from the API and monitors. Ingest is not affected, and no data has been lost. Failed queries...

2. monitoring 2026-10-14 19:30 UTC

   We've rolled back the change behind the US query errors. Since 19:15 UTC, query error rates have been back to normal. We're monitoring and will post another update by 20:00 UTC.

3. resolved 2026-10-14 20:03 UTC

   Query performance in the US region has been fully restored since 19:15 UTC. Between 18:05 and 19:15 UTC, some queries and dashboards returned errors and some monitor runs were missed or delayed. Ingest was not affected and no data was lost. We'll share a follow-up describing...

Excerpts are shortened. Read the full incident on Axiom's status page.

Failure mode: Exhausted memory, disk, CPU, queues, connection pools or quotas. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

1. monitoring 2026-10-07 17:28 UTC

   A backlog in monitor executions in the EU region is causing delays in monitor evaluation. A fix has been deployed for the issue, and the backlog is decreasing in size. We are monitoring the backlog progress and will update once it has cleared

2. resolved 2026-10-07 18:36 UTC

   The backlog has been fully drained and monitor processing latency is back within normal range.

Excerpts are shortened. Read the full incident on Axiom's status page.

Failure mode: Login, tokens, certificates or credentials failed. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

1. investigating 2026-08-10 17:08 UTC

   Access to the Axiom Console is only available through Email, Google, and SAML/SSO authentication at present. Authentication with Github is unavailable. There is no impact on Axiom data or other services.

2. identified 2026-08-11 06:35 UTC

   Access to Axiom Console through Github is unavailable. Other authentication methods are not affected. Resolution is expected soon. All Axiom data and other services are unaffected.

3. identified 2026-08-11 12:13 UTC

   Access to the Axiom Console is only available through Email, Google, and SAML/SSO authentication at present. Authentication with Github is unavailable. There is no impact on Axiom data or other services.

4. monitoring 2026-08-11 17:13 UTC

   Access to the Axiom Console through Github is available as well as all other authentication methods. We are monitoring for any further disruption. Users might experience a one-time email verification step for their Primary Github email. The Primary email address should be...

5. resolved 2026-08-12 20:08 UTC

   Access to the Axiom Console through Github is available as well as all other authentication methods. Users might experience a one-time email verification step for their Primary Github email. The Primary email address should be configured as the email used with an Axiom account....

Excerpts are shortened. Read the full incident on Axiom's status page.

Failure mode: A code release introduced the failure. Root cause: A bug in the vendor's own code. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

1. investigating 2026-07-15 10:17 UTC

   Some queries are failing due to recent deployment, we are on it.

2. resolved 2026-07-15 10:26 UTC

   A revert has happened, the query service is now operating normally. The root cause has also been identified and we are actively working on it so queries are not affected again.

Excerpts are shortened. Read the full incident on Axiom's status page.

Failure mode: CDN or edge layer failed or served bad content. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

1. investigating 2026-06-30 18:30 UTC

   We are investigating reports of 403 errors when using Firehose (`/v1/ingest/.../firehose`) with our US edge domain. As a workaround, impacted customers can use our legacy ingest endpoint: `https://api.axiom.co/v1/datasets/.../ingest/firehose`.

2. resolved 2026-06-30 21:39 UTC

   We have rolled back the changes we have made in our infrastructure to fix the firehose ingestion. Firehose ingestion should be back now.

Excerpts are shortened. Read the full incident on Axiom's status page.

Failure mode: Orchestration, control plane, service discovery or load balancing failed. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

1. identified 2026-06-19 10:24 UTC

   We are experiencing some infrastructure issues on our EU Edge Deployment which may effect query performance for Metrics.

2. monitoring 2026-06-19 11:50 UTC

   We have rolled out a fix and currently monitoring to ensure query performance is optimal.

3. resolved 2026-06-19 14:45 UTC

   Underlying infrastructure issues have been resolved, metrics queries on EU Edge Deployment should now be fully operational.

Excerpts are shortened. Read the full incident on Axiom's status page.

Failure mode: Data delayed, stale, lost or corrupted, or needing replay after recovery. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

1. investigating 2026-05-15 17:05 UTC

   Team have noticed increased error rates in both ingest and queries. We started investigating...

2. identified 2026-05-15 17:15 UTC

   We have identified a problem with one of our internal databases. Mitigations are in progress.

3. monitoring 2026-05-15 17:24 UTC

   We see return to nominal error levels.

4. resolved 2026-05-15 21:27 UTC

   This incident is fully resolved.

Excerpts are shortened. Read the full incident on Axiom's status page.

Failure mode: A provider or third-party service this vendor depends on failed. Root cause: A failure at a provider or vendor they depend on. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

1. investigating 2026-05-08 00:03 UTC

   We are investigating increase in query errors

2. identified 2026-05-08 00:33 UTC

   We have identified a problem in our cluster related to an ongoing incident at our infrastructure provider and are mitigating it.

3. resolved 2026-05-08 00:50 UTC

   We see full recovery of the query subsystem. For a small number of metrics datasets, there might be data missing between 00:15 and 01:24 UTC. We're working on recovery with our infrastructure provider.

Excerpts are shortened. Read the full incident on Axiom's status page.

Failure mode: A code release introduced the failure. Root cause: A bug in the vendor's own code. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

1. investigating 2026-04-28 13:50 UTC

   We have noticed an increase to the Error Rate on the ingest path, we are working on it

2. monitoring 2026-04-28 14:21 UTC

   We have identified the root cause of the issue, which was introduced in the most recent deployment, and have reverted the changes. A permanent fix is currently being developed and will be deployed to production shortly.

3. resolved 2026-04-28 15:09 UTC

   All systems are operating normally. The issue has been fully resolved and we are continuing to monitor to ensure stability.

Excerpts are shortened. Read the full incident on Axiom's status page.

Failure mode: Data delayed, stale, lost or corrupted, or needing replay after recovery. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

1. identified 2026-04-17 15:00 UTC

   Dataset trimming operations in the US region are currently experiencing delays. This may cause temporary increases in dataset size and delays in enforcing data retention policies across affected datasets. The root cause has been identified, and remediation is in progress. Due to...

2. resolved 2026-04-18 05:12 UTC

   This incident is now resolved.

Excerpts are shortened. Read the full incident on Axiom's status page.

Failure mode: A configuration change caused the failure. Root cause: A configuration change. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

1. resolved 2026-03-30 09:40 UTC

   Between March 30 and April 13, a configuration issue prevented email notifications from firing for monitors on our Legacy EU infrastructure (not EU edge deployment). Monitor evaluation was functioning correctly during this period — monitors were being evaluated and other...

Excerpts are shortened. Read the full incident on Axiom's status page.

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

1. investigating 2026-04-12 11:05 UTC

   We are experiencing a partial query outage on our EU Edge Deployment.

2. identified 2026-04-12 12:44 UTC

   We've identified the cause of the disruption, we're evaluating possible mitigations.

3. monitoring 2026-04-12 14:29 UTC

   We've applied mitigations and are observing full recovery of the query service.

4. resolved 2026-04-12 15:13 UTC

   This incident is resolved

Excerpts are shortened. Read the full incident on Axiom's status page.

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

1. investigating 2026-03-06 01:29 UTC

   We are investigating increased ingest error rate

2. monitoring 2026-03-06 01:46 UTC

   We've identified the source of the problem and applied mitigations, we observe decrease in ingest errors.

3. resolved 2026-03-06 02:02 UTC

   This incident is resolved

Excerpts are shortened. Read the full incident on Axiom's status page.

Failure mode: A configuration change caused the failure. Root cause: A human mistake during operations. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

1. resolved 2026-02-26 15:00 UTC

   Just before 3pm on 26th of Feb '26, an incorrect configuration was accidentally deployed to our EU environment which impacted query performance. We quickly identified the issue and released a fix.

Excerpts are shortened. Read the full incident on Axiom's status page.

Failure mode: A configuration change caused the failure. Root cause: A configuration change. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

1. investigating 2026-02-26 15:30 UTC

   We are investigating reports of widespread query failures on the legacy EU instance. Users experiencing internal server errors when executing queries.

2. identified 2026-02-26 15:40 UTC

   We've identified the root cause as a network configuration change that disrupted internal service-to-service communication. This caused query requests to fail, resulting in HTTP 500 errors.

3. resolved 2026-02-26 15:55 UTC

   All services have returned to normal operation. Queries are executing successfully, and internal traffic between services has been fully restored. We are continuing to monitor the system to ensure stability.

Excerpts are shortened. Read the full incident on Axiom's status page.

Failure mode: A configuration change caused the failure. Root cause: A configuration change. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

1. investigating 2026-01-27 17:08 UTC

   We are experiencing increased error rate for queries.

2. monitoring 2026-01-27 17:44 UTC

   We identified a misconfiguration in monitor processing, which has now been mitigated. Query performance might be degraded while queues are processed.

3. monitoring 2026-01-27 19:10 UTC

   All queues have finished processing, query processing is nominal.

4. resolved 2026-01-27 19:40 UTC

   This incident has been resolved.

Excerpts are shortened. Read the full incident on Axiom's status page.

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

1. monitoring 2026-01-17 01:06 UTC

   We have encountered issues with one of our ingestion pipelines, a fix has been rolled out and we are currently monitoring the situation.

2. monitoring 2026-01-17 11:43 UTC

   We are still monitoring the system, updates: • Error rates are decreasing • System stability is improving • We are actively monitoring as the backlog clears We will continue to provide updates as the situation progresses. We apologize for any disruption this may have caused.

3. resolved 2026-01-17 13:01 UTC

   The incident has been fully resolved and all systems are now stable. Services are operating normally and no further action is required. We will continue monitoring to ensure stability.

4. monitoring 2026-01-17 14:21 UTC

   We're monitoring increase in error rates.

5. resolved 2026-01-17 16:55 UTC

   All services are operating nominally.

Excerpts are shortened. Read the full incident on Axiom's status page.

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

1. investigating 2026-01-13 22:08 UTC

   We are experiencing issues with our query runners which may cause some queries to timeout or fail to execute.

2. resolved 2026-01-13 22:31 UTC

   We have identified and solved the issue at hand. Optimal query performance should not be restored.

Excerpts are shortened. Read the full incident on Axiom's status page.

Failure mode: Traffic surge or denial-of-service attack. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

1. investigating 2026-01-09 20:57 UTC

   We're currently investigating increase in ingest error rates.

2. identified 2026-01-09 21:45 UTC

   We have identified a specific high-volume traffic source causing instability within our ingestion pipeline and have mitigated the issue to ensure continued stability for all other users.

3. resolved 2026-01-09 23:41 UTC

   All error rates are back to nominal levels.

Excerpts are shortened. Read the full incident on Axiom's status page.

Failure mode: A code release introduced the failure. Root cause: A bug in the vendor's own code. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

1. investigating 2025-12-15 14:51 UTC

   We're investigating increase in query error rates

2. identified 2025-12-15 15:00 UTC

   We have identified a problem with latest deploy which is being rolled back.

3. resolved 2025-12-15 15:01 UTC

   This incident is now fully resolved.

Excerpts are shortened. Read the full incident on Axiom's status page.

Failure mode: Login, tokens, certificates or credentials failed. Root cause: A failure at a provider or vendor they depend on. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

1. resolved 2025-12-05 14:45 UTC

   Access to Axiom Console was temporary disrupted. Users were unable to access Axiom and were shown a Vercel sign-in screen.

Excerpts are shortened. Read the full incident on Axiom's status page.

Failure mode: Exhausted memory, disk, CPU, queues, connection pools or quotas. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

1. investigating 2025-11-02 20:00 UTC

   We are investigating failures to monitor processing in EU region

2. resolved 2025-11-02 20:10 UTC

   We have identified and resolved issues with monitor processing in the EU region Some of our EU customers could have experienced monitor processing issues. During this period, gaps in alerting were a possibility due to instability issues caused by underlying scaling issues....

Excerpts are shortened. Read the full incident on Axiom's status page.