">
sevanalysis

Iterable

Apps and SaaS. Source: Iterable status page.

Incidents, last 90 days
4
Median to mitigation
35m
Median to resolution
1h 40m
p95 to resolution
2h 40m
Naming an upstream provider
0%
With a postmortem on the status page
0%

What failed, last 90 days

The text does not say enough to tell2
A provider or third-party service this vendor depends on failed2

Tagged by claude-haiku-5-5 from each incident's updates. See the methodology.

Incidents per month

All recorded incidents

Timeouts on User Update API Endpoints minor 2026-09-22 19:02 UTC 56m

Affected: Global API Success

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    We are currently investigating an issue leading to slowness and timeouts across all API user update endpoints beginning at 2:45 PM EDT. Next update at 3:15 PM EDT

  2. monitoring

    We are continuing to investigate the issue but are seeing positive signs of recovery. API endpoints for user updates are back to normal success rates. Next update at 3:45 PM EDT or sooner.

  3. resolved

    All API issues are now resolved and the user update endpoints are fully operational. If you continue to experience any issues, please contact Support.

Excerpts are shortened. Read the full incident on Iterable's status page.

EU platform slowness and gateway errors minor 2026-09-10 14:53 UTC 2h 23m

Affected: Global Web Application, Global API Success, Global Journey Processing

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. investigating

    We're currently investigating an issue for EU customers with platform slowness, intermittent 502/503 Gateway errors, and Journey changes failing to save

  2. monitoring

    The intermittent platform issues have been mitigated and there is currently no ongoing customer impact. The observed customer impact window was approximately 9:25 a.m.–11:13 a.m. EDT. We will continue to monitor the issue.

  3. resolved

    We completed our monitoring and there are no further issues. The observed customer impact window was approximately 9:25 a.m.–11:13 a.m. EDT.

Excerpts are shortened. Read the full incident on Iterable's status page.

Subset of customers using Twilio for SMS seeing a portion of messages getting droppped minor 2026-09-01 17:19 UTC 58m

Affected: Global Campaign Sends

Failure mode: A provider or third-party service this vendor depends on failed. Root cause: A failure at a provider or vendor they depend on. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

  1. identified

    A subset of customers using Twilio for SMS sending may have some messages getting dropped due to an incident on Twilio’s side with impact currently ranging from 5%-20%. We are actively monitoring the impact and status updates from Twilio. There is not impact to customers using...

  2. resolved

    Twilio has reported that the issue causing 5xx errors on their side has resolved and we have seen SMS sends return to normal. Customers using Twilio that saw sends get dropped should re-send those messages if they want to. No further updates will be posted.

Excerpts are shortened. Read the full incident on Iterable's status page.

Email Send Delays with Partial Sparkpost outage on US-WEST-2. minor 2026-07-24 16:30 UTC 2h 40m

Affected: Global Campaign Sends

Failure mode: A provider or third-party service this vendor depends on failed. Root cause: A failure at a provider or vendor they depend on. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

  1. investigating

    Email sends delayed for US-hosted customers using SparkPost in US-WEST-2 region.

  2. monitoring

    We are currently experiencing email delivery delays due to a partial outage affecting SparkPost's US-West-2 region. Emails continue to be accepted and queued, but delivery is delayed. Our team is working closely with SparkPost to resolve this issue. We will provide updates as...

  3. monitoring

    We are continuing to monitor Sparkpost outage as they work to continue draining the backlog of delayed messages.

  4. monitoring

    We are continuing to monitor Sparkpost outage as they work to continue draining the backlog of delayed messages. Sparkpost estimates that they expect to have most email delivered within the next 30-45 minutes.

  5. resolved

    All Sparkpost sends that were previously delayed have been delivered and Sparkpost reports that their systems are now fully functional. If you continue to experience any issues please contact Support.

Excerpts are shortened. Read the full incident on Iterable's status page.

Iterable Application Issues in US Datacenter critical 2026-07-16 09:27 UTC 40m

Affected: Global Web Application, Global API Success, Global Links, Global System Webhooks, Global Partner Webhooks, Global Analytics Processing, Global Catalog, Global API Ingestion, Global Campaign Sends, Global Proof Sends, Global Journey Processing

Failure mode: A code release introduced the failure. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. investigating

    We are currently investigating an issue impacting the main Iterable application in the US region.

  2. identified

    Iterable engineering has identified the issue and is currently deployed a fix across the US environment. We are starting to see successful logins on the Iterable application.

  3. identified

    We are continuing to work on a fix for this issue.

  4. resolved

    The issues with the Iterable application and API are now resolved and all sends and journeys that may have been delayed during the impact window have now been processed. Please reach out to Support if you continue to experience any issues.

Excerpts are shortened. Read the full incident on Iterable's status page.

WhatsApp Messaging Delays and Failures none 2026-06-12 16:23 UTC 54m

Failure mode: A provider or third-party service this vendor depends on failed. Root cause: A failure at a provider or vendor they depend on. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

  1. identified

    We are aware of an issue affecting WhatsApp messaging. Some customers may experience delays or failures sending WhatsApp messages. This appears to be caused by an external service disruption, and we are monitoring for updates. We will share another update by 1pm PST or sooner if...

  2. resolved

    The issue affecting WhatsApp messaging has been resolved. Customers should no longer experience delays or failures sending WhatsApp messages. We will continue to monitor to ensure service remains stable.

Excerpts are shortened. Read the full incident on Iterable's status page.

Iterable Web Application Outage - US and EU Regions critical 2026-05-12 23:31 UTC 37m

Affected: Global Web Application

Failure mode: A code release introduced the failure. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    We are currently investigating an issue involving users being unable to access the Iterable web app. API traffic, Sends, and Journeys already in progress are unaffected. This outage is limited to the Iterable web application itself.

  2. investigating

    Engineering has identified a fix and is in the process of deploying to both EU and US web environments to resolve the issue. Next update at 5PM PDT or sooner.

  3. identified

    Our engineering team is continuing to work toward a full resolution. While we have mostly restored web application access, there may be some pages within the web app that periodically fail to load until the fix is fully rolled out across the entire application. Next update by...

  4. resolved

    Our engineering team has completed the fix to the web application in both the US and EU regions. All services are currently functioning normally. Email sends, API traffic, webhooks, and journeys functioned normally throughout this incident. The only impact was being unable to...

Excerpts are shortened. Read the full incident on Iterable's status page.

Journey pages are not responding none 2026-04-16 10:09 UTC 50m

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. investigating

    We’re currently investigating an issue where Journey pages are not responding. Engineering is actively applying a fix and will continue investigating the root cause. Next update at 4:00 PM PST / 11:00 PM GMT.

  2. monitoring

    Journey pages are loading normally following the fix. Engineering will continue monitoring.

  3. resolved

    This incident has been resolved.

Excerpts are shortened. Read the full incident on Iterable's status page.

Elevated API Error Rates in US Region minor 2026-04-14 18:02 UTC 2h 34m

Affected: Global API Success, Global API Ingestion

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    We are currently experiencing elevated error rates on several API endpoints in our US region, including Bulk Update, Register Device, and Target. We are actively working to resolve this issue and will provide another update within 30 minutes.

  2. monitoring

    We are seeing improved success rates on the Register Device and Target endpoints. Our engineering team is continuing to work on other endpoints including Bulk Update. Next update within 30 minutes.

  3. monitoring

    Our engineering team is continuing to work on improving success rates for all API endpoints including Bulk Update. Next update in 60 minutes.

  4. monitoring

    We are continuing to monitor for any further issues.

  5. resolved

    This incident has been resolved.

Excerpts are shortened. Read the full incident on Iterable's status page.

Ingestion and processing delay minor 2026-03-25 19:00 UTC 22m

Affected: Journey Processing, Journey Processing, Journey Processing, User Updates, User Updates, User Updates, List Uploads, List Uploads, List Updates

Failure mode: A code release introduced the failure. Root cause: A bug in the vendor's own code. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    Current status We have identified an issue in our ingestion service that can cause some updates to be delayed. Customer impact Some customers are seeing delays in list uploads and events appearing in Iterable, in some cases taking 30+ minutes to render. Clusters impacted...

  2. monitoring

    We have caught up with ingestion. Currently monitoring our systems.

  3. resolved

    This incident has been resolved.

Excerpts are shortened. Read the full incident on Iterable's status page.

Customers on c102 are experiencing degraded performance minor 2026-03-17 18:07 UTC 1h 14m

Affected: Email Sends, Journey Processing, Push Sends, SMS Sends, User Updates, List Updates, User Deletions

Failure mode: Orchestration, control plane, service discovery or load balancing failed. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. investigating

    We are currently investigating an issue for one of ES clusters - c102 - The cluster is in a red state, leading to very slow ingestion and widespread API 500 errors. Customers are experiencing failed searches and errors on multiple API endpoints. Ingestion rates are near zero,...

  2. investigating

    Engineering is continuing to investigate this issue. At this time there is no improvement yet to any customers on C102. Next Update at 12:15 PDT or sooner.

  3. monitoring

    Engineering has restored C102 back to a "green" state and ingestion and API calls are succeeding again for this cluster. We will continue to monitor until we are fully through the accumulated backlog. Next update at 1 PM PDT or sooner.

  4. resolved

    We have fully caught up with any backlog on C102 and all queries and activity through C102 are functioning normally. If you continue to experience any issues please contact Support.

Excerpts are shortened. Read the full incident on Iterable's status page.

Proof Send Functionality Degraded minor 2026-03-16 16:18 UTC 2h 32m

Affected: Global Proof Sends

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. investigating

    We have been alerted to an issue where proofs sends are not being sent as expected as of 9:00 am PT. Iterable Engineers are actively investigating the issue. Next update by 11:00 am PT or sooner.

  2. monitoring

    The engineering team has implemented a fix and proofs are now successfully sending. We will continue to monitor. Next update at 12:00 PST or earlier.

  3. resolved

    This incident has been resolved.

Excerpts are shortened. Read the full incident on Iterable's status page.

Email send delays for some US-hosted SparkPost traffic major 2026-02-24 19:51 UTC 1m

Affected: Global Campaign Sends

Failure mode: A provider or third-party service this vendor depends on failed. Root cause: A failure at a provider or vendor they depend on. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

  1. monitoring

    Email sends for US-hosted customers using SparkPost were delayed due to elevated 5xx errors and increased API latency from SparkPost’s Transmissions API. EU-hosted traffic was not impacted. All customers using ESPs other than Sparkpost were similarly not impacted. SparkPost has...

  2. resolved

    SparkPost’s API issue has been resolved, and we’re no longer seeing elevated 5xx errors or increased latency. Incident has resolved.

Excerpts are shortened. Read the full incident on Iterable's status page.

app.iterable.com was unavailable in the US region, returning 502 Bad Gateway errors to users. major 2026-02-24 18:18 UTC 11m

Affected: Global Web Application

Failure mode: A zone, region or data center became unavailable. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    We are currently investigating an issue where users could not access app.iterable.com and received 502 Bad Gateway errors. EU users were not affected.

  2. monitoring

    A fix has been implemented and we are monitoring the results.

  3. resolved

    Service has fully recovered and all traffic has been restored to app.iterable.com in the US region. The incident lasted from 9:50 am to 10:15 am Pacific Time, impacting both web and API access with elevated 502, 503, and 504 errors. EU region was unaffected.

Excerpts are shortened. Read the full incident on Iterable's status page.

Delayed sends from 7am to 9am ET (retro-active) none 2026-02-16 12:00 UTC 0m

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. resolved

    We experienced a period of delayed sends (including proofs) as well as errors in user updates from 7am to 9am ET. The issue has been resolved. Some delayed messages may still be processing and will be sent shortly. If customers received 500 responses to API requests, they should...

Excerpts are shortened. Read the full incident on Iterable's status page.

Iterable web elevated error rates minor 2026-01-08 20:17 UTC 1h 26m

Affected: Global Web Application

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. investigating

    Our engineering team is actively investigating an issue causing elevated error rates and intermittent page loads in the Iterable web app. A subset of users may see error messages or blank pages when attempting to log in or navigate the UI. At this time, API-based message sending...

  2. investigating

    We are continuing to investigate this issue.

  3. monitoring

    All affected web components are functioning as expected, and our monitoring shows error rates and login success back within normal ranges. The Iterable platform is fully operational. If you continue to see errors or unexpected behavior in the web app: Clear your browser’s cache...

  4. resolved

    This incident has been resolved.

Excerpts are shortened. Read the full incident on Iterable's status page.

API Update Timing Out none 2026-01-01 22:05 UTC 21h 53m

Affected: Global API Success

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    API Update is experiencing timeouts starting at 11:00 AM PST. The Engineering team is investigating. Next update at 3:00 PM PST.

  2. monitoring

    API Update has recovered. The engineering team will continue to monitor—next update on January 2nd, at 9 AM PST.

  3. monitoring

    API Update is stable. The engineering team will continue to monitor. Next update at 12:00 PM PST.

  4. resolved

    The engineering team determined that the issue has been resolved.

Excerpts are shortened. Read the full incident on Iterable's status page.

Delayed Message Sends none 2025-11-18 14:46 UTC 1h 52m

Affected: Global System Webhooks, Global Campaign Sends, Global Proof Sends, Global Journey Processing

Failure mode: A provider or third-party service this vendor depends on failed. Root cause: A failure at a provider or vendor they depend on. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

  1. identified

    We are currently experiencing delays in processing certain message sends and exit-trigger workflows. This is due to an upstream outage affecting some of our backend components that handle message publishing.

  2. monitoring

    Upstream issue has been resolved. we will continue monitoring.

  3. monitoring

    All processing is back to normal. We continue to monitor.

  4. resolved

    This incident has been resolved.

Excerpts are shortened. Read the full incident on Iterable's status page.

Platform Impact – Third-Party Vendor Disruption (Regional) major 2025-10-20 13:15 UTC 8h 40m

Affected: Global Web Application, Global API Success, Global Links, Global System Webhooks, Global Partner Webhooks, Global Analytics Processing, Global Catalog, Global API Ingestion, Global Campaign Sends, Global Proof Sends

Failure mode: A provider or third-party service this vendor depends on failed. Root cause: A failure at a provider or vendor they depend on. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

  1. investigating

    We’re currently investigating an issue that is impacting multiple parts of our platform. Some customers may experience increased error rates, timeouts, or degraded performance when accessing our services. Issues are related to a third-party experiencing issues. Our engineering...

  2. identified

    We’re continuing to experience issues impacting multiple parts of our platform. Customers will continue to experience increased error rates, timeouts, or degraded performance when accessing our services. The issue remains related to a third-party provider experiencing ongoing...

  3. identified

    We're continuing to experience issues impacting multiple parts of our platform. Customers may continue to experience increased error rates, timeouts, or degraded performance when accessing our services. Sends or Journeys that are delayed will be queued automatically on our end...

  4. identified

    We are now also seeing issues connecting to the Iterable application itself. We are continuing to work through these network issues with our upstream provider to restore full access to our customers. Any scheduled campaigns or journeys are still being queued to our backend. Next...

  5. identified

    We are now also seeing issues connecting to the Iterable application itself. We are continuing to work through these network issues with our upstream provider to restore full access to our customers. Any scheduled campaigns or journeys are still being queued to our backend. Next...

  6. identified

    We are seeing improvements for customer access to the Iterable web app and global API endpoints. Customers may still continue to experience increased error rates, timeouts, or degraded performance when accessing our services due to continued issues with our provider. Sends or...

  7. identified

    While we are continuing to see improvements for customer access to the Iterable web app and global API endpoints, customers may still experience an elevated error rate, timeouts, or degraded performance when accessing our services due to continued issues with our provider. Sends...

  8. monitoring

    Our provider has informed us that they have applied the necessary mitigations across the whole affected region and we are seeing our services resume normal operations. There are still some queued sends and journeys that we are working through but there is no additional ongoing...

  9. monitoring

    Iterable's web application and global API are now fully operational, and service levels have returned to normal. All queued sends are up to date. Some downstream effects to note: our data warehouse providers will experience some data sync delays. This includes Iterable AI...

  10. resolved

    All ongoing issues from today have now been resolved, including all AI and datasharing services that were previously still experiencing occasional latency. For any questions, please contact support@iterable.com. Thank your for patience and understanding as we worked through...

Excerpts are shortened. Read the full incident on Iterable's status page.

Incident Status Update: Journey Exit Malfunction minor 2025-10-09 14:48 UTC 5d 5h

Affected: Global Web Application

Failure mode: A provider or third-party service this vendor depends on failed. Root cause: A failure at a provider or vendor they depend on. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

  1. investigating

    Any visitor who entered a delay tile longer than 24hrs that received an exit match post 8:30 PM PST yesterday October 8th, with no subsequent updates to their exit rules may continue to experience incorrect journey exits until October 10th 8:30 PM PST. Any user receiving an exit...

  2. investigating

    We are continuing to investigate this issue.

  3. monitoring

    The engineering team has successfully identified the root cause of this incident: a major infrastructure issue experienced by one of our key vendors. We are actively partnering with the vendor to ensure the issue is fully resolved and to prevent any future impacts to our...

  4. monitoring

    The key vendor has confirmed the defect on their side and has taken steps to prevent recurrence. We will continue to closely monitor the affected systems. We will provide our next status update before 3:00 PM PT.

  5. monitoring

    There is no further recurrence of the infrastructure issue from our key vendor. We will continue to monitor the affected infrastructure for recurrence. Next Update: We will provide our next status update before 9:00 AM PT on October 10, 2025.

  6. investigating

    The fix applied by our key vendor did not work. The impact is that a limited amount of exit rule data was lost. We began observing a recurrence of the same limited data loss on October 10th, 2025, at 7:00 AM PST. We are working with the vendor to investigate further. Next...

  7. investigating

    We worked with the vendor to fix this issue. The vendor recommended an approach to prevent additional data loss. We will apply the fix and monitor the affected systems for recurrence. Next Update: We will provide our next status update before 12:00 PM PT.

  8. monitoring

    We have successfully deployed the fix and will start monitoring for recurrences. Next Update: We will provide our next status update before 3:00 PM PT.

  9. monitoring

    There are no further recurrences after the fix was deployed today. We will continue to monitor the affected systems over the weekend.

  10. monitoring

    There were no recurrences observed from October 10th afternoon to October 13th morning. We will continue to monitor until the end of the day. Next Update: We will provide our next status update before 5:00 PM PT.

  11. monitoring

    There were no recurrences observed today. We will continue to monitor for recurrence. Next Update: We will provide our next status update before 12:00 PM PT tomorrow.

  12. resolved

    We have observed no recurrence since 10/10. The issue now is resolved. Please feel free to reach out to support for further questions.

Excerpts are shortened. Read the full incident on Iterable's status page.

Catalog Service Disruption none 2025-09-22 07:24 UTC 1h

Failure mode: Data delayed, stale, lost or corrupted, or needing replay after recovery. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. identified

    We are currently experiencing a service disruption with our catalog service, which may prevent some customers from sending messages that use templates with catalog data. Messages that do not use catalog data are not impacted. We've identified the root cause and are actively...

  2. identified

    We have started rolling out the fix to individual machines and are seeing some of our messaging capabilities recover. Some customers may still have a delay in sends that use templates with catalog data, and ingestion delays for updating catalog data. We are continuing to apply...

  3. resolved

    The issue with our catalog service has been fully resolved, and all services are now operating normally. Any delayed sends and catalog updates are being processed currently. Messages that were queued during the disruption will be automatically retried and delivered.

Excerpts are shortened. Read the full incident on Iterable's status page.

[Europe] 504 Errors on Data Export API (EU) minor 2025-09-15 17:28 UTC 29m

Affected: Global API Success

Failure mode: CDN or edge layer failed or served bad content. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. investigating

    Status: Investigating Summary: We are currently investigating an issue causing 504 Gateway Time-out errors for requests to the /api/export/data.json endpoint affecting customers hosted in our European Data Center (EUDC) ONLY. All other APIs are functional. The issue was first...

  2. resolved

    This incident has been resolved.

Excerpts are shortened. Read the full incident on Iterable's status page.

Resolved: api-hightraffic Errors in Europe none 2025-09-11 16:37 UTC 0m

Affected: Global API Success

Failure mode: A zone, region or data center became unavailable. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. resolved

    Status: Resolved Between 3:50 pm and 5:55 pm CEST customers in our EUDC (European Data Center) region may have experienced an increased 5xx error rate, approximately 10-15%, when making api requests. This issue was isolated to Europe; no impact was observed in other regions. If...

Excerpts are shortened. Read the full incident on Iterable's status page.

Iterable Auth service degraded major 2025-08-08 23:44 UTC 45m

Failure mode: Login, tokens, certificates or credentials failed. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    We are seeing issues accessing app.iterable.com. We have identified an issue in one of our services and Engineering is working on a fix. Iterable API is not impacted. All campaigns and journeys are processing as expected. Next update will be by 5PM PDT.

  2. monitoring

    We have addressed the issue in our service and app.iterable.com is accessible again. Engineering is monitoring the situation. Next update will be at or before 5:30PM PDT.

  3. resolved

    Our login service has recovered and app.iterable.com is fully accessible. If you are still having issues, please reach out to support.

Excerpts are shortened. Read the full incident on Iterable's status page.

Embedded Messaging Send Skips are Impacting Customers minor 2025-08-01 20:32 UTC 4h 55m

Affected: SMS Sends, Global Campaign Sends

Failure mode: A code release introduced the failure. Root cause: A bug in the vendor's own code. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    We are currently investigating an increase in embedded Messaging skips, which appears to be impacting most customers. Other sends globally are not impacted. Our team is actively working to identify the root cause of this issue. Next Update: We will provide our next update by...

  2. investigating

    We are continuing to investigate this issue.

  3. identified

    The issue with embedded messages incorrectly being send skipped has been identified and the team is actively working to remediate it. We have confirmed that the impact is only to embedded messages triggered by Journeys. The aforementioned remediation should be completed shortly...

  4. identified

    The send skip behavior for journey triggered embedded messages has been resolved. The team is still working on validating that the remediation has restored full functionality. Next update: We will provide our next update at 4:00 PM PDT.

  5. identified

    During the validation of the remediation, the team identified that another fix was necessary to fully restore the functionality of journey triggered embedded messages. They are working on testing and releasing the fix. Next update: We will provide our next update at 6:00 PM PDT.

  6. identified

    The additional fix has been tested successfully and deployment is in progress. This process will take another 15-30 minutes to rollout to all customers. This will fully restore functionality for journey triggered embedded messaging, which were the only sends impacted. Next up:...

  7. resolved

    This incident has been resolved and functionality has been fully restored for journey triggered embedded messaging sends. If more information is needed, please reach out to support@iterable.com. This will be the final update.

Excerpts are shortened. Read the full incident on Iterable's status page.

Ingestion Delays and API Failures on C18 minor 2025-07-28 10:21 UTC 27m

Affected: User Updates, List Uploads, User Deletions

Failure mode: Exhausted memory, disk, CPU, queues, connection pools or quotas. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. investigating

    Beginning at 2:30 AM PDT, Iterable Engineering detected an issue with our C18 clusters leading to delays on user updates, list uploads, user deletion, and event processing. Additionally, there were failures on API calls to endpoints such as event/track for a period of time. Next...

  2. monitoring

    Ingestion lag for C18 is recovering and success rates for all services are back to normal. We will continue to monitor the cluster until the ingestion backlog has fully cleared. Next update at 4:30 AM PDT or sooner.

  3. monitoring

    We are continuing to monitor for any further issues.

  4. resolved

    The system has finished working on backlog of data and the data is caught up. All systems are operational and have no issues. Engineering will continue to monitor the cluster performance.

Excerpts are shortened. Read the full incident on Iterable's status page.

Slow Journey Processing none 2025-07-03 14:27 UTC 2h 2m

Failure mode: Data delayed, stale, lost or corrupted, or needing replay after recovery. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. investigating

    Around 2:16 AM PDT, we detected delays on journey processing. Current impact to our platform is stuck users and slow email sends, we are confirming other impact. Our team is actively working to resolve this as soon as possible. Next update will be provided at 8:30am PDT.

  2. monitoring

    We have deployed a fix to production at 7:30 AM PDT. Engineering continues to monitor effectiveness of the fix.

  3. resolved

    This incident is resolved and the backlog of processing has recovered. After further analysis engineering has provided an update that journey recovery started 6:30 to 7AM PST.

Excerpts are shortened. Read the full incident on Iterable's status page.

Ingestion Delays minor 2025-06-12 19:17 UTC 1h 58m

Affected: Global API Ingestion

Failure mode: A provider or third-party service this vendor depends on failed. Root cause: A failure at a provider or vendor they depend on. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

  1. identified

    Starting around 11:30 AM PT, Iterable Engineering detected a growing lag across clusters causing delays with user updates, list uploads, user deletion, and event processing. Engineering has identified the cause related to an upstream network provider and is working to remediate...

  2. identified

    We are still seeing ingestion lag across all clusters. No data is being dropped; however, there are delays in user updates, list uploads, user deletion, and event processing, which may also impact journey triggers. Next update at 2:00 PM PT or sooner

  3. monitoring

    We are seeing ingestion lag recovering across all clusters and catching up on the remaining backlog. The team is continuing to monitor. Next update at 2:15 PM PT or sooner

  4. resolved

    Ingestion lag has recovered across all clusters. This incident is now resolved.

Excerpts are shortened. Read the full incident on Iterable's status page.

Sends Outage/Delays Across all Mediums none 2025-06-06 04:00 UTC 15m

Failure mode: A zone, region or data center became unavailable. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. resolved

    US Data Center Sends Cluster Outage Summary: Between 12:14 AM and 12:55 AM EDT, our US Data Center's sends cluster experienced a critical outage, effectively preventing message delivery. All publishes during this period were safely backed up to our secondary queues. Resolution:...

Excerpts are shortened. Read the full incident on Iterable's status page.

C22 ingestion delays minor 2025-05-20 14:52 UTC 4h 11m

Affected: Journey Processing, User Updates, List Uploads, User Deletions

Failure mode: Data delayed, stale, lost or corrupted, or needing replay after recovery. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    Starting at around 9 pm PT 5/19, cluster 22 began experiencing delays delays on user updates, list uploads, user deletion, and event processing. Customers could experience delays of up to 6 hours currently. The engineering team is currently investigating and working to remediate...

  2. investigating

    We are continuing to investigate this issue.

  3. monitoring

    Our engineering team has rolled out a fix to remediate the delays seen on c22 for user updates, list uploads, user deletion, and event processing. The system is currently working through the backlog of data and we should see it drained in a couple of hours. Engineering will...

  4. resolved

    The system has finished working on backlog of data and the data is caught up. All systems are operational and have no issues. Engineering will continue to monitor the cluster performance.

Excerpts are shortened. Read the full incident on Iterable's status page.

Mailgun has Degraded Performance none 2025-05-19 21:42 UTC 46m

Failure mode: A provider or third-party service this vendor depends on failed. Root cause: A failure at a provider or vendor they depend on. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    Mailgun is reporting degraded performance across their application https://status.mailgun.com/. We are monitoring mailgun and their performance.

  2. monitoring

    We are monitoring the affect of the mailgun degradation

  3. monitoring

    Mailgun has implemented a fix and is in Monitoring status

  4. resolved

    This incident has been resolved.

Excerpts are shortened. Read the full incident on Iterable's status page.

Queries to data stored on C120 cluster are failing major 2025-04-23 06:50 UTC 40h 27m

Affected: Email Sends, Journey Processing, Push Sends, SMS Sends, User Updates, List Updates, User Deletions

Failure mode: Data delayed, stale, lost or corrupted, or needing replay after recovery. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    All queries to Cluster C120 are failing as there was a problem in cluster formation. The problem started around 4/22/2025 19:10 pm PDT. Any campaign sent during this time would have been impacted. Ingestion lag is currently building up. The team is restoring the data into an...

  2. investigating

    We are continuing to investigate this issue.

  3. investigating

    The team has finished restoring data from C120 to a new cluster. Engineering team is working on migrating the in-flight data. This can take few hours, and we will update once we receive a time frame. The ingestion lag will deplete once the data for the organizations are...

  4. identified

    The team is still working on a cutover to the new cluster. Once the changes propagate to all of the services we should start to see the ingestion lag deplete. Once cutover is complete Blast campaigns should resume. Next update will be at 4/23/2025 10 am PDT.

  5. identified

    Iterable's engineering team is continuing their work on the cutover to the new cluster. They are working on finalizing details to make sure we minimize impact as much as possible. Next update will be at 4/23/2025 11 am PDT.

  6. monitoring

    At this point all services have up-to date changes. Iterable's engineering team have cutover to the new cluster. The impacted customers are in the process of being notified. We are actively monitoring. Next update will be at 4/23/2025 around 12:15 pm PDT.

  7. monitoring

    Engineering successfully completed the cluster cutover at 11:03 AM PDT. Ingestion for customers previously on Cluster 120 is now flowing normally through the new Cluster 141. We’ve also confirmed that campaign scheduling has resumed and is running successfully on Cluster 141....

  8. resolved

    Engineering has confirmed that the cluster is performing as expected. No new issues have occurred since the last update. We also wanted to share that further investigation post-incident has uncovered that, during the impact window, users that were set to start journeys via...

Excerpts are shortened. Read the full incident on Iterable's status page.

Ingestion Delay on Clusters 100 and Above critical 2025-03-14 16:56 UTC 4h 17m

Affected: Global API Ingestion

Failure mode: A code release introduced the failure. Root cause: A bug in the vendor's own code. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    Our engineering team has identified an issue causing ingestion delay for all Clusters 100 and above. We are deploying the fix right now. Customers on these impacted clusters could be experiencing delays with user updates, list uploads, user deletion, and event processing. Email...

  2. investigating

    The engineering team is still working on the fix, and customers on Clusters 100 and above will still be experiencing some delays with user updates, list uploads, user deletion, and event processing. Email sends and journey processing may see impact as well. Data is only being...

  3. identified

    Engineering has put in the fix. Some clusters have fully recovered, and we are actively working on the ones that have not. For those customers that are still impacted, you will still be experiencing some delays with user updates, list uploads, user deletion, and event...

  4. monitoring

    Engineering has finished pushing the fix. Customers on Clusters 116, 120, and 132 may still see an ingestion lag greater than 1 hour. All other Clusters have recovered. Data was never dropped. The next update is at 3 PM PDT or sooner.

  5. resolved

    Our engineering team has confirmed at 2:06 PDT that all ingestion should be back to normal and is stable. This incident is now resolved.

Excerpts are shortened. Read the full incident on Iterable's status page.

Degraded Performance with Campaign Sends, Scheduled Journeys and Exports minor 2025-03-12 15:45 UTC 6d 4h

Affected: Global Campaign Sends

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    The engineering team has received reports of customers experiencing degraded performance with campaign sends, scheduled journeys, and exports. They are currently investigating the issue. If you believe you're seeing this impact, please contact support@iterable.com. The next...

  2. investigating

    Investigating - The engineering team is still investigating reports of customers experiencing degraded performance with campaign sends, scheduled journeys, and exports. If you believe you're seeing this impact, please contact support@iterable.com. The next update will be before...

  3. investigating

    The engineering team is continuing to investigate reduced performance with campaign sends, scheduled journeys, and exports. This issue occurs sporadically and does not affect every campaign or journey. If you think you're affected by this issue, please contact...

  4. investigating

    Engineering has continued working through its remediation steps. Currently, it is focusing on testing a fix on certain services individually to help ensure that our customers do not feel a greater impact. We will provide the next update by 12 PM PDT or sooner.

  5. investigating

    Engineering is continuing to pursue remediation steps. They are still investigating different solutions and attempting to test fixes on certain services individually to help ensure that more significant impact is not felt for our customers. We will provide the next update by 3...

  6. investigating

    Engineering continues to work on remediation steps and is testing fixes on individual services to minimize customer impact. The scope of the issue remains limited, affecting less than 1% of calls across our customer base. Customers may experience sporadic errors or failures in...

  7. investigating

    Engineering has implemented remediation steps, but they have not resolved the issue. The team is actively exploring alternative solutions and evaluating their effectiveness. The scope of the issue remains unchanged, affecting less than 1% of calls across our customer base. We...

  8. investigating

    Engineering continues to pursue different fixes. Performance for our customers has been partially remediated and improved. Customers should be experiencing a reduction in delays in campaign sends, journey triggers, and processing. The scope of the issue remains unchanged,...

  9. investigating

    Engineering is still investigating a complete fix, but is still seeing performance improvements across delays in campaign sends, journey triggers, and processing. We are continuing to work on rolling out improvements and solutions. The scope of the issue remains unchanged,...

  10. investigating

    Engineering is still investigating a complete fix, and are monitoring changes we put in last night to see if it mitigates this issue. We are seeing performance improvements across delays in campaign sends, journey triggers, and processing. The scope of the issue remains...

  11. investigating

    Engineering is actively monitoring the fix implemented last night. While initial metrics show signs of improvement, we are still assessing whether delays in campaign sends, journey triggers, and processing have been fully resolved. To ensure a thorough resolution, we will...

  12. investigating

    Engineering continues to monitor the changes we made this weekend. So far, our monitoring has indicated significant performance improvements. The next step is to observe and confirm that the changes have mitigated the delays in campaign sends, journey triggers, and processing...

  13. monitoring

    Engineering has reviewed the changes it pushed through this weekend and has not seen any delays in scheduled campaigns, scheduled journey triggers, partial sends for a subset of scheduled campaigns, and export list calls. We are currently monitoring the fix across all our...

  14. monitoring

    Engineering is no longer seeing any delays in scheduled campaigns, scheduled journey triggers, partial sends for a subset of scheduled campaigns, and export list calls. We will continue to monitor this fix across all our systems and ensure there are no errors for the next day....

  15. resolved

    Our Engineering team has confirmed at 12:43PM PDT that all scheduled campaigns, scheduled journey triggers, sends for scheduled campaigns, and export list calls are behaving normally and are stable. This incident is now resolved.

Excerpts are shortened. Read the full incident on Iterable's status page.

Journeys are having issues fetching, creating and saving journeys. major 2025-03-07 09:57 UTC 8h 38m

Affected: Global Web Application

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. investigating

    We are currently investigating an issue affecting Journeys where fetching, creation and saving of journeys is not working correctly. Customers may experience delays or unexpected behavior when using Journeys. Our Engineering team is actively working to identify the root cause....

  2. monitoring

    A fix has been deployed and journey function is operating normally. Moving to monitoring state.

  3. resolved

    We do not see any more errors in journey edits.

Excerpts are shortened. Read the full incident on Iterable's status page.

ingestion delays on c22 minor 2025-01-13 22:41 UTC 2h 22m

Affected: User Updates, List Uploads

Failure mode: Data delayed, stale, lost or corrupted, or needing replay after recovery. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. identified

    Our engineers have identified and are deploying a fix for an issue with ingestion delays. The current delay is roughly 4 hours. This issue only affects customers on cluster 22. You can find your cluster id displayed on the Project Settings page. Next update by 3:30PST.

  2. identified

    Ingestion backlogs on c22 are shrinking due to the deployed fix. Next update by 4:30PST.

  3. monitoring

    Ingestion on c22 has fully caught up. Our engineering team is continuing to monitor the situation.

  4. resolved

    Our engineering team hasn't detected any additional anomalies, and ingestion has been stable. This incident is now resolved.

Excerpts are shortened. Read the full incident on Iterable's status page.

Intermittent API errors major 2025-01-07 19:56 UTC 2h 14m

Affected: Global API Success

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. investigating

    Our on-call engineers are investigating intermittent partial API outages lasting several minutes over the past couple hours. Our monitoring and alerting first detected the issue near 8:53 am PST. Next update by 1PM PST.

  2. monitoring

    Our engineering team implemented a fix for the issue and have seen no further API outages. We are continuing to monitor the situation.

  3. resolved

    We have not seen further API outages since the last update. This incident is now resolved.

Excerpts are shortened. Read the full incident on Iterable's status page.

Event-Triggered Journeys Delays Ingesting New Users major 2024-12-31 14:29 UTC 3h 56m

Affected: Global Web Application

Failure mode: A code release introduced the failure. Root cause: A bug in the vendor's own code. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. identified

    Summary: Event-Triggered Journeys Delays Ingesting New Users. The issue originated from errors in the workflow-entrance-trigger pods, causing a significant backlog in processing. There is no impact to Scheduled Journeys and API Triggered Journeys . Actions Taken The...

  2. identified

    We are continuing to work on a fix for this issue.

  3. resolved

    This issue is now resolved and the backlog is completely drained. Event trigger journey's as of 7:25 AM PST are back to normal and processing as expected. If you still have any further questions please reach out to support@iterable.com.

Excerpts are shortened. Read the full incident on Iterable's status page.

Segmentation page intermittent slowness minor 2024-11-25 17:07 UTC 3h 11m

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. investigating

    Intermittent slowness observed on segmentation page. The engineering team is currently investigating and we will update at 9:45am PT.

  2. identified

    The issue has been identified and fix has been implemented. The engineering team will continue monitoring systems and we will update at 10:15am PT or sooner.

  3. monitoring

    The issue has been mitigated. The engineering team will continue monitoring systems and we will update at 11:30am PT or sooner.

  4. monitoring

    The issue has been mitigated. The engineering team continues to monitor systems. If you are still seeing any impact please reach out to support@iterable.com

  5. resolved

    The issue has been resolved.

Excerpts are shortened. Read the full incident on Iterable's status page.

Web Application Errors major 2024-11-24 14:49 UTC 26h 19m

Affected: Global Web Application, Global API Success, Global System Webhooks, Global Partner Webhooks, Global Campaign Sends

Failure mode: Data delayed, stale, lost or corrupted, or needing replay after recovery. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    Beginning at 5 AM PST we were alerted to a spike in API errors and Web App errors. Our engineering team is investigating. Next update will be at 7:30 AM PST or sooner. If you have questions please reach out to support@iterable.com

  2. investigating

    Iterable engineering has put in mitigations and is seeing recovery in the web application and API endpoints. We are continuing to work on remediating delays to sends and journey processing. Next update by 8:30 am PT

  3. investigating

    We're seeing continued improvements in the web application and API performance. We are still working to resolve delays in sends and webhooks. Next update by 9:00 AM PT or sooner

  4. investigating

    We have put in further mitigations to restore send speeds and are working through the send backlog. We are continuing to work on recovering webhooks. Next update by 9:30 AM PT or sooner

  5. investigating

    The Engineering team is investigating web and API errors. Next update by 9:45 AM PT or sooner.

  6. investigating

    The Engineering team is implementing mitigations to address the intermittent web and API errors as well as send and webhook delays. Next update by 10:15 AM PT or sooner.

  7. investigating

    The Engineering team is continuing to investigate and implement mitigations to address DB issues causing web and API errors. Sends, journey processing, and webhooks will continue to see delays. Customers may also experience issues with pages of the web app failing to fully load...

  8. investigating

    The Engineering team is continuing to investigate and implement mitigations to address DB issues causing web and API errors. Sends, journey processing, and webhooks will continue to see delays. Customers may also experience issues with pages of the web app failing to fully load...

  9. investigating

    The Engineering team is continuing to investigate and implement mitigations to address DB issues. Web errors and journey latency are going down and send rate is going up. Customers may still experience degradation and issues with pages of the web app failing to fully load and an...

  10. identified

    The Engineering team is continuing to address DB issues. We are slowly bringing up the scaled down instances of various services. Web errors and journey latency are going down and send rate is going up. Customers may still experience issue with web app failing to fully load and...

  11. identified

    The issue is now identified. The engineering team is slowly bringing up the scaled down instances of various services. Customers should begin seeing improvements in access to the web app, and message send rates are beginning to recover. All services except webhooks are scaled up...

  12. identified

    The issue is identified. The engineering team continues to scale up instances of various services. Customers should begin seeing improvements in access to the web app, and message send rates are beginning to recover. We should continue to see improvements across the app as we...

  13. identified

    The issue is identified. The engineering team continues to scale up instances of various services. Customers should continue seeing improvements in access to the web app, and message send rates continue to recover. We continue to see improvements across the app as we catch up on...

  14. monitoring

    The engineering team has completed the instances scale up of various services. Access to the web app, and message send rates are back to normal processing. We continue to see improvements across the app as we catch up on the messaging backlog. Most systems are caught up with...

  15. monitoring

    Engineering team has confirmed that we continue to process at normal rates. Our processing of messaging backlog has caught up. Engineering is continuing to monitor our systems, if you are still seeing any impact please reach out to support@iterable.com

  16. monitoring

    We are continuing to monitor for any further issues.

  17. monitoring

    The Engineering team continues to monitor system performance. There have been no performance degradations. Next update by 12 pm PT or sooner. If you are still seeing any impact please reach out to support@iterable.com

  18. monitoring

    This incident has been resolved.

  19. resolved

    This incident has been resolved.

Excerpts are shortened. Read the full incident on Iterable's status page.

Spike in web-app errors causing platform degredation minor 2024-11-22 15:15 UTC 5h 5m

Affected: Global Web Application

Failure mode: Traffic surge or denial-of-service attack. Root cause: Not enough capacity for the load. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    Iterable engineering became alerted to high error rates on web-app this morning around 6:00 a.m. PT. After investigating the this issue it appears that these spikes could be happening at the top of each hour as higher volumes of campaigns are scheduled to be sent out. The first...

  2. monitoring

    Iterable engineering has mitigated the errors and slowdown affecting the web UI and API endpoints at the top of the hour. The last one occurred between 7:02 - 7:12 am PT. We are continuing to monitor. Next update by 9 am PT

  3. monitoring

    There have been no further application degradations, and the engineering team will continue monitoring through the morning. Next update by 12 pm PT or sooner

  4. resolved

    There have been no further application degradations and the platform looks stable. This will be our last update as we are marking this incident as resolved. Engineering team will continue to monitor closely.

Excerpts are shortened. Read the full incident on Iterable's status page.

Errors appearing and delays accessing the platform minor 2024-11-20 21:56 UTC 19h 41m

Affected: Global Web Application

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. investigating

    We are currently experiencing errors and slowness in accessing the Iterable platform. This is only impacting the Web Interface and is not impacting sends, journey processing. The engineering team is currently investigating. Next update by 3pm PT.

  2. monitoring

    Engineering has confirmed that the errors and slowness were from 1:30-1:50 PM PT. We have recovered since then, and we're continuing to monitor and work on further preventative measures. Next update by 4 pm PT.

  3. identified

    The engineering team has identified a fix for the intermittent degradations and is working on a remediation. We're continuing to monitor and add further preventative measures. Next update by 5 pm PT.

  4. monitoring

    The engineering team has deployed a fix for the intermittent degradations. We are continuing to monitor. If you are still seeing issues with platform slowness, please contact Iterable support. Iterable campaign sends and journey processing were not impacted during this incident....

  5. monitoring

    There have been no further application degradations, and the engineering team will continue monitoring through the morning. Next update by 9am PT

  6. resolved

    This incident has been resolved.

Excerpts are shortened. Read the full incident on Iterable's status page.

Website Degraded minor 2024-11-20 15:34 UTC 4h 59m

Affected: Global Web Application

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    From 4:45 AM to 5:45 AM PT, we saw errors and slowness reaching app.iterable.com. Accessing all areas of the platform may have been met with slow loads, but sends and logins are not affected. The engineering team is currently investigating and we will update at 8:30am PT.

  2. monitoring

    Iterable Engineering has confirmed that we've recovered from the errors that were previously noted as reaching app.iterable.com. Users should be seeing platform speeds return to normal. Engineering is continuing to monitor and put preventative measures in place, but if you are...

  3. resolved

    This incident has been resolved.

Excerpts are shortened. Read the full incident on Iterable's status page.

Website Degraded none 2024-11-19 14:00 UTC 0m

Failure mode: Orchestration, control plane, service discovery or load balancing failed. Root cause: Hardware or facility. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. resolved

    From 5:50am PST until 6:16am PST, the Iterable website was slower to respond due to a degraded host. The host was removed and service returned to normal.

Excerpts are shortened. Read the full incident on Iterable's status page.

Issue With Dynamic Content For InApp Messages none 2024-11-14 17:45 UTC 0m

Affected: Global Campaign Sends

Failure mode: A code release introduced the failure. Root cause: A bug in the vendor's own code. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. resolved

    Today, November 14th around 5:00 am PST, Iterable was alerted to an issue where inApp campaigns were not rendering handlebars correctly. Iterable engineers began investigating the issue and identified the cause of the issue had started around 10:00 am PST on November 13th. Once...

Excerpts are shortened. Read the full incident on Iterable's status page.

Ingestion Delay On Subset Of Clusters minor 2024-11-12 17:18 UTC 2h 6m

Affected: Email Sends, Email Sends, Email Sends, Email Sends, Email Sends, Email Sends, Journey Processing, Journey Processing, Journey Processing, Journey Processing, Journey Processing, Journey Processing, User Updates, User Updates, User Updates, User Updates, User Updates, User Updates, List Uploads, List Uploads, List Uploads, List Updates, List Updates, List Updates, User Deletions, User Deletions, User Deletions, User Deletions, User Deletions, User Deletions

Failure mode: Data delayed, stale, lost or corrupted, or needing replay after recovery. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    Starting at around 6:30 am PT, Iterable Engineers were notified that lag on our ingestion topics was growing on a subset of clusters. Customers on these impacted clusters could be experiencing delays with user updates, list uploads, user deletion, and event processing. Email...

  2. monitoring

    The engineering team has identified the cause and remediated the delays on these subset of clusters. Most impacted ingestion topics have recovered and customers should start to see data being ingested normally again. We are still working through some backlog, so the team will...

  3. resolved

    Ingestion has returned to normal for all impacted clusters and this issue is now resolved.

Excerpts are shortened. Read the full incident on Iterable's status page.

Delays in processing System Webhooks minor 2024-10-25 18:48 UTC 24h 47m

Affected: Global System Webhooks

Failure mode: Data delayed, stale, lost or corrupted, or needing replay after recovery. Root cause: A bug in the vendor's own code. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. identified

    The code fix for the slowness of System webhooks is deployed. Currently the queue is depleting. This could take several hours to deplete the backlog completely. During this time the performance will be degraded. Next update at 6pm PST.

  2. monitoring

    System webhook backlog is depleted and the incident is resolved.

  3. resolved

    This incident has been resolved.

Excerpts are shortened. Read the full incident on Iterable's status page.

Embedded Messaging API down none 2024-10-22 17:33 UTC 19h 46m

Failure mode: A code release introduced the failure. Root cause: A bug in the vendor's own code. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    Errors with our Embedded Messaging API are not occurring any more. The engineering team remediated the issue by clearing the offending database requests that were queued up. This is a temporary fix. A permanent remediation with code fix is still in progress.

  2. monitoring

    Errors with our Embedded Messaging API are not occurring any more. A code fix has been deployed.

  3. resolved

    This issue has been fully resolved as of October 22nd at 5pm PST.

Excerpts are shortened. Read the full incident on Iterable's status page.

Issues impacting Email Sending major 2024-10-15 16:06 UTC 5h 25m

Affected: Global Campaign Sends, Global Proof Sends

Failure mode: A provider or third-party service this vendor depends on failed. Root cause: A failure at a provider or vendor they depend on. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

  1. monitoring

    Starting around 7:27 AM PST, Iterable engineers began receiving alerts about elevated error rates on a 3rd-party ESP provider's API endpoints. Iterable has been working directly with our 3rd party vendor and they have released a fix and as of 8:07 AM PST we are seeing successful...

  2. monitoring

    The fix from our third party vendor has been stable and we're seeing email campaigns are sending successfully again. We are currently working to resend the emails that were impacted during the impact window. Next update will be at 11:00 am pst or sooner.

  3. monitoring

    Engineering is identifying alternative methods to identify affected campaigns and will contact customers to confirm whether redelivery is needed. Additionally, 3rd party vendor recovered as of 8:07 am PT and there are no issues with send. Next update at 12 pm PT.

  4. monitoring

    We are continuing to monitor for any further issues.

  5. monitoring

    Engineering is identifying alternative methods to identify affected campaigns and will contact customers to confirm whether redelivery is needed. Additionally, the 3rd party vendor recovered as of 8:07 am PT, and there are no issues with send. Next update at 12 pm PT.

  6. monitoring

    Engineering is continuing to work on identifying alternative methods to determine affected campaigns. We will be reaching out to impacted customers to confirm whether redelivery is necessary. The 3rd party vendor recovered at 8:07 AM PT, and there are no ongoing issues with...

  7. resolved

    As of 4:16 PM PST, all affected messages have been successfully processed, and the incident is now resolved. Our engineering team has confirmed that all campaigns have been reviewed, and there are no outstanding issues. The 3rd party vendor's recovery at 8:07 AM PT resolved the...

Excerpts are shortened. Read the full incident on Iterable's status page.

Data Ingestion Delay on c14 minor 2024-09-24 13:34 UTC 2h 14m

Affected: User Updates, List Uploads

Failure mode: Data delayed, stale, lost or corrupted, or needing replay after recovery. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    We are experiencing data ingestion delays on c14, affecting bulk updates, list uploads, and custom events. However, no data is being dropped.

  2. monitoring

    Engineering has put in the recovery steps, and are seeing ingestion begin to catch-up. Customers might still see some slight delays while we fully recover. We will update next when ingestion has completely caught up.

  3. resolved

    This incident has been resolved.

Excerpts are shortened. Read the full incident on Iterable's status page.