">
sevanalysis

Opsgenie

Apps and SaaS. Source: Opsgenie status page.

Incidents, last 90 days
2
Median to mitigation
3d 10h
Median to resolution
3d 10h
p95 to resolution
6d 19h
Naming an upstream provider
0%
With a postmortem on the status page
50%

What failed, last 90 days

A configuration change caused the failure1
A provider or third-party service this vendor depends on failed1

Tagged by claude-haiku-5-5 from each incident's updates. See the methodology.

Incidents per month

All recorded incidents

Disrupted Opsgenie/JSM availability critical 2026-09-14 16:46 UTC 57m

Affected: Web Application, Web Application, Mobile Application, Mobile Application

Failure mode: A configuration change caused the failure. Root cause: A configuration change. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

  1. investigating

    We are actively investigating reports of a service disruption affecting Opsgenie and Jira Service Management. We will share updates here as more information is available.

  2. identified

    We have identified the likely cause of the issue, and our teams are diligently working on a mitigation. We will continue to share additional updates here as more information is available.

  3. monitoring

    The issue has now been resolved, and services are operating normally for all affected customers. We will continue to monitor closely to confirm stability.

  4. resolved

    On September 14, 2026, Opsgenie and Jira Service Management experienced a disruption, and impacted users saw delayed alerts notification and inability to view the alerts in the user interface. The issue has now been resolved, and the service is operating normally for all...

  5. postmortem

    ### Summary On September 14, 2026, between 15:30 and 17:10 UTC, Atlassian customers using Opsgenie, Jira Service Management, and Compass experienced delays in receiving alert notifications and intermittent failures when using Ops features. The issue was triggered by long-running...

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Issues with Microsoft Teams notifications for Opsgenie customers major 2026-08-27 10:26 UTC 6d 19h

Affected: Outgoing Integration Flow, Outgoing Integration Flow

Failure mode: A provider or third-party service this vendor depends on failed. Root cause: A human mistake during operations. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. identified

    On August 19, 2026, we identified an issue affecting the delivery of Microsoft Teams notifications for Opsgenie standalone customers. The root cause has been identified and our teams are working with our partners to resolve this issue. This issue does not impact Jira Service...

  2. identified

    Our teams continue to work with our partners towards resolving the Microsoft Teams notification delivery issue for Opsgenie standalone customers. We will provide the next update in 72 hours or sooner if we have a material update.

  3. identified

    Our teams continue to work with our partners, towards resolving the Microsoft Teams notification delivery issue for Opsgenie standalone customers. We will provide the next update in 24 hours, or sooner if there is a significant update.

  4. identified

    Our teams continue to work with our partners towards resolving the Microsoft Teams notification delivery issue for Opsgenie standalone customers. We will provide the next update in 24 hours, or sooner if there is a significant update.

  5. identified

    Our teams continue to work with our partners towards resolving the Microsoft Teams notification delivery issue for Opsgenie standalone customers. We will provide the next update in 24 hours, or sooner if there is a significant update.

  6. resolved

    On August 19, we identified an issue affecting the delivery of Microsoft Teams notifications for Opsgenie standalone customers. The root cause was attributed to an operational error caused by our external partner, and they have taken corrective measures to restore services. If...

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Voice call failures for some India based numbers minor 2026-07-21 09:48 UTC 3d 3h

Affected: Incident Flow, Incident Flow, Alert Flow, Alert Flow, Email Notification Delivery, Email Notification Delivery, Public Website, SMS Notification Delivery, SMS Notification Delivery, Voice Notification Delivery, Voice Notification Delivery, Mobile Notification Delivery, Mobile Notification Delivery, Heartbeat Monitoring, Heartbeat Monitoring, Incident REST API, Incident REST API, Alert REST API, Alert REST API, Heartbeat REST API, Heartbeat REST API, Incoming Email Service, Incoming Email Service, Incoming Integration Flow, Incoming Integration Flow, Outgoing Integration Flow, Outgoing Integration Flow, Signup, Login & Authorization, Signup, Login & Authorization, Opsgenie Actions, Opsgenie Actions, Web Application, Web Application, Mobile Application, Mobile Application, Configuration REST APIs, Configuration REST APIs, Reporting & Analytics, Reporting & Analytics, Pricing & Billing, Pricing & Billing, Logs, Logs, Incoming Call Routing, Incoming Call Routing

Failure mode: A provider or third-party service this vendor depends on failed. Root cause: A failure at a provider or vendor they depend on. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

  1. identified

    We are working with our notification provider to resolve the issue. We will provide another update in 24 hours or as soon as more information becomes available.

  2. identified

    A small number of voice notifications for Jira Service Management and Opsgenie customers using India based mobile numbers may fail. Customers may see failed voice call logs and not receive the expected paging call. Our 3rd party notification provider is actively working towards...

  3. resolved

    Our notification provider has identified the root of the issue that is causing voice call failures to some India-based mobile numbers. We understand the inconvenience this may cause and appreciate your patience while our provider works on restoring the service. If you have...

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Atlassian Cloud Services impacted major 2025-10-20 07:56 UTC 21h 28m

Affected: Incident Flow, Incident Flow, Alert Flow, Alert Flow, Email Notification Delivery, Email Notification Delivery, Public Website, SMS Notification Delivery, SMS Notification Delivery, Voice Notification Delivery, Voice Notification Delivery, Mobile Notification Delivery, Mobile Notification Delivery, Heartbeat Monitoring, Heartbeat Monitoring, Incident REST API, Incident REST API, Alert REST API, Alert REST API, Heartbeat REST API, Heartbeat REST API, Incoming Email Service, Incoming Email Service, Incoming Integration Flow, Incoming Integration Flow, Outgoing Integration Flow, Outgoing Integration Flow, Signup, Login & Authorization, Signup, Login & Authorization, Opsgenie Actions, Opsgenie Actions, Web Application, Web Application, Mobile Application, Mobile Application, Configuration REST APIs, Configuration REST APIs, Reporting & Analytics, Reporting & Analytics, Pricing & Billing, Pricing & Billing, Logs, Logs, Incoming Call Routing, Incoming Call Routing

Failure mode: A provider or third-party service this vendor depends on failed. Root cause: A failure at a provider or vendor they depend on. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

Names an upstream provider: AWS.

  1. investigating

    We have noticed that Atlassian Cloud services are impacted and our teams are actively investigating the same. We shall keep you informed of the progress every hour.

  2. investigating

    Atlassian Cloud services are impacted and we are aware that our customers might not be able to create support tickets. Our teams are actively investigating the same. We shall keep you informed of the progress every hour.

  3. investigating

    We are experiencing an outage due to some issue at the end of our public cloud provider. We are working closely with them to get this resolved or mitigated as quickly as possible. ETA of the same is not know at the moment. We shall continue to share updates every hour, if not...

  4. investigating

    We are experiencing an outage due to some issue at the end of our public cloud provider. We are working closely with them to get this resolved or mitigated as quickly as possible. ETA of the same is not know at the moment. We shall continue to share updates every hour, if not...

  5. identified

    We understand that our public cloud provider has identified the cause of the issue. We are starting to see some recovery and is working towards mitigation. We appreciate your patience. We shall continue to share updates every hour, if not sooner.

  6. identified

    We continue to work with our public cloud provider towards mitigating the issue at the earliest. We appreciate your patience. We shall continue to share updates every hour, if not sooner.

  7. identified

    Atlassian team is actively engaged and continues to work with our public cloud provider to mitigate this issue at the earliest. We are starting to see partial operations succeed. We appreciate your patience. We shall continue to share updates every hour, if not sooner.

  8. identified

    Our public cloud provider is working to mitigate this issue quickly. We are seeing some early positive indicators and are continuing to monitor. We appreciate your patience and will continue to provide updates every hour or sooner.

  9. identified

    Our public cloud provider is working to mitigate this issue quickly. We are seeing some early positive indicators and are continuing to monitor. We appreciate your patience and will continue to provide updates every hour or sooner.

  10. identified

    Our public cloud provider is working to mitigate this issue quickly. We are seeing some early positive indicators and are continuing to monitor. We appreciate your patience and will continue to provide updates every hour or sooner.

  11. identified

    We understand your pain and mitigating or fixing this issue is of utmost importance. Our public cloud provider is actively working to mitigate this issue on priority. We have been seeing partial operational success. We appreciate your patience and will continue to provide...

  12. identified

    Update - We understand the impact this issue is having on your operations and want to assure you that resolving this matter is our highest priority. Our public cloud provider is actively working to mitigate this issue with urgency. While we do not have a definitive ETA at this...

  13. identified

    Update - We understand the impact this issue is having on your operations and want to assure you that resolving this matter is our highest priority. Our public cloud provider is actively working to mitigate this issue with urgency. While we do not have a definitive ETA at this...

  14. identified

    Update - Thank you for your continued patience. We understand the impact this issue is having on your operations and want to assure you that resolving this matter is our highest priority. Our public cloud provider is still actively working to mitigate this issue with urgency and...

  15. identified

    Update - Thank you for your continued patience. We understand the impact this issue is having on your operations and want to assure you that resolving this matter is our highest priority. Our public cloud provider is still actively working to mitigate this issue with urgency and...

  16. identified

    Update - We understand the impact this issue is having on your operations and want to assure you that resolving this matter is our highest priority. Our public cloud provider is actively working to mitigate this issue with urgency. While we do not have a definitive ETA at this...

  17. identified

    We are currently aware of an ongoing incident impacting Atlassian Cloud services due to an outage with our public cloud provider, AWS. We understand the impact this issue is having on your operations and want to assure you that resolving this matter is our highest priority and...

  18. monitoring

    There have been no changes since our last update. We will provide our next updated by 9:00PM UTC or sooner as new information becomes available. We are currently aware of an ongoing incident impacting Atlassian Cloud services due to an outage with our public cloud provider, AWS....

  19. monitoring

    Monitoring - We've started seeing continued product experience improvement. While we still have a backlog of event processing, we are seeing improvements in systems operational capabilities across all products. We estimate a significant improvement with the next few hours and...

  20. monitoring

    Our teams are continuing to monitor the recovery of systems across Atlassian products. This update is to inform that the Atlassian Support portal is fully operational at this time for customers that wish to contact support.

  21. monitoring

    Our team is now seeing recovery across all impacted Atlassian products. We are continuing to monitor for individual products that may still be processing backlogged items now that services are restored. The Atlassian Support portal is currently still displaying a message...

  22. monitoring

    We continue to see recovery progressing across all impacted products as backlogged items continue to be processed. The Atlassian Support portal is currently displaying a message directing customers to our temporary support channel. Please note that our support portal is...

  23. monitoring

    The issue relating to the Atlassian Support portal displaying a message to customers to use our temporary support channel has now been resolved. The Atlassian Support portal is fully functional for any ongoing support issues. With regards to other Atlassian products, we continue...

  24. resolved

    Our team is now able to see full recovery across the vast majority of Atlassian products. We are aware of some ongoing issues with specific components such as migrations and JSM virtual service agents, and our team is continuing to investigate with urgency. We apologise for the...

  25. postmortem

    ### Postmortem publish date: Nov 19th, 2025 ### Summary All dates and times below are in UTC unless stated otherwise. Customers utilizing Atlassian products experienced elevated error rates and degraded performance between Oct 20, 2025 06:48 and Oct 21, 2025 04:05. The service...

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Delays in Opsgenie, JSM Ops and Compass Ops notifications minor 2025-09-30 14:07 UTC 25m

Affected: Email Notification Delivery, SMS Notification Delivery, Voice Notification Delivery, Mobile Notification Delivery

Failure mode: Data delayed, stale, lost or corrupted, or needing replay after recovery. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    We are observing delays in Opsgenie, JSM ops and Compass ops notification flows. No alert has been lost and our team is actively working on it to mitigate the delays. We'll keep you posted with further updates.

  2. identified

    We continue to work on resolving the notification delays for Jira Service Management, Opsgenie, and Compass. We have identified the root cause and expect recovery shortly.

  3. monitoring

    We have identified the root cause of the notification delay for Opsgenie, JSM Ops and Compass Ops and have mitigated the problem. We are now monitoring closely.

  4. resolved

    Between 13:10 UTC to 14:00 UTC, we experienced notification delay for Jira Service Management, Opsgenie, and Compass in only EU-region. The issue has been resolved and the service is operating normally.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Delays observed at JSM, Opsgenie and Compass alert search functionality in US Region major 2025-09-10 15:23 UTC 27h 22m

Affected: Incident Flow, Incident Flow, Alert Flow, Alert Flow, Email Notification Delivery, Email Notification Delivery, Public Website, SMS Notification Delivery, SMS Notification Delivery, Voice Notification Delivery, Voice Notification Delivery, Mobile Notification Delivery, Mobile Notification Delivery, Heartbeat Monitoring, Heartbeat Monitoring, Incident REST API, Incident REST API, Alert REST API, Alert REST API, Heartbeat REST API, Heartbeat REST API, Incoming Email Service, Incoming Email Service, Incoming Integration Flow, Incoming Integration Flow, Outgoing Integration Flow, Outgoing Integration Flow, Signup, Login & Authorization, Signup, Login & Authorization, Opsgenie Actions, Opsgenie Actions, Web Application, Web Application, Mobile Application, Mobile Application, Configuration REST APIs, Configuration REST APIs, Reporting & Analytics, Reporting & Analytics, Pricing & Billing, Pricing & Billing, Logs, Logs, Incoming Call Routing, Incoming Call Routing

Failure mode: Exhausted memory, disk, CPU, queues, connection pools or quotas. Root cause: Not enough capacity for the load. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    We identified degraded performance in web experiences and Alert Rest API for some Jira Service Management and Opsgenie Cloud customers in the US Region. The team has taken actions to mitigate the issue and minimize the impact on search functionality.

  2. identified

    We continue to work on resolving the incident for Jira Service Management and Opsgenie. We have identified the root cause and taken actions to mitigate the issue and minimize the impact on search functionality.

  3. identified

    We are continuing to work on a fix for this issue.

  4. identified

    Our team is still actively working to resolve the issue and making progress toward a resolution. Thank you for your patience.

  5. identified

    Our team has been able to identify the root cause of these performance issues and have put a mitigation into place. We are now continuing to see recovery of Jira Service Management operations and Opsgenie services. Existing alerts are continuing to be processed at this time. We...

  6. identified

    We are continuing to see recovery of Jira Service Management operations and Opsgenie services. There are a large number of missed alerts to be processed due to the prior service issue and our team is actively investigating methods to help expedite this processing. Alerts should...

  7. identified

    While new alerts should now correctly notify users, the content of those alerts for impacted customers will remain empty until missed alerts are fully processed. Our team is continuing to investigate with urgency potential infrastructure options to expedite this process. We will...

  8. identified

    New alerts should still continue to notify users as they are generated, however the alert content will still be missing. Our teams are still actively engaging with infrastructure teams to try and expedite historical processing of messages to fully restore services. We will...

  9. identified

    We have received reports that some customers are experiencing further issues relating to loading pages within Opsgenie and Jira Service Management which may be related to the infrastructure efforts underway to restore services to these products. Please be aware we also have...

  10. identified

    Our team is continuing to investigate with the highest level of urgency in order to restore Jira Service Management and Opsgenie services. At this time we are prioritising infrastructure to try and restore new and active alerts to be populated with their alert content as soon as...

  11. identified

    Our infrastructure teams are working diligently to try and ensure all services are restored to Jira Service Management and Opsgenie as soon as possible. New alerts are continuing to notify users, however we are still prioritising infrastructure to restore the content inside...

  12. identified

    Our teams priority remains to restore full functionality to new alerts to users from Jira Service Management and Opsgenie. Escalation teams are fully engaged on this incident to try and ensure we can get to a resolution as soon as possible. At this time these alerts are still...

  13. identified

    All concerned teams are engaged and are working to restore full functionality to new alerts to users from Jira Service Management and Opsgenie. Escalation teams are also engaged on this incident to try and ensure we can get to a resolution as soon as possible. Currently, the...

  14. identified

    Atlassian teams remain engaged with priority to restore full functionality to new alerts to users from Jira Service Management, Opsgenie and Compass. Escalation teams are also engaged on this incident to ensure we can get to a resolution as soon as possible. At this point, the...

  15. identified

    Atlassian teams continue to remain engaged with priority to restore full functionality to new alerts to users from Jira Service Management, Opsgenie and Compass. Escalation teams also continue to remain engaged on this incident to ensure our focus is not diluted from a...

  16. identified

    Our teams have are identified some potential resolution steps for this issue and continue to test the same on priority to be able to restore full functionality to users from Jira Service Management, Opsgenie and Compass. Currently, the alerts continue to successfully notify...

  17. identified

    Atlassian teams continue to test the identified potential resolution steps for this issue. Our goal is to restore full functionality to users from Jira Service Management, Opsgenie and Compass. Currently, the alerts continue to successfully notify users, except that the...

  18. identified

    We are progressing well with the testing of identified potential mitigation steps for this issue; and we are seeing positive early results. We will update with an ETA and further information in the next update. Alerts continue to successfully notify users, in some cases the...

  19. identified

    We have progressed further in preparing our system with a probable fix. Initial testing has been successful. We are in the process of completing further testing and finalising the fix. We expect the fix to be completed in next few hours. Alerts continue to successfully notify...

  20. monitoring

    We have deployed the fix and all operations are back to normal. We will continue to monitor the operations. We will continue providing specific updates when available.

  21. resolved

    Between September 10, 2025, 3:19 PM UTC and September 11, 2025, 2:44 PM UTC, there was degraded performance in web experiences and the REST APIs for some Jira Service Management, Opsgenie, and Compass Cloud customers in the US region. We have deployed a fix to mitigate the issue...

  22. postmortem

    ### **SUMMARY** Between September 10, 2025, at 13:56 UTC and September 11, 2025, at 14:40 UTC, Jira Service Management, Compass Cloud Operations, and Opsgenie customers in the U.S. region experienced degraded performance across web and mobile applications, as well as Alert REST...

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Degraded performance in Opsgenie and JSM operations none 2025-06-19 15:21 UTC 1h 16m

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. investigating

    We are investigating degraded performance issues in Jira Service Management, Opsgenie, and Compass Cloud customers. We will provide more details within the next hour.

  2. monitoring

    A fix has been implemented and we are monitoring the results.

  3. resolved

    This incident has been resolved.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Degraded performance in Opsgenie and JSM operations none 2025-06-17 12:37 UTC 10m

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. investigating

    We are investigating cases of degraded performance for some Jira Service Management and Opsgenie Cloud customers. We will provide more details within the next hour.

  2. monitoring

    We have identified the root cause of the performance degraded service and have mitigated the problem. We are now monitoring closely.

  3. resolved

    This incident has been resolved, and Jira Service Management and Opsgenie are back in operational mode.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Delays observed at JSM and Opsgenie alert search functionality minor 2025-05-10 07:08 UTC 3h 43m

Affected: Web Application

Failure mode: A provider or third-party service this vendor depends on failed. Root cause: A failure at a provider or vendor they depend on. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

  1. identified

    We identified degraded performance at alert search functionality for some Jira Service Management and Opsgenie Cloud customers due to the infrastructure issue from cloud provider. No impact has been observed at alert critical flows like notification. The team has taken actions...

  2. monitoring

    The problem is mitigated, and we are now monitoring closely.

  3. resolved

    All search functionality is operational without any latency. Thank you for your patience.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Schedule API are getting timed out none 2025-04-04 10:05 UTC 47m

Failure mode: Exhausted memory, disk, CPU, queues, connection pools or quotas. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. investigating

    We are investigating cases of degraded performance for Alert Schedules experiencing timeouts and slowness for Opsgenie Cloud customers. Requests have been taking more than 30s and some have been timing out. We will provide more details within the next hour.

  2. resolved

    This incident has been resolved.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

EU OpsGenie API calls having intermittent routing issues major 2025-03-05 06:34 UTC 50m

Affected: Alert Flow

Failure mode: Network, DNS, routing or connectivity failure. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    We are aware of an issue where some API calls configured to use api.opsgenie.com/eu are intermittently returning a 404 error with the message 'no Route matched with those values'. API calls to api.eu.opsgenie.com and api.opsgenie.com (without /eu) are not affected. Our team is...

  2. resolved

    Issues where some API calls configured to use api.opsgenie.com/eu were intermittently returning a 404 error with the message 'no Route matched with those values' should now be resolved. API calls to api.eu.opsgenie.com and api.opsgenie.com (without /eu) were not affected at this...

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Services menu in Opsgenie is not responding none 2025-01-14 15:48 UTC 59m

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. investigating

    Our engineering team is actively investigating this incident and working to bring the Opsgenie service back up as quickly as possible. Users affected by this incident may notice that Services functionality is slow or completely unavailable for the web page We will update this...

  2. identified

    Our team has identified the issue with the Services page in Opsgenie and is working to fix it.

  3. resolved

    Our engineers have been closely monitoring the platform and are declaring this incident resolved. Thank you for your patience.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Elevated 5XX errors in Schedule API at Opsgenie USA region minor 2024-12-16 15:30 UTC 0m

Failure mode: A code release introduced the failure. Root cause: A bug in the vendor's own code. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

  1. resolved

    Our team has identified the issue in Schedule API between 15:30 UTC and 17:00 UTC. We saw performance degradation and 5XX errors in response. Faulty deployment has been reverted quickly in the USA region and rapid recovery is seen. We are monitoring the system for a full...

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Opsgenie Web UI is slow or unavailable in US region minor 2024-11-06 19:47 UTC 12m

Affected: Web Application

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. investigating

    We've noticed that Opsgenie Web UI is responding slowly or unavailable in US region. Our engineering team is actively investigating this incident and working to bring Opsgenie back up to speed as quickly as possible. We'll keep you posted with further updates on this page.

  2. monitoring

    Our engineering team has implemented fixes. We will continue to monitor all systems. Thank you for your patience.

  3. resolved

    Our engineers have been closely monitoring the platform and are declaring this incident resolved. Thank you for your patience.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Reported issues with OEC functionality major 2024-09-24 19:55 UTC 59m

Affected: Outgoing Integration Flow, Outgoing Integration Flow

Failure mode: A configuration change caused the failure. Root cause: A configuration change. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

  1. investigating

    We've been notified that a number of OEC clients have been failing to create Jira tickets. We're currently investigating the issue.

  2. investigating

    We are continuing to investigate this issue.

  3. identified

    We've identified a recent change that has broken some endpoint routing configurations and caused OEC endpoint requests to be directed to wrong service. We're reverting that change on production at the moment.

  4. monitoring

    We reverted the faulty routing configuration change and started getting traffic on OEC endpoints

  5. resolved

    We've verified that OEC endpoints are back online.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Users are experiencing reCaptcha errors while signing up critical 2024-09-12 02:08 UTC 1h 21m

Affected: Signup, Login & Authorization

Failure mode: Login, tokens, certificates or credentials failed. Root cause: A failure at a provider or vendor they depend on. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. investigating

    Users attempting to sign up are encountering reCaptcha errors that are preventing a successful signup.

  2. monitoring

    We have identified the root cause and the issue appears to be resolved.

  3. resolved

    This issue has been resolved.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Unable to edit Opsgenie rotations in US, EU and Sydney regions in Web Application minor 2024-08-28 15:01 UTC 16m

Affected: Web Application, Web Application

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. identified

    Our team has identified the issue with Opsgenie Web Application / Edit Rotation feature in US, EU and Sydney regions and is working to fix it. Check back soon for another update! Our team is working hard to get the feature up and running again.

  2. identified

    We are continuing to work on a fix for this issue.

  3. resolved

    Our engineers have been closely monitoring the platform and are declaring this incident resolved. Thank you for your patience.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Some products are hard down major 2024-07-03 20:51 UTC 3h 45m

Affected: Incident Flow, Incident Flow, Alert Flow, Alert Flow, Email Notification Delivery, Email Notification Delivery, Public Website, SMS Notification Delivery, SMS Notification Delivery, Voice Notification Delivery, Voice Notification Delivery, Mobile Notification Delivery, Mobile Notification Delivery

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. investigating

    We are investigating an issue with that is impacting Atlassian, Atlassian Partners, Atlassian Support, Confluence, Jira Work Management, Jira Service Management, Jira, Opsgenie, Atlassian Developer, Atlassian (deprecated), Trello, Atlassian Bitbucket, Guard, Jira Align, Jira...

  2. monitoring

    We have mitigated the problem and continue looking into the root cause. The outage was between 8:08pm 03/07 UTC - 08:31pm 03/07 UTC We are now monitoring closely.

  3. resolved

    Between 03-07-2024 20:08 UTC to 03-07-2024 20:31 UTC, we experienced downtime for Opsgenie. The issue has been resolved and the service is operating normally.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Intermittent error accessing content none 2024-06-21 00:17 UTC 2h 30m

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. investigating

    We are investigating an intermittent issue with accessing Atlassian Cloud services that is impacting some Atlassian Cloud customers. We will provide more details once we identify the root cause.

  2. monitoring

    We have identified the root cause of the intermittent errors and have mitigated the problem. We are now monitoring closely.

  3. resolved

    Between 2024-06-20 22:04 UTC to 2024-06-20 22:28 UTC, we experienced intermittent issue for users to access the services for some Atlassian Cloud customers. The issue has been resolved and the service is operating normally.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Error responses across multiple Cloud products major 2024-06-03 23:11 UTC 1h 21m

Affected: Signup, Login & Authorization, Signup, Login & Authorization, Web Application, Web Application, Mobile Application, Mobile Application

Failure mode: A configuration change caused the failure. Root cause: A configuration change. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

  1. identified

    We are investigating an issue with error responses for some Cloud customers across multiple products. We have identified the root cause and expect recovery shortly.

  2. resolved

    Between 22:18 UTC to 22:56 UTC, we experienced errors for multiple Cloud products. The issue has been resolved and the service is operating normally.

  3. postmortem

    ### Summary On June 3rd, between 09:43pm and 10:58 pm UTC, Atlassian customers using multiple product\(s\) were unable to access their services. The event was triggered by a change to the infrastructure API Gateway, which is responsible for routing the traffic to the correct...

Excerpts are shortened. Read the full incident on Opsgenie's status page.

US - Increased delays on alert flow minor 2024-04-30 12:52 UTC 3h 21m

Affected: Alert Flow, Heartbeat Monitoring, Alert REST API, Web Application, Mobile Application, Incoming Call Routing

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. identified

    We are observing delays for our alert flow. No alert has been lost and our team is actively working on it to mitigate the delays. We'll keep you posted with further updates.

  2. identified

    We are continuing to work on a fix for this issue.

  3. monitoring

    Our engineering team has implemented fixes. We will continue to monitor all systems. Thank you for your patience.

  4. resolved

    This incident has been resolved.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Admin Portal Feature Access Issue none 2024-04-03 09:52 UTC 31m

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. identified

    We are investigating an issue causing failures in accessing some features from the Admin Portal, which is impacting some of our Cloud customers. We have identified the root cause and anticipate recovery shortly.

  2. resolved

    Between 6:30 AM UTC to 9:50 AM UTC, we experienced failures in accessing some features from the Admin Portal. The issue has been resolved and the service is operating normally.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Investigating new product purchasing none 2024-02-29 01:27 UTC 1h 25m

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. investigating

    We are investigating an issue with new product purchasing that is impacting for all products. Customers adding new cloud products may have experienced a long waiting page or an error page after attempting to add a product. We have mitigated the root cause and are working to...

  2. resolved

    Between 28th Feb 2024 23:15 UTC to 29th Feb 2024 00:05 UTC, we experienced issue with new product purchasing for all products. All new sign up products have been successfully provision and confirmed issue has been resolved and the service is operating normally.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Opsgenie SAML login at eu region is not working major 2024-02-27 14:25 UTC 34m

Affected: Signup, Login & Authorization

Failure mode: Login, tokens, certificates or credentials failed. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    Opsgenie SAML Login functionality is not working at only EU region due to signature verification error at login certificates. Already logged in customers haven't been affected by the error.

  2. monitoring

    The team has identified the issue causing the signature validation error at EU SAML Login. The fix has been started to deploy and SAML login activities are being monitored by team currently

  3. resolved

    We fixed the problem and verified that there are no more login issues.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Service Disruptions Affecting Atlassian Products none 2024-02-14 21:57 UTC 1h 35m

Affected: Incident Flow, Incident Flow, Alert Flow, Alert Flow, Email Notification Delivery, Email Notification Delivery, Public Website, SMS Notification Delivery, SMS Notification Delivery, Voice Notification Delivery, Voice Notification Delivery, Mobile Notification Delivery, Heartbeat Monitoring, Heartbeat Monitoring, Incident REST API, Incident REST API, Alert REST API, Alert REST API, Heartbeat REST API, Heartbeat REST API, Incoming Email Service, Incoming Email Service, Incoming Integration Flow, Incoming Integration Flow, Outgoing Integration Flow, Outgoing Integration Flow, Signup, Login & Authorization, Signup, Login & Authorization, Opsgenie Actions, Opsgenie Actions, Web Application, Web Application, Mobile Application, Mobile Application, Configuration REST APIs, Configuration REST APIs, Reporting & Analytics, Reporting & Analytics, Pricing & Billing, Pricing & Billing, Logs, Logs, Incoming Call Routing

Failure mode: A configuration change caused the failure. Root cause: A human mistake during operations. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

  1. investigating

    We are investigating reports of intermittent errors for all Cloud Customers across all Atlassian products. We will provide more details once we identify the root cause.

  2. identified

    We have identified the root cause of the increased errors and have mitigated the problem. We continue to work on resolving the issue and monitoring this closely.

  3. monitoring

    We have identified the root cause of the Service Disruptions affecting all Atlassian products and have mitigated the problem. We are now monitoring this closely.

  4. resolved

    We experienced increased errors on Confluence, Jira Work Management, Jira Service Management, Jira Software, Opsgenie, Trello, Atlassian Bitbucket, Atlassian Access, Jira Align, Jira Product Discovery, Atlas, Compass, and Atlassian Analytics. The issue has been resolved and the...

  5. postmortem

    ### **Summary** On February 14, 2024, between 20:05 UTC and 23:03 UTC, Atlassian customers on the following cloud products encountered a service disruption: Access, Atlas, Atlassian Analytics, Bitbucket, Compass, Confluence, Ecosystem apps, Jira Service Management, Jira...

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Major outages in heartbeat services pinging via email and email integration in EU region major 2024-01-29 15:08 UTC 21m

Affected: Heartbeat Monitoring, Incoming Email Service

Failure mode: A configuration change caused the failure. Root cause: A configuration change. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

  1. identified

    Due to faulty domain configurations, heartbeats updates via email and incoming email services have been rejected starting from 13:45 UTC. The team has been working on the fix. Except for email updates and email integrations, heartbeat feature and integrations are still fully...

  2. monitoring

    The team has reverted the changes and identified that corresponding heartbeat and email integration start working now

  3. resolved

    This incident has been resolved.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

IOS based Alert Notifications delivered as non-critical major 2023-12-13 13:18 UTC 1h 19m

Affected: Mobile Notification Delivery, Mobile Notification Delivery

Failure mode: A code release introduced the failure. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. investigating

    As of 12.12.2023 14:46 UTC, we have observed that some alert notifications are not being delivered to IOS-based devices as critical. We are currently investigating the issue.

  2. identified

    The issue has been identified and a fix is being implemented.

  3. monitoring

    A fix has been implemented and we are monitoring the results.

  4. resolved

    This incident has been resolved.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

We observe increased error rates due to the cloud provider minor 2023-12-10 03:40 UTC 11m

Affected: Incident Flow, Alert Flow, Email Notification Delivery, SMS Notification Delivery, Voice Notification Delivery, Mobile Notification Delivery, Heartbeat Monitoring, Incident REST API, Alert REST API, Heartbeat REST API, Incoming Email Service, Incoming Integration Flow, Outgoing Integration Flow, Opsgenie Actions, Web Application, Mobile Application, Configuration REST APIs, Pricing & Billing

Failure mode: A provider or third-party service this vendor depends on failed. Root cause: A failure at a provider or vendor they depend on. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

  1. monitoring

    We observe increased error rates due to the cloud provider. It caused notification delays and elevated API errors for our customers. All systems are recovering now, we are monitoring.

  2. resolved

    Our engineers have been closely monitoring the platform and are declaring this incident resolved. Thank you for your patience.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Egress connectivity timing out none 2023-12-04 16:33 UTC 42m

Failure mode: Network, DNS, routing or connectivity failure. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

Names an upstream provider: AWS.

  1. identified

    We are currently investigating an incident that result in connection time outs on service egress proxy. This affects Jira, JSM, Confluence, BitBucket, Trello, Ecosystem products. The features affected for these products are those that require a connection to service egress.

  2. identified

    Including Atlassian Developer

  3. identified

    We are currently investigating an incident that result in outbound connections from Atlassian cloud in us-east-1 intermittently timing out. This affects Jira, Trello, Confluence, Ecosystem products. The features affected for these products are those that require opening a...

  4. monitoring

    The issue was identified and a fix implemented. We are monitoring currently.

  5. resolved

    The systems are stable after the fix and monitoring for a specified duration

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Scheduled report functionality is disabled minor 2023-11-18 11:41 UTC 6d 2h

Affected: Reporting & Analytics, Reporting & Analytics

Failure mode: Security incident or emergency response to a vulnerability. Root cause: An external attack. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    Scheduled report functionality is disabled as we suspect a possible spam activity. Only the reports with a custom schedules are disabled, periodic emails are not impacted. The team is investigating the issue and will provide more update.

  2. identified

    The cause of the issue is identified, and the team is working on the fix.

  3. identified

    The root cause is identified and the spam activity is mitigated. The team is working on adding more controls to prevent further spam activities. The scheduled report feature will be kept disabled for a while until the further controls implemented. However, the reporting service...

  4. identified

    We have implemented additional controls and are introducing domain restrictions as well as limitations to the number of recipients for mitigation purposes. Scheduled reports will be enabled for all customers on November 24th, PST. Users will begin receiving emails for the...

  5. monitoring

    The changes have been shipped to production, and scheduled reports are enabled now.

  6. resolved

    We're excited to inform you that we've shipped upgrades to our production environment, enabling scheduled reports once again. What's Changed: To continuously improve and ensure the security of our services, we've implemented additional controls including domain restrictions and...

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Atlassian Account login issues major 2023-09-13 14:08 UTC 5h 23m

Affected: Public Website, Signup, Login & Authorization, Signup, Login & Authorization

Failure mode: Login, tokens, certificates or credentials failed. Root cause: A configuration change. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

  1. investigating

    We are investigating an issue with Atlassian Accounts login that is impacting some Cloud customers. We will provide more details within the next hour.

  2. monitoring

    We have identified the root cause of the Atlassian Accounts login issues impacting Cloud Customers and have mitigated the problem. We are now monitoring this closely.

  3. monitoring

    We can see a reduction in the Atlassian Accounts login issues after the mitigation actions were taken. We are still monitoring closely and will continue to provide updates.

  4. monitoring

    We are no longer seeing occurrences of the Atlassian Accounts login errors, all clients should be able to successfully login now. We will continue to monitor.

  5. resolved

    Between 12:45 UTC to 15:30 UTC, we experienced login and signup issues for Atlassian Accounts. The issue has been resolved and the service is operating normally. We will publish a post-incident review with the details of the incident and the actions we are taking to prevent...

  6. postmortem

    ### **SUMMARY** On Sep 13, 2023, between 12:00 PM UTC and 03: 30 PM UTC, some Atlassian users were unable to sign in to their accounts and use multiple Atlassian cloud products. The event was triggered by a misconfiguration of rate limits in an internal service which caused a...

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Opsgenie has experienced delay on Android Notifications minor 2023-09-10 18:37 UTC 40m

Affected: Mobile Notification Delivery

Failure mode: Data delayed, stale, lost or corrupted, or needing replay after recovery. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. identified

    We are seeing delays with Android notifications. We have identified the cause and are currently working on mitigation of this issue

  2. monitoring

    No delay on Android Notification is experienced now. All Android Notification delay has return to normal.

  3. resolved

    This incident has been resolved.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Multiple product logins none 2023-08-30 05:19 UTC 58m

Failure mode: CDN or edge layer failed or served bad content. Root cause: A bug in the vendor's own code. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

  1. investigating

    We are investigating reports of intermittent errors for login to Atlassian Support, Confluence, Jira Work Management, Jira Service Management, Jira Software, Opsgenie, Trello, Jira Product Discovery, Compass, and Atlassian Analytics Cloud customers. We will provide more details...

  2. resolved

    Between 4:30AM UTC to 6:00AM UTC, we experienced issues for users attempting to login for Atlassian Support, Confluence, Jira Work Management, Jira Service Management, Jira Software, Opsgenie, Trello, Jira Product Discovery, Compass, and Atlassian Analytics. The issue has been...

  3. postmortem

    ### **SUMMARY** On August 30, 2023, between 4:07 and 5:30 UTC, some customers were unable to login to Atlassian's Cloud products using [id.atlassian.com](http://id.atlassian.com). Logged-in users were also unable to switch accounts, change passwords, or log out. Users with...

Excerpts are shortened. Read the full incident on Opsgenie's status page.

We observe degraded performance on incident timeline functionality minor 2023-08-28 17:57 UTC 1h 32m

Affected: Incident Flow

Failure mode: A configuration change caused the failure. Root cause: A configuration change. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

  1. investigating

    We observe degraded performance on incident timeline functionality. We are investigating the issue and we will provide more details within the next hour.

  2. identified

    Some misconfiguration caused the incident. We identified the root cause and reverting the change.

  3. monitoring

    The rollback completed successfully and we observed the remediation. The functionality is fully working now, and we are closely monitoring the system.

  4. resolved

    The issue has been resolved completely and all the functionality is fully working now. The customers experienced some latencies for the entries added to the incident timeline during the incident. However, there is not data loss and all the messages processed successfully.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Sign-ups, Product Activation, and Billing not working none 2023-08-02 13:41 UTC 2h 18m

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. investigating

    We are investigating an issue with Sign-ups, Product Activation, and Billing that is impacting all of our Cloud Customers. We will provide more details within the next hour.

  2. monitoring

    We have identified the root cause of the Sign-ups, Product Activation, and Billing not working and have mitigated the problem. We are now monitoring closely.

  3. resolved

    We mitigated the issue with Sign-ups, Product Activation, and Billing, and the systems are back to BAU, and all functionality is restored.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Performance issues and outages with Cloud products none 2023-07-06 11:18 UTC 4h 18m

Failure mode: A code release introduced the failure. Root cause: A failure at a provider or vendor they depend on. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. identified

    We are investigating an issue with an internal infrastructure component that is impacting multiple Cloud products, including Jira Software, Bitbucket, Jira Service Management and Confluence, and customers. These issues include performance impact and, in some cases, outages....

  2. monitoring

    We have identified the root cause of an issue with an internal infrastructure component that has been impacting multiple Cloud products - including Jira Software, Jira Service Management and Confluence - and customers. This issue had lead to a performance impact and, in some...

  3. resolved

    We experienced performance issues and outages for several Atlassian Cloud Products. The issue has been resolved and the service is operating normally.

  4. postmortem

    ### **SUMMARY** We understand the importance of providing reliable and consistent service to our valued customers. On July 6, 2023, from 03:52 to 15:11 UTC, we experienced an issue with an upgraded version of a third-party tool that functions as our internal artifact management...

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Intermittent errors during login for some customers major 2023-07-06 09:03 UTC 4h 29m

Affected: Signup, Login & Authorization, Signup, Login & Authorization

Failure mode: Login, tokens, certificates or credentials failed. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    We are investigating reports of errors during login for some customers that is impacting some Confluence, Jira Work Management, Jira Service Management, Jira Software, Opsgenie, Trello, Atlassian Bitbucket, Atlassian Access, Jira Product Discovery, and Atlassian Analytics Cloud...

  2. investigating

    We are investigating reports of errors during login for some customers that is impacting some Atlassian Support, Confluence, Jira Work Management, Jira Service Management, Jira Software, Opsgenie, Trello, Atlassian Bitbucket, Atlassian Access, Jira Product Discovery, and...

  3. investigating

    We are investigating reports of errors during login for some customers that is impacting some Atlassian Support, Confluence, Jira Work Management, Jira Service Management, Jira Software, Opsgenie, Trello, Atlassian Bitbucket, Atlassian Access, Jira Product Discovery, and...

  4. identified

    We are investigating reports of errors during login that is impacting some Atlassian Support, Confluence, Jira Work Management, Jira Service Management, Jira Software, Opsgenie, Trello, Atlassian Bitbucket, Atlassian Access, Jira Product Discovery, Compass, and Atlassian...

  5. monitoring

    We have identified the root cause of the errors during login and have mitigated the problem. We are now monitoring closely.

  6. resolved

    Between 07:31 UTC to 12:32 UTC, we experienced errors during login for Atlassian Support, Confluence, Jira Work Management, Jira Service Management, Jira Software, Opsgenie, Trello, Atlassian Bitbucket, Atlassian Access, Jira Product Discovery, Compass, and Atlassian Analytics....

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Increased delays on all flows in EU region minor 2023-06-26 06:25 UTC 6h 11m

Affected: Alert Flow, Email Notification Delivery, SMS Notification Delivery, Voice Notification Delivery, Mobile Notification Delivery, Heartbeat Monitoring, Incoming Integration Flow, Outgoing Integration Flow, Signup, Login & Authorization, Web Application

Failure mode: Exhausted memory, disk, CPU, queues, connection pools or quotas. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    Our platform is experiencing some delays for all system in EU region. Team is actively working on it to mitigate the delays. We'll keep you posted with further updates.

  2. investigating

    The team has identified the cause of the problem and actively working on it to mitigate the delays.

  3. identified

    The issue has been identified and a fix is being implemented.

  4. identified

    We are continuing on a fix for the issue.

  5. identified

    We detected delays in login flow. We identified the problem and working on fix.

  6. identified

    Due to extreme load, we are experiencing problem in cache layer. Our team is actively addressing the problem and working on implementing a fix as quickly as possible. We appreciate your patience.

  7. monitoring

    Our engineering team has implemented fixes. We are monitoring all systems. Thank you for your patience.

  8. resolved

    Our engineers have been closely monitoring the platform and are declaring this incident resolved. Thank you for your patience.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Partial Outage in Schedule API major 2023-06-06 17:06 UTC 19m

Affected: Configuration REST APIs

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. investigating

    We are investigating an issue with Schedule API that is impacting some Opsgenie US Cloud customers. We will provide more details within the next hour.

  2. resolved

    Our engineering team has fixed the problem and we are closely monitoring the platform. Thank you for your patience.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Partial outage in ICC sessions due to network problem minor 2023-04-06 14:45 UTC 23m

Affected: Incident Flow, Incident Flow

Failure mode: Network, DNS, routing or connectivity failure. Root cause: A configuration change. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

  1. identified

    The problem is related to a misconfiguration on our network causing ICC sessions to fail. We are working on a fix and adjusting our network. Incident flow is continue to work without a problem and there is no data loss. We appreciate your patience as our teams continue with our...

  2. resolved

    Our engineers have been closely monitoring the platform and are declaring this incident resolved. Thank you for your patience.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Delays in Android push notifications major 2023-04-04 14:30 UTC 3h 46m

Affected: Mobile Notification Delivery, Mobile Notification Delivery

Failure mode: A provider or third-party service this vendor depends on failed. Root cause: A failure at a provider or vendor they depend on. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

  1. investigating

    We are investigating an issue with our Android push notifications that is impacting some of our notifications for Android. We will provide more details within the next hour.

  2. monitoring

    The issue has been identified as caused by an error on Firebase (https://status.firebase.google.com/incidents/9ZPv9faHLen8bzLVSaft). We continue to monitor the situation and send update within the next hour.

  3. monitoring

    Android push notification delivery is fully operational for now, but we are still monitoring the Firebase outage (https://status.firebase.google.com/incidents/9ZPv9faHLen8bzLVSaft).

  4. monitoring

    We are continuing to monitor for any further issues.

  5. resolved

    This incident on Firebase has been resolved. Android push notifications are operational.

  6. postmortem

    ### **SUMMARY** On April 4, 2023, between 13:32 and 14:50 UTC, Atlassian customers using Opsgenie faced significant delays while receiving Android push notifications. This was caused by an incident in a third party messaging service, which is responsible for Android push...

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Increased delays on Jira Cloud and Jira Service Management Cloud integrations while creating/updating Opsgenie alerts in US region none 2023-04-03 17:34 UTC 0m

Affected: Incoming Integration Flow

Failure mode: Exhausted memory, disk, CPU, queues, connection pools or quotas. Root cause: A configuration change. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. resolved

    We observed some delays while creating/updating alerts from Jira Cloud and Jira Service Management Cloud integrations in US region. The problem is resolved now.

  2. postmortem

    ### **SUMMARY** On April 3, 2023, from 1:15 pm UTC to 5:20 pm UTC Atlassian customers using Opsgenie product to integrate with a separate Jira Service Management Cloud instance faced significant delays while creating and updating alerts from Jira Cloud and Jira Service...

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Opsgenie On Call analytics dashboards are not showing data minor 2023-02-27 08:49 UTC 5d 9h

Affected: Reporting & Analytics

Failure mode: Data delayed, stale, lost or corrupted, or needing replay after recovery. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    Opsgenie "On Call Reports", "On Call Time Analytics" & "Total On Call Time per User" Analytics dashboards are not showing data. We are working to identify the root cause and we'll keep you posted with further updates.

  2. investigating

    We have identified the root cause and rolled out a fix for the last month of data. All data generated in the last month should be accessible. We have triggered the fix for the rest of the data and will update once the data is fully accessible.

  3. investigating

    We are continuing to investigate this issue.

  4. identified

    The issue has been identified and a fix is being implemented.

  5. identified

    Job triggered to fix rest of the data is taking more time then expected, we are actively monitoring it and will update once we have more information.

  6. identified

    Data is fixed for last six months and impacted dashboards can be generated with last six month of data. We will continue to work on fixing rest of the data and will update once we have more information.

  7. identified

    While we continue to fix the data, we have observed latency in data shown for a few of the other analytics dashboards. We will continue to work to fix the rest of the data and will update you once we have more information.

  8. identified

    While we continue to increase the data accessibility for the 3 impacted dashboards, customers can continue to use them for the last 6 months of data. Rest of the reporting dashboards are working as expected. We will keep you posted on further updates.

  9. identified

    The team is still working to increase the data accessibility for the 3 impacted dashboards. We will keep you posted on further updates.

  10. resolved

    For the 3 impacted dashboards, we have made data accessible for upto 1 year. For date ranges longer than 1 year, please reach out to the customer support. We will continue to monitor the system.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

"_incomingData" and "_actionSource" fields are missing in Opsgenie Debug logs minor 2023-01-13 11:52 UTC 3d 18h

Affected: Logs, Logs

Failure mode: Data delayed, stale, lost or corrupted, or needing replay after recovery. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    We have identified that "_incomingData" and "_actionSource" fields are missing in Opsgenie Debug Logs due to internal issues.

  2. identified

    We have identified the problem and worked on the solution to fix debug logs now.

  3. monitoring

    "_incomingData" and "_actionSource" fields are visible in debug logs generated after January 13, 2023 1:55:39 PM UTC

  4. resolved

    This incident has been resolved.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Observing delays in incoming webhook integration processing in EU region major 2022-12-02 19:09 UTC 42m

Affected: Incoming Integration Flow

Failure mode: Data delayed, stale, lost or corrupted, or needing replay after recovery. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. investigating

    We are currently investigating this issue.

  2. monitoring

    A fix has been implemented and we are monitoring the results.

  3. resolved

    This incident has been resolved.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Opsgenie analytics dashboards was not accessible none 2022-11-29 20:30 UTC 0m

Failure mode: Traffic surge or denial-of-service attack. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. resolved

    Opsgenie analytics dashboards were not available between 2022-11-29 18:15 (UTC) and 2022-11-29 19:30 (UTC). We observed a spike in traffic patterns which caused degradation in one of our services. After detecting the problem, our engineering team worked towards getting the...

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Android users of Jira, Confluence and Opsgenie app with Compromised device check feature turned on is getting locked out of their app major 2022-11-11 21:02 UTC 2h 59m

Affected: Mobile Application

Failure mode: Login, tokens, certificates or credentials failed. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. identified

    We are investigating reports of intermittent errors for Confluence, Jira Software, and Opsgenie Cloud customers. We will provide more details once we identify the root cause.

  2. identified

    We are investigating an issue where Android users of Jira, Confluence and Opsgenie app with the Compromised device check feature turned on is getting locked out of their app. Note that this is only affecting the Android mobile app for customers who have turned on the Compromised...

  3. resolved

    Between 2022-11-06 and 2022-11-11, 18:45 EST, we experienced an issue where Android users of Jira, Confluence, and Opsgenie apps with the Compromised device check feature turned on is getting locked out of their apps for Confluence, Jira Software, and Opsgenie. The issue has...

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Logs are delayed on search and download API in EU region minor 2022-11-04 14:22 UTC 1h 6m

Affected: Logs

Failure mode: Data delayed, stale, lost or corrupted, or needing replay after recovery. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with medium confidence (taxonomy v0.2).

  1. investigating

    Logs are delayed on the log search page and the download API in EU Region. We are currently investigating the issue.

  2. investigating

    We are continuing to investigate this issue.

  3. investigating

    We are continuing to investigate this issue.

  4. identified

    We have identified the root cause of the problem . Our engineers are working on fixing the problem now.

  5. monitoring

    A fix has been implemented and the delay of the logs are decreasing rapidly. We are monitoring the progress. Alert logs and all other functionality is working as expected.

  6. resolved

    The incident has been resolved. Logs are available without any delay.

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Schedule overrides were not editable/deletable none 2022-10-20 14:00 UTC 0m

Failure mode: Orchestration, control plane, service discovery or load balancing failed. Root cause: The vendor does not say why it happened. Tagged by claude-haiku-5-5 with low confidence (taxonomy v0.2).

  1. resolved

    Between 2022-10-20 14:23 (UTC) and 2022-10-20 17:39 (UTC), due to latency in Opsgenie system, we were not allowing any edit or delete for schedule overrides and returning error to some of our users. After detecting the problem, our engineering team worked towards getting...

Excerpts are shortened. Read the full incident on Opsgenie's status page.

Elevated error rate in all Opsgenie services none 2022-10-20 09:00 UTC 24h

Failure mode: A code release introduced the failure. Root cause: A bug in the vendor's own code. Tagged by claude-haiku-5-5 with high confidence (taxonomy v0.2).

  1. resolved

    Between 2022-10-21 09:01 UTC and 2022-10-21 09:12 UTC, in the US region, we started to see elevated error rate in our infrastructure due to faulty deployment. We have deployed a fix to mitigate the issue and have verified that all services have recovered without data loss....

Excerpts are shortened. Read the full incident on Opsgenie's status page.