Cloud Insights floods customer with alert notifications after maintenance event
- Views:
- 30
- Visibility:
- Public
- Votes:
- 0
- Category:
- data-infrastructure-insights
- Specialty:
- bluexp_insights
- Last Updated:
- 11/4/2024, 1:28:04 AM
Applies to
- Cloud Insights (CI)
- Pager Duty
Issue
- Customer performs CVO upgrades to all clusters across multiple regions and suspends Cloud Insight (CI) alert notification during maintenance
- After completion of the maintenance, alert notifications were resumed which cause CI to flood the customer with over 400 "Resolved" alerts
- Since the alerts are configured in CI to send to a 3rd party ticket system, over 400 false alarm tickets are created for alerts triggered during the maintenance event that have already resolved themselves