Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[draft] fix: running workflows for suppressed alerts #4188

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

Matvey-Kuk
Copy link
Contributor

@Matvey-Kuk Matvey-Kuk commented Mar 23, 2025

It's pretty confusing when we run Workflows for suppressed alerts: #4005

I don't like idea of just not allowing to run workflows for suppressed alerts at all, so I allowed to run workflows for them if suppressed status is mentioned in the filter.

close #4005
close #3858

Copy link

vercel bot commented Mar 23, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
keep ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 27, 2025 5:23pm

@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Mar 23, 2025
@Matvey-Kuk Matvey-Kuk marked this pull request as draft March 23, 2025 20:08
Copy link
Contributor

github-actions bot commented Mar 23, 2025

Hey there and thank you for opening this pull request! 👋🏼

We require pull request titles to follow the Conventional Commits specification and it looks like your proposed title needs to be adjusted.

Details:

No release type found in pull request title "[draft] fix: running workflows for suppressed alerts". Add a prefix to indicate what kind of release this pull request corresponds to. For reference, see https://www.conventionalcommits.org/

Available types:
 - feat: A new feature
 - fix: A bug fix
 - docs: Documentation only changes
 - style: Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc)
 - refactor: A code change that neither fixes a bug nor adds a feature
 - perf: A code change that improves performance
 - test: Adding missing tests or correcting existing tests
 - build: Changes that affect the build system or external dependencies (example scopes: gulp, broccoli, npm)
 - ci: Changes to our CI configuration files and scripts (example scopes: Travis, Circle, BrowserStack, SauceLabs)
 - chore: Other changes that don't modify src or test files
 - revert: Reverts a previous commit

@dosubot dosubot bot added the Bug Something isn't working label Mar 23, 2025
@@ -183,6 +183,19 @@ def insert_events(self, tenant_id, events: typing.List[AlertDto | IncidentDto]):
if not trigger.get("type") == "alert":
self.logger.debug("trigger type is not alert, skipping")
continue
# Skipping suppressed alerts unless it's explicitly mentioned in the filter
if event.status == "suppressed":
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AlertStatus.SUPPRESSED

if trigger.get("filters", []) == [
{"key": "status", "value": "suppressed"}
]:
self.logger.debug(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be info

@Matvey-Kuk
Copy link
Contributor Author

@shahargl in general, how do you think if we should avoid running workflows for suppressed alerts? I wonder if we shouldn't run them by default also for resolved if it's not mentioned in the filter.

Copy link

codecov bot commented Mar 23, 2025

Codecov Report

Attention: Patch coverage is 20.00000% with 4 lines in your changes missing coverage. Please review.

Project coverage is 46.74%. Comparing base (b1d0c31) to head (0b9a7ae).
Report is 23 commits behind head on main.

Files with missing lines Patch % Lines
keep/workflowmanager/workflowmanager.py 20.00% 4 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #4188       +/-   ##
===========================================
+ Coverage   30.95%   46.74%   +15.78%     
===========================================
  Files          92      164       +72     
  Lines       10257    16538     +6281     
===========================================
+ Hits         3175     7730     +4555     
- Misses       7082     8808     +1726     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@shahargl
Copy link
Member

@Matvey-Kuk idk - is there any context for this? did someone ask for it? because if not I would probably keep things simple

@Matvey-Kuk
Copy link
Contributor Author

@shahargl it's based on #4005

@talboren
Copy link
Member

@shahargl it's based on #4005

@shahargl in general, how do you think if we should avoid running workflows for suppressed alerts? I wonder if we shouldn't run them by default also for resolved if it's not mentioned in the filter.

i think that in general it should be configurable, but I would take the approach where you can specify which statuses you ignore, instead of marking those you want to execute for.
So workflows run for every event, based on filters, and you can exclude certain statuses (it's already possible, btw, by filtering out)

  triggers:
    - type: alert
      filters:
        - key: status
          value: r"(firing|acknowledged)"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working size:L This PR changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[🐛 Bug]: muted alert called workflow [🐛 Bug]: AI summary doing nothings
3 participants