Event Inhibit processor — automatically identify and suppress secondary alerts so on-call engineers can focus on the most important issues.

Overview

Alert inhibition (Inhibit) is a noise-reduction mechanism. It automatically identifies and suppresses related secondary alerts so you can focus on the most important issues. When the system experiences a severe failure, many related alerts are typically triggered; the alert-inhibit feature helps reduce unnecessary noise and improve incident-handling efficiency.

Core Value

1. Reduce alert storms

When a core service fails, dependent downstream services also produce many alerts. Alert inhibition can automatically suppress derivative secondary alerts based on configuration.

2. Focus on key issues

By suppressing low-priority alerts, operations staff can focus on high-priority core issues without being drowned in alert noise.

Use Cases

Case 1: Inhibit by severity

Problem: a service triggers P1, P2, and P3 alerts at the same time, but you only want to be notified for the most severe P1 alert.

Example configuration:

  • New-alert condition: severity >= 2 (P2 or P3)
  • Active-alert condition: severity = 1 (P1)
  • Same-item match: labels service, host
  • Effect: when a P1 alert exists, P2 and P3 alerts for the same service and host are inhibited

Case 2: Service-dependency-chain inhibition

Problem: a database failure leads to multiple application-service alerts; you only want the database alert and want application-layer alerts inhibited.

Example configuration:

  • New-alert condition: service_type = "application"
  • Active-alert condition: service_type = "database"
  • Same-item match: labels cluster, env
  • Effect: when an alert exists in the database layer, application-layer alerts in the same cluster and environment are inhibited

Configuration Steps

Step 1: Enter the event pipeline configuration

  1. Log in to the monitoring platform
  2. Navigate to Notification management > Edit notification rule
  3. Click Event pipeline settings or use an existing pipeline Event pipeline settings

Step 2: Add an inhibit processor

  1. In the processor management list, click New
  2. Select the processor type Inhibit (alert inhibition) UwDceG

Step 3: Configure inhibit conditions

3.1 Set new-alert filter conditions

Define which new alerts should be evaluated for inhibition.

Configuration items:

  • Label condition: filter based on alert labels
    • Example: severity="warning" or priority>=2
  • Attribute condition: filter based on alert attributes
    • Example: business_group="ecommerce platform"

k0AdZW

3.2 Set active-alert filter conditions

Define what kind of active alerts will trigger the inhibition.

Configuration items:

  • Time window: how many seconds within which a matching active alert must exist (default 300s)
  • Label condition: label conditions the active alert must satisfy
    • Example: severity="critical" or priority=1
  • Attribute condition: attribute conditions the active alert must satisfy

Yc6csi

3.3 Configure match rules

Define what labels or attributes must be the same between the new alert and the active alert.

Key settings:

  • Same labels: label keys that must match exactly
    • Common: service, host, cluster, env, region
  • Same attributes: attributes that must match exactly
    • Common: business group, datasource

RDEec1

Step 4: Test the configuration

  1. Click the Test button at the bottom-right of the processor card
  2. Pick or enter a test alert event
  3. Review the result to confirm the inhibit logic works as expected

Step 5: Save and enable

  1. After verifying all settings, click Save
  2. Ensure the event pipeline is enabled
  3. The configuration takes effect immediately

Advanced Tips

1. Combine multiple processors

You can chain multiple processors in the same event pipeline, for example:

  • First, use Relabel to normalize labels
  • Then, use Inhibit for alert suppression
  • Finally, use AI Summary to generate alert summaries

2. Layered inhibition strategy

For complex architectures, create layered inhibit strategies:

  • Infrastructure layer: network/storage alerts inhibit host alerts
  • Platform layer: database/middleware alerts inhibit application alerts
  • Application layer: core service alerts inhibit edge service alerts

FAQ

Q1: An alert was inhibited by mistake — what should I do?

A: Check the following:

  1. Whether the inhibit conditions are too broad
  2. Whether label matching is correct
  3. Whether the time window is reasonable
  4. Use the Test feature to validate the configuration

Q2: How can I see inhibited alerts?

A: Inhibited alerts appear in the notification records of the event detail. You can:

  1. Open the corresponding alert event detail page
  2. Click the notification records below to check whether it was inhibited

Q3: How is the priority of inhibit rules determined?

A: When multiple inhibit processors are configured:

  1. They execute in the order they appear in the pipeline
  2. Once an alert is inhibited, subsequent inhibit processors no longer process it
  3. You can adjust the order by dragging processors

References

快猫星云 联系方式 快猫星云 联系方式
快猫星云 联系方式
快猫星云 联系方式
快猫星云 联系方式
快猫星云