An Alert may have one or more conditions that will be verified, and if all or any of these match an established threshold, a notification is triggered.
To set up an Alert, define:
- Source: Metric and tags;
- Transformation: mean, sum, count, max or min;
- Condition: below/above or any/no data received;
- Threshold: value; and
- Duration: time window.
An Alert condition can be read as: “The mean value of the metric
- Conditions are evaluated as a group of one-dimensional logical operations.
- Conditions are based on a simplified query - group by or baseline queries are not supported.
The process for the analysis of triggering conditions to activate or deactivate an Alert (ON or OFF) is based on the concept of a rolling temporal window of Metric data coming into Statful. We provide below a detailed explanation of how this is achieved for Transformation and Any/No Data conditions.
- Received data points are stored in buckets on a per Alert basis.
- Data points are pulled from buckets and processed on a rolling window of real-time (based on the Alert configuration trigger duration) in which every X seconds all data points received during the period are processed and a comparison with the specified threshold is performed.
- On each processing iteration, data points that fall outside of the defined window of time are removed from the bucket and not considered for processing purposes.
- A processing iteration is only performed if there are at least two data points that represent an interval higher than the defined duration in order to guarantee that a minimum data sample is received before processing.
- When receiving a data point, the current timestamp is stored in a bucket on a per Alert basis.
- It follows the same rolling window of time approach as the transformation conditions in which the timestamp of the last data point received is checked and compared with the current timestamp - validating if any or no data was received as per defined condition.
- A processing iteration begins the moment the Alert is configured.
Alert Events are generated by user actions or automatically by data processing. They are persisted and associated with the respective Alert to provide a historical view of the Alert.
Alerts may take one of the following States:
- OFF - The Alert is CLOSED or OFF, ready for activation when trigger conditions are verified.
- ON - The Alert is OPEN or ON, activated once trigger conditions were verified.
- ACK - The Alert has been acknowledged by a User and can be considered silent for any future notifications.
- MUTE - The Alert is temporarily disabled - the User can trigger this status manually or by seting up a specific date and time.
Depending on the current status of the Alert, an Event may or may not generate an Alert notification.
Situations that trigger a user notification are:
- OFF to ON -The Alert was OPEN after receiving an ON Event.
- ON to OFF - The Alert was CLOSED after receiving an OFF Event.
- ON to ACK -The Alert was ACKNOWLEDGE after receiving an ACK Event.
- ACK to OFF - The Alert was CLOSED after receiving an OFF Event.
- ACK to ON - The previous acknowledgment of the Alert was canceled after receiving a UNACK Event that resets the status to ON.