Working with alerts

When an observed metric either exceeds or falls below a threshold, the Alerts feature displays a notification in the Fastly application and sends messages to your custom integrations (e.g., Slack channels, email addresses, and webhooks). From the Alerts page, you can view triggered and resolved alert instances and manage your alert definitions and integrations.

Before you begin

The Alerts feature is disabled by default and can be purchased by contacting sales@fastly.com. Once enabled, any user with the appropriate roles and permissions will be able to manage alerts for Fastly services and historic stats metrics.

Limitations and considerations

The maximum number of alert definitions that can exist at any one time per customer is 50.

How alerts work

An alert definition is a configuration rule that defines when an alert should be triggered and how notifications should be handled. Specifically, it outlines:

  • the observed service, data pipeline, and metric.
  • the set of conditions that must be met for an alert to trigger.
  • the custom destinations for alert messages (e.g., Slack channels, email addresses, and webhooks).

The condition set defines the threshold, the evaluation period, and whether the alert should trigger when the metric exceeds or remains below the threshold or rate. Specifically:

  • A threshold is a static number or a percentage of all requests that the observed metric must either be above or below for the alert to trigger.
  • The evaluation period is the number of minutes during which the observed metric is evaluated every minute to determine if the metric has gone above or remained below the specified threshold.

Alerting when a metric exceeds a static threshold

The Alerts feature supports alerting above a threshold. When creating an alert definition, you can define an above threshold based condition by setting the Trigger an alert when the metric is menu to Above. This alert will fire when the observed metric goes above the specified threshold during an evaluation period at any one minute polling interval. The alert will resolve when there are zero one minute intervals in the evaluation period that exceed the threshold.

Above a static threshold alert example

Let's say you want to be notified in your #website-alerts Slack channel when the number of 5XX responses exceed 1,000. When configuring the alert definition, you set:

  • the Data source menu to Historic Stats
  • the Metric menu to status_5xx
  • the Trigger an alert when the metric is: menu to Above
  • the Threshold field to 1000
  • the Evaluation period menu to 5 minutes
  • the Integration(s) menu to website-alerts

After saving the alert definition, the system starts evaluating the observed metric in 5 minute rolling time windows (the Evaluation period). During a polling minute in one of those evaluation periods, the observed metric goes above 1,000 (the Threshold):

Evaluation period minuteNumber of 5XX responsesThreshold exceeded
1988No
2765No
31021Yes
4843No
51000No

At the end of the fifth minute of that evaluation period, an alert is triggered because the observed metric went above the threshold during the third minute of the evaluation period. Notifications are also sent. Specifically, the Fastly application indicates that an alert has been triggered via the Alert bell icon and a message is sent to the #website-alerts Slack channel.

That alert then resolves at the end of this evaluation period because there are zero one minute intervals in the evaluation period that exceed the threshold:

Evaluation period minuteNumber of 5XX responsesThreshold exceeded
1576No
2674No
3516No
4577No
5732No

Alerting when a metric is below a static threshold

The Alerts feature supports alerting below a threshold. When creating an alert definition, you can define a below threshold based condition by setting the Trigger an alert when the metric is menu to Below. This alert will fire when the observed metric remains below the specified threshold for two consecutive minutes in an evaluation period. The alert will resolve when there aren't two consecutive minutes in an evaluation period that are below the threshold. We look for two consecutive minutes in an evaluation period because network latency occasionally delays metric aggregation across POPs, and we want to ensure accuracy and limit false positives.

Below a static threshold alert example

Let's say you want to be notified in your #website-alerts Slack channel when the number of 2XX responses is below 500. When configuring the alert definition, you set:

  • the Data source menu to Historic Stats
  • the Metric menu to status_2xx
  • the Trigger an alert when the metric is: menu to Below
  • the Threshold field to 500
  • the Evaluation period menu to 5 minutes
  • the Integration(s) menu to website-alerts

After saving the alert definition, the system starts evaluating the observed metric in 5 minute rolling time windows (the Evaluation period). During two consecutive polling minutes in one of those evaluation periods, the observed metric is below 500 (the Threshold):

Evaluation period minuteNumber of 2XX responsesBelow the threshold
11098No
21254No
3372Yes
4415Yes
5500No

At the end of the fifth minute of that evaluation period, an alert is triggered because the observed metric was below the threshold during the third and forth minutes of the evaluation period. Notifications are also sent. Specifically, the Fastly application indicates that an alert has been triggered via the Alert bell icon and a message is sent to the #website-alerts Slack channel.

That alert then resolves at the end of this evaluation period because there aren't two consecutive minutes in the evaluation period that are below the threshold:

Evaluation period minuteNumber of 2XX responsesBelow the threshold
11108No
2285Yes
31284No
41065No
5396Yes

Alerting when a metric exceeds a rate-based threshold

The Alerts feature supports alerting above a rate of all requests. When creating an alert definition, you can define an above rate-based condition by selecting an error rate or success rate metric to observe and setting the Trigger an alert when the metric is menu to Above. This alert will fire when the observed metric goes above the specified percentage of overall requests during an evaluation period at any one minute polling interval. The alert will resolve when there are zero one minute intervals in the evaluation period that exceed the threshold.

Above a rate-based threshold alert example

Let's say you want to be notified in your #website-alerts Slack channel when the number of 4XX responses exceeds 5% of all responses sent by your site. When configuring the alert definition, you set:

  • the Data source menu to Historic Stats
  • the Metric menu to 4XX Error Rate
  • the Trigger an alert when the metric is: menu to Above
  • the Threshold menu to 5%
  • the Evaluation period menu to 5 minutes
  • the Integration(s) menu to website-alerts

After saving the alert definition, the system starts evaluating the observed metric in 5 minute rolling time windows (the Evaluation period). During a polling minute in one of those evaluation periods, the observed metric goes above 5% (the Threshold):

Evaluation period minuteNumber of 4XX responsesTotal number of requestsPercentage of overall requestsThreshold exceeded
109880%No
2137651.67%No
36410216.27%Yes
4148431.66%No
5310000.3%No

At the end of the fifth minute of that evaluation period, an alert is triggered because the observed metric went above the threshold during the third minute of the evaluation period. Notifications are also sent. Specifically, the Fastly application indicates that an alert has been triggered via the Alert bell icon and a message is sent to the #website-alerts Slack channel.

That alert then resolves at the end of this evaluation period because there are zero one minute intervals in the evaluation period that exceed the threshold:

Evaluation period minuteNumber of 4XX responsesTotal number of requestsPercentage of overall requestsThreshold exceeded
1195763.30%No
2306744.45%No
3145162.71%No
405770%No
507320%No

Alerting when a metric is below a rate-based threshold

The Alerts feature supports alerting below a rate of all requests. When creating an alert definition, you can define a below rate-based condition by selecting an error rate or success rate metric to observe and setting the Trigger an alert when the metric is menu to Below. This alert will fire when the observed metric goes below the specified percentage of overall requests during an evaluation period at any one minute polling interval. The alert will resolve when there are zero one minute intervals in the evaluation period that exceed the threshold.

Below a rate-based threshold alert example

Let's say you want to be notified in your #website-alerts Slack channel when the Success Rate, that is, the percentage of responses with non-5XX codes, falls below 80%. When configuring the alert definition, you set:

  • the Data source menu to Historic Stats
  • the Metric menu to Success Rate
  • the Trigger an alert when the metric is: menu to Below
  • the Threshold menu to 80%
  • the Evaluation period menu to 5 minutes
  • the Integration(s) menu to website-alerts

After saving the alert definition, the system starts evaluating the observed metric in 5 minute rolling time windows (the Evaluation period). During a polling minute in one of those evaluation periods, the observed metric goes below 80% (the Threshold):

Evaluation period minuteSuccess RateBelow the threshold
199%No
279%Yes
375%Yes
495%No
590%No

At the end of the fifth minute of that evaluation period, an alert is triggered because the observed metric was below the threshold during two consecutive minutes in the evaluation period. Notifications are also sent. Specifically, the Fastly application indicates that an alert has been triggered via the Alert bell icon and a message is sent to the #website-alerts Slack channel.

That alert then resolves at the end of this evaluation period because there aren't two consecutive minutes in the evaluation period that are below the threshold:

Evaluation period minuteSuccess RateBelow the threshold
190%No
295%No
399%No
499%No
589%No

Viewing alerts

To view details for an alert that is firing or resolved, follow these steps:

  1. Log in to the Fastly web interface.
  2. Click Observability.
  3. Click Alerts.
  4. Click on an alert instance that is firing or resolved. The details page for that alert appears.

To view a list of your alert definitions, follow these steps:

  1. Log in to the Fastly web interface.
  2. Click Observability.
  3. Click Alerts.
  4. Click Alert definitions. The Alert definitions page appears.

Managing alert definitions

An alert definition is a configuration rule that defines when an alert should be triggered and how notifications should be handled.

Creating alert definitions

To create an alert definition, start by navigating to the Add alert definition page using one of these methods:

  • On the Alerts page in the Firing section, click Add alert definition.
  • From the Alerts page, click Alert definitions and then Add alert definition.

the Add alert definition page

Next, fill out the general alert fields as follows:

  1. In the Name field, enter the name of the alert.
  2. In the Description field, enter a description of the alert definition.
  3. From the Service menu, select the service that you want to monitor.
  4. From the Data Source menu, select a data pipeline. Data pipeline options include Historic Stats, Origin Inspector, and Domain Inspector.

Next, fill out the fields that are specific to the data source that you selected:

  1. If you selected Historic Stats from the Data source menu then, from the Metric menu, select the metric that you want to monitor.
  2. If you selected Domain Inspector from the Data source menu, fill out the following fields:
    • From the Domains menu, select the domains that you want to monitor.
    • From the Metric menu, select the metric that you want to monitor.
  3. If you selected Origin Inspector from the Data source menu, fill out the following fields:
    • From the Origins menu, select the origins that you want to monitor.
    • From the Metric menu, select the metric that you want to monitor.

Once you've done that, define the conditions that will trigger the alert:

If you selected an error rate or success rate alert type:

  1. From the Trigger an alert when the metric is: menu, select whether the alert should be triggered when the metric is above or below a percentage of overall requests.
  2. From the Threshold menu, select a percentage of overall requests that the metric must be above or below for the alert to trigger.
  3. From the Evaluation period menu, select the number of minutes during which the observed metric is evaluated every minute to determine if the metric is above or below the specified threshold. Options include 5 minutes, 15 minutes, and 30 minutes.

For all other alert types:

  1. From the Trigger an alert when the metric is: menu, select whether the alert should be triggered when the metric is above or below the threshold.
  2. In the Threshold field, enter the number that the metric must be above or below for the alert to trigger. To set this field, you can also click and drag the threshold line on the graph.
  3. From the Evaluation period menu, select the number of minutes during which the observed metric is evaluated every minute to determine if the metric is above or below the specified threshold. Options include 5 minutes, 15 minutes, and 30 minutes.

Next, optionally specify where alert notifications should be sent:

  1. (Optional) From the Integrations(s) menu, select the notification channels for the alert.
  2. (Optional) Click Test Alert. A test notification is sent to the notification channel you selected.

Finally, click Save Alert to save the alert. The Alert definitions page appears.

Editing alert definitions

To edit an alert definition, start by navigating to the Edit alert definition page using one of these methods:

  • On the Alerts page in the Firing section, click an instance of an alert definition that you want to edit and then click Edit alert definition.
  • From the Alerts page, click Alert definitions and then click the pencil to the right of the alert definition that you want to edit.

Next, update and save the alert definition:

  1. Fill out the Edit alert definition fields as follows:
    • In the Name field, enter the name of the alert.
    • In the Description field, enter a description of the alert definition.
    • If the value of the Data source menu is Domain Inspector then, from the Domains menu, select the domains that you want to monitor.
    • If the value of the Data source menu is Origin Inspector then, from the Origins menu, select the origins that you want to monitor.
    • From the Metric menu, select the metric that you want to monitor.
    • From the Trigger an alert when the metric is: menu, select whether the alert should be triggered when the metric is above or below the threshold.
    • In the Threshold field, enter the number that must be above or below for the alert to trigger. To set this field, you can also click and drag the threshold line on the graph.
    • From the Evaluation period menu, select the number of minutes during which the observed metric is evaluated every minute to determine if the metric is above or below the specified threshold. Options include 5 minutes, 15 minutes, and 30 minutes.
    • From the Integration(s) menu, optionally select or remove notification channels for the alert.
  2. (Optional) Click Test Alert. A test notification is sent to the notification channel you selected.
  3. Click Save Alert to save the alert.

Deleting alert definitions

To delete an alert definition, follow these steps:

  1. From the Alerts page, click Alert definitions.
  2. Click the trash to the right of the alert definition that you want to delete. The alert definition is removed from the Alert definitions page.

Managing alert integrations

Alert integrations are external channels that receive notifications when alerts are triggered. When creating an alert definition, you specify the alert integrations that should be used. Multiple alert definitions can use the same alert integration.

Creating alert integrations

To create an alert integration, follow these steps:

the Add integration page

  1. Slack
  2. PagerDuty
  3. Microsoft Teams
  4. New Relic
  1. In Slack, add an incoming webhook and copy the URL for the webhook.
  2. Log in to the Fastly web interface.
  3. Click Observability and then Alerts.
  4. Click Alert integrations and then Add alert integration.
  5. Fill out the Add integration controls as follows:
    • Click Slack.
    • In the Name field, enter the name of the integration.
    • In the Webhook URL field, enter the URL of the webhook you created in the first step.
    • In the Description field, enter a description of the notification channel.
  6. Click Save integration. The integration is added to the Alert integrations page.

Deleting alert integrations

To delete an alert integration, follow these steps:

  1. From the Alerts page, click Alert integrations.
  2. Click the trash to the right of the alert integration that you want to delete.
  3. Click Confirm and delete. The alert integration is removed from the Alert integrations page.
Was this guide helpful?

Do not use this form to send sensitive information. If you need assistance, contact support. This form is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.