Working with advanced rate limiting rules

Advanced rate limiting rules put a cap on how often an individual client can send requests that meet set conditions before all or some requests from that same client are blocked or logged.

Limitations and considerations

When working with advanced rate limiting rules, keep the following things in mind:

• Advanced rate limiting is only included with the Premier platform and certain packaged offerings. It is not included as part of the Professional or Essential platforms.
• Each site (also known as workspace) is limited to a maximum of 15 rate limit rules.
• A given signal can only be used as the threshold signal for a single rate limit rule. A signal can't be used as the threshold signal in more than one rate limit rule.

If you have an Edge WAF deployment, keep these additional considerations in mind:

• The faster requests are received the quicker they can be blocked.
• Rules with a lower threshold block or log requests sooner than rules with a higher threshold, subject to the rate the requests are received.
• You may need up to two times the number of requests over any 60 second window to declare a match.
• When testing rules in multiple physical locations, your results may vary due to the distributed nature of Fastly’s network and the different number of cache nodes per POP.
• For the most consistent results, test with rates of 100 RPS or more.

How advanced rate limiting rules work

When the web application you're protecting starts receiving requests that meet the conditions of a rate limit rule, the requests are tagged with the threshold signal. Threshold signals are tallied and counted towards the threshold for that rule. When the threshold signal count for a single client exceeds the rule's threshold, that client is rate limited.

How the client is rate limited depends on the Match type and Action type fields for that rate limit rule. The Match type field defines which requests from the client should be blocked or logged after the threshold has been passed. Field options include:

• Rule conditions: rate limit requests from the client that match the rate limit rule's conditions. See an example use case of this option.
• Other signal: rate limit requests from the client that are tagged with the action signal. When the action signal is not an attack or anomaly signal, you need a request rule that tags requests with the selected signal. See an example use case of this option.
• All requests: rate limit all requests from the client.

The Action type field defines whether requests that meet the Match type condition should be blocked or logged. When the Action type field is set to Block, subsequent requests that meet the Match type are blocked and tagged with these signals:

• the threshold signal defined in the rule
• Rate Limit system signal
• Blocked Request system signal

When the Action type field is set to Log, subsequent requests that meet the Match type are logged and tagged with these signals:

• the threshold signal defined in the rule
• Rate Limit system signal

Threshold counting for Edge WAF deployments

Edge WAF deployments operate on Fastly's Edge Cloud platform, which is a distributed global network. This mode of deployment uses both local and global counting mechanisms to track the number of threshold signals per client and to determine when a client exceeds the threshold of an advanced rate limit rule.

Located at the Fastly POP-level, the local counting mechanism tracks threshold signals over a 60 second window during the interval set by the advanced rate limit rule, either 1 minute or 10 minutes. The total threshold signal count from each window is added to the next window for the duration of the interval. If the count from the last completed window causes the total threshold signal count to exceed the threshold of the advanced rate limit rule, the client is rate limited. As the local counting mechanism is distributed across a number of cache nodes in a single POP, there may be a slight synchronization time delay.

Located in our cloud engine, the global counting mechanism aggregates the threshold signal count from our POP network. The cloud engine then forwards the aggregated count to the Edge WAF deployment, which takes approximately one to two minutes.

Data storage

We store signal data based on the storage category of the signal.

Creating advanced rate limiting rules

To create an advanced rate limiting rule, start by navigating to the Add form for rules and selecting a type of rule:

1. Log in to the Next-Gen WAF control panel.
2. From the Sites menu, select a site if you have more than one site.

3. From the Rules menu, select Site Rules.

4. Click Add site rule.

   

5. In the Type section, select Rate limit.

Next, define the logic that the rule should use to identify requests that count towards the threshold. In the Conditions section:

1. Fill out these fields to create a condition:
   • From the Field menu, select the request field that the condition is based on.
   • From the Operator menu, select an operator to specify how the selected field and value relate.
   • In the Value field, enter a value for the specified field.
2. (Optional) Click Add condition to add another condition or Add group to create a group of conditions.
3. Decide whether a request must meet one or all conditions in order to count towards the threshold:
   • Select Any from the conditions menu to specify that a request must meet only one of the conditions you've created.
   • Leave All selected in the conditions menu to specify that a request must meet every condition you've created.

Once you've done that, specify how the rate limit rule should identify an individual client. In the Client identifier section:

1. From the Client key menu, select how the rate limit rule should identify a client. Depending on the Client key option you selected, additional client identifier fields may appear.
2. (Optional) Fill out any additional fields that appeared in the Client identifier section.

Next, specify a signal that should be applied to requests that meet the rule's condition set and define the threshold. In the Actions section, fill out the Tracking subsection as follows:

1. From the Threshold signal menu, select the signal that you want applied to requests that match the rule conditions. Requests tagged with the threshold signal are tallied and counted towards the threshold of the rule.
2. In the Threshold field, enter the number of requests that must be detected before a client is rate limited.
3. From the Interval menu, select the period of time requests must be detected during to pass the threshold.

Next, define how a client that exceeds the threshold should be rate limited. In the Actions section, fill out the Rate limiting subsection as follows:

1. From the Action type menu, select the action that should be taken when the threshold has been exceed and requests meet the conditions of the match type. Action types include Block and Allow. When you select Block, the Change response link appears.
2. (Optional) Click Change response to specify a custom response code to return when the rule blocks a request and fill out the related fields as follows:
   • In the Response code (optional) field, enter a custom response code. Supported custom response codes are 301, 302, and 400-599.
   • If you entered 301 or 302 in the Response code (optional) field then, in the Redirect URL (optional) field, enter the absolute or relative URL of the redirect location. See Using redirect custom response codes.
3. From the Match type menu, select the conditions that determine what should be rate limited once the threshold is exceeded. Options include:
   • Rule conditions: rate limit requests from the client that match the rule's conditions. See an example use case of this option.
   • Other signal: rate limit requests from the client that are tagged with the selected Action signal. When the action signal is not an attack or anomaly signal, you need a request rule that tags requests with the selected signal. See an example use case of this option.
   • All requests: rate limit all requests from the client.
4. From the Duration menu, select the amount of time the client should be rate limited.

Finally, add a description of the rule and save the rule:

1. Fill out the Details section as follows:
   • Leave the Status switch enabled.
   • In the Description field, enter a description of the rule.
2. Click Create site rule. The advanced rate limit rule is created, and the Site Rules page appears.

Example rate limit rules

The following examples demonstrate how to configure rate limit rules for common use-cases. Be aware that values (e.g., paths and response codes) used in these examples may not be the same as those used by your particular web application.

Rate limit comment submissions

This example rule demonstrates how to rate limit the ability to submit comments:

Specifically, the rule looks for POST requests to the /comments.php file and tags them with the Comment Submission custom signal (the Threshold signal). Because someone may attempt to change their IP address to circumvent the rate limit, the rule uses both the IP address and the value of the User-Agent request header (the Client identifier) to track requests from them.

When 10 requests (the Threshold) tagged with the Comment Submission signal are detected from a unique IP address and User-Agent within 1 minute (the Interval), any subsequent requests with the Comment Submission signal from that IP address and User-Agent will be blocked (the Action type) for the next 15 minutes (the Duration).

Credit card validation attempts

This example demonstrates how to rate limit credit card validation attempts after too many failed attempts. This use-case requires two separate rules:

• a request rule to track credit card validation attempts.
• a rate limit rule to track credit card validation failures and rate limit the originating IP address.

The request rule looks for POST requests to the /checkout-payment.php file and tags them with the Credit Card Attempt custom signal.

The rate limit rule looks for requests tagged with the Credit Card Attempt custom signal, as well as if the request received a 401 response code indicating the credit card validation attempt was a failure. The rule applies a Credit Card Failure custom signal (the Threshold signal) to these requests.

When 5 requests (the Threshold) tagged with the Credit Card Failure signal are detected from a signal IP within 10 minutes (the Interval), any subsequent requests tagged with the Credit Card Attempt signal (the Action signal) from that IP will be blocked (the Action type) for the next 5 minutes (the Duration).

Glossary