search close

Rules

access_time Updated Sep 23, 2022

The rules feature allows you to block, allow, and tag requests and exclude system signals for arbitrary sets of conditions. Rules can be created on individual sites (site rules) as well as the corp as a whole (corp rules) to be easily used in multiple sites.

Corp rules can be managed by going to Corp Rules > Corp Rules.

Site rules can be managed by going to Rules > Site Rules.

Rules precedence

When conflicting rules are created, the corp rules take precedence over the site rules. However, allow rules take precedence over block rules, regardless of whether they were created at the corp or site level.

Request rules

Request rules allow you to define arbitrary conditions and either block, allow, or tag requests indefinitely or for a specific period of time. To create a request rule, follow these steps:

  1. On the Site Rules page, click the Add site rule button. The Add form appears.

    A request rule designed to block requests to the '/login' page from the IP address '198.51.100.50', as described above.

  2. In the Type section, select Request.

  3. Fill out the fields in the Conditions section as follows:

    • From the Field menu, select the request field that the condition is based on.
    • In the Value field, enter a value for the specified field.
    • From the Operator menu, select an operator to specify how the selected field and value relate.
    • Optionally click the Add condition button to add another condition or the Add group button to create a group of conditions.
    • Select All to specify that a request must meet every condition or Any to specify that a request must meet only one condition.
  4. Fill out the fields in the Actions section as follows:

    • From the Action type menu, select the action that should be taken when a request meets the rule’s conditions. Action types include Block, Allow, and Add signal.
    • Optionally click the Change response link to specify a response code. The default response code is 406.
    • Optionally click the Add action button to add another action.
  5. Fill out the fields in the Details section as follows:

    • From the Request logging menu, select Sampled to store the logs for requests that match the rule’s criteria and None to not store the logs. When you select None, the time series graphs will still include data from requests that match the rule’s criteria. This field is only available for block and allow actions. See Data Storage and Sampling for more information.
    • Leave the Status switch enabled.
    • Click the Change expiration link and select from the menu when the rule should be disabled.
    • In the Description field, enter a description of the rule.
  6. Click the Create site rule button. The request rule is created, and the Site Rules page appears.

Request fields

Field Type Properties
Agent name String Text or wildcard
Country Enum ISO countries
Domain String Text or wildcard
IP address IP Text or wildcard, supports CIDR notation
Method Enum GET, POST, PUT, PATCH, DELETE, HEAD, TRACE
Path String Text or wildcard
POST parameter Multiple Name (string), Value (string)
Query parameter Multiple Name (string), Value (string)
Request cookie Multiple Name (string), Value (string)
Request header Multiple Name (string), Value (string), Value (IP)
Response code String Text or wildcard
Response header Multiple Name (string), Value (string)
Scheme Enum http, https
Signal Multiple Type (signal), Parameter name (string), Parameter value (string)
User agent String Text or wildcard

Signal exclusion rules

Signal exclusion rules allow you to define arbitrary conditions to exclude a specific system signal.

The below example signal exclusion rule prevents POST requests originating from a list of known internal IP addresses from being tagged with the NO-CONTENT-TYPE signal.

  • The “Signal” is set to No Content Type.
  • The first condition has Method selected for the “Field”, Equals selected for the “Operator”, and POST selected for the “Value”.
  • The second condition has IP address selected for the “Field”, Is in list selected for the “Operator”, and the Developer IPs (IP) list selected for the “Value”.

A signal exclusion rule designed to prevent POST requests originating from a list of known internal developer IP addresses from being tagged with the 'No Content Type' signal, as described above.

Signal exclusion fields

Signal exclusion rules have the same fields as request rules as well as additional fields specific to the particular signal that’s being excluded.

Field Type Properties
Parameter name String Text or wildcard
Parameter value String Text or wildcard

Rate limit rules

See Rate Limit Rules for information about using rate limit rules.

Templated rules

See Templated Rules for information about using templated rules.

Converting Requests to Rules

Field Type Properties
Agent name String Text or wildcard
Country Enum ISO countries
Domain String Text or wildcard
IP Address IP Text or wildcard, supports CIDR notation
Method Enum GET, POST, PUT, PATCH, DELETE, HEAD, TRACE, PROPFIND, OPTIONS, CONNECT
Path String Text or wildcard
POST parameter Multiple Name (string), Value (string)
Query parameter Multiple Name (string), Value (string)
Request cookie Multiple Name (string), Value (string)
Request header Multiple Name (string), Value (string), Value (IP)
Response code String Text or wildcard
Response Header Multiple Name (string), Value (string)
Scheme Enum http, https
Signal Multiple Type (signal), Parameter name (string), Parameter value (string)
User Agent String Text or wildcard

Individual requests in the Requests page can be converted into pre-populated rules, enabling you to easily allow, block, and tag similar requests.

How to convert a request to a rule

  1. Log in to the Signal Sciences console.

  2. Select a site if you have more than one site.

  3. Click Requests. The requests page appears.

  4. Locate or search for the request you want to convert into a rule.

  5. Click View request detail. The request details page appears.

  6. Click Convert to rule in the upper-right corner. The rule builder menu page appears.

  7. Under Type, select the type of rule you want to make (Request, Rate limit, or Signal exclusion).

  8. Under Conditions, select which characteristics of the request you want to convert into rule conditions. For example, selecting “IP Address” and “Path” will create conditions in the rule that look for the specific IP address and path featured in the request.

  9. Click Continue. A pre-built rule with conditions featuring the request characteristics you selected

  10. Under Conditions, modify the rule as needed by adding and editing rule conditions.

  11. Under Actions, select which action(s) the rule should take (e.g., Block, Allow, or Add signal). Additional actions can be added by clicking Add action.

  12. Under Status, optionally disable the rule by deselecting the Always enabled toggle. By default, rules are automatically enabled when created unless specifically disabled.

    • You can optionally set the rule to automatically disable after a set period of time. Click Change expiration and select a duration from the menu.
  13. In the Description field, enter a description for the rule.

  14. Click Create site rule.

Operators

When creating rules, operators are used to specify the logic of your rule when matching conditions. For example, the “equals” operator is used to check if a value in the request matches the value in the rule condition exactly—for example, to match a specific IP address or path.

Operator Function Example Match
Equals Checks if the request value matches the rule condition value exactly 203.0.113.169 Equals 203.0.113.169
Does not equal Checks if the request value does not match the rule condition value exactly 203.0.113.169 Does not equal 192.0.2.191
Contains Checks if the rule condition value being checked is contained within the request value; for example, to check if a substring is found within a larger string thisisanexamplestring Contains example
Does not contain Checks if the rule condition value being checked is not contained within the request value; for example, to check if a substring is not found within a larger string thisisanexamplestring Does not contain elephant
Like (wildcard) Allows the use of wildcard characters in matching; checks if the request value matches the rule condition value bats Like (wildcard) [bcr]ats
Not like (wildcard) Allows the use of wildcard characters in matching; checks if the request value does not match the rule condition value bats Not like (wildcard) [hps]ats
Matches (regexp) Allows the use of regular expressions in matching; checks if the request value matches the rule condition value bats Matches (regexp) (b|c|r)ats
Does not match (regexp) Allows the use of regular expressions in matching; checks if the request value does not match the rule condition value bats Does not match (regexp) (h|p|s)ats
Is in list Checks if the request value matches any of the values in a specific list 203.0.113.169 Is in list “Known IP Addresses”
Is not in list Checks if the request value does not match any of the values in a specific list 192.0.2.191 Is not in list “Known IP Addresses”

Wildcards

The “Like (wildcard)” operator supports 0-or-many wildcards (*), single-character wildcards (?), character-lists ([abc]), character-ranges ([a-c], [0-9]), alternatives ({cat,bat,[fr]at}), and exclusions ([!abc], [!0-9]).

If you need to match a literal *, ?, [, or ] character, escape them with the \ character. For example: \*.

The “Like (wildcard)” operator requires a full string match. If you’re trying to match part of a string, you may need to include the * wildcard at the beginning or end to include the rest of the string for correct matching.

Regular expressions are not supported with the “Like (wildcard)” operator. If you want to use regular expressions, you must use the “Matches (regexp)” operator.

Case sensitivity

All fields in rules are case sensitive with the exception of header names.

For example, if you create a rule that looks for a header named X-Custom-Header, it will match on requests with headers named X-Custom-Header and x-custom-header because header names aren’t case sensitive. However, if the rule looks for the value Example-Value, it will only match on Example-Value and not example-value because all other rule fields—such as header values in this example—are case sensitive.

Note: When constructing a regular expression pattern (regexp) that matches on a header name, POST parameter name, or query parameter name, write the pattern in all lowercase or prefix the pattern with a case insensitive regex matching mode of (i?).

Path syntax best practices

Always use leading slashes

For a URL like https://example.com/some-path, the correct path syntax to use would be /some-path.

Use relative paths instead of absolute URLs

For example, if the absolute URL to the login page on your site is https://example.com/login, then /login is the correct path syntax to enter when configuring your login signals.

Take care when using trailing characters in your paths

Since our path syntax uses exact matching, trailing characters can sometimes return zero matches. Consider an example where the path to your login page is https://example.com/login/:

  • /login/ will return a match
  • /login with not return a match

JSON POST body

When creating rules that inspect the JSON body of POST requests, Post Parameter names require a leading /. For example, if the JSON payload is:

{
  "foo": "bar"
}

Then the name of the Post Parameter will need to be /foo in the rule.

An example rule using the condition of a Post Parameter with the name '/foo/ and the value of 'bar'.

The leading / on of Post Parameter name facilitates nested values. For example, /foo/bar for a payload such as:

{
  "foo": {
    "bar": ["value1", "value2"]
  }
}

Geolocation

Geolocation allows you to specify conditions that match against a particular country to block or allow traffic. Geolocation can be combined with other conditions like path or domain.

Where does the geodata come from?

We license MaxMind’s Geolite2 data and distribute it within our agent. This data is updated periodically and included with newer agent releases as well as dynamically updated similar to rule updates as of agent version 3.21.0.

How often is geodata updated?

We update our geodata and release an agent monthly (typically the second week of the month). At the same time as the agent release, the new geodata is deployed to our cloud tagging so that the latest country information is present. This will be a minor agent increment, such as 3.0.0 to 3.1.0. As of agent version 3.21.0, this data is also dynamically updated similar to rule updates and these agents will download and cache the updated geolocation data.

What happens if my agent is out-of-date?

If your agent is out-of-date or is not version 3.21.0 or later which will dynamically update, then an IP may be blocked or allowed based on outdated geo information. Or requests may display in the console that would have been blocked with newer geodata. The country displayed in the console will reflect the latest available geodata.

How do dynamic geolocation data updates work?

As of agent 3.21.0 the geolocation data is packaged up for the agent to download whenever there is an update. This data is cached locally on the agent machine. The cache location is under the shared-cache-dir directory which defaults to {$TMPDIR|%TMP%|%TEMP%}/sigsci-agent.cache/). The geolocation data is only downloaded if it does not exist locally or the data is not up-to-date.

Requirements for this functionality:

  • The filesystem where this cache directory resides must be:
    • Writeable by the user running the agent
    • Have at least 5MB of free space
    • While auto-detection of the cache directory normally works fine, you may need to configure shared-cache-dir on some systems where a TEMP space is not defined (e.g., where $TMPDIR or %TMP% or %TEMP% environment vars are not set properly)
  • The network must be capable of:
    • Downloading from the base download-url (this is the same base URL as normal rule updates)
    • Downloading the data (currently about 2MB) within the timeout limit (currently 60 seconds)

If the dynamic geolocation data cannot be downloaded, then the agent will default to the geolocation data packaged with the agent, reverting to functionality from agents prior to 3.21.0 as if the dynamic update feature was disabled.

How do I update my agent?

See Upgrading the Agent for documentation on how to upgrade the agent.