search close

Rules

access_time Updated Oct 22, 2021

The rules feature allows you to block, allow, and tag requests and exclude system signals for arbitrary sets of conditions. Rules can be created on individual sites (site rules) as well as the corp as a whole (corp rules) to be easily used in multiple sites.

Corp rules can be managed by going to Corp Rules > Corp Rules.

Site rules can be managed by going to Site Rules > Site Rules.

Request rules

Request rules allow you to define arbitrary conditions and either block, allow, or tag requests indefinitely or for a specific period of time.

The below example request rule blocks all requests to the /login page from the IP address 198.51.100.50.

  • The first condition has IP Address selected for the “Field”, Equals selected for the “Operator”, and 198.51.100.50 entered for the “Value”.
  • The second condition has Path selected for the “Field”, Equals selected for the “Operator”, and /login entered for the “Value”.
  • The “Action type” is set to Block.

A request rule designed to block requests to the '/login' page from the IP address '198.51.100.50', as described above.

Request fields

Field Type Properties
Agent name String Text or wildcard
Country Enum ISO countries
Domain String Text or wildcard
IP address IP Text or wildcard, supports CIDR notation
Method Enum GET, POST, PUT, PATCH, DELETE, HEAD, TRACE
Path String Text or wildcard
POST parameter Multiple Name (string), Value (string)
Query parameter Multiple Name (string), Value (string)
Request cookie Multiple Name (string), Value (string)
Request header Multiple Name (string), Value (string), Value (IP)
Response code String Text or wildcard
Response header Multiple Name (string), Value (string)
Scheme Enum http, https
Signal Multiple Type (signal), Parameter name (string), Parameter value (string)
User agent String Text or wildcard

Signal exclusion rules

Signal exclusion rules allow you to define arbitrary conditions to exclude a specific system signal.

The below example signal exclusion rule prevents POST requests originating from a list of known internal IP addresses from being tagged with the NO-CONTENT-TYPE signal.

  • The “Signal” is set to No Content Type.
  • The first condition has Method selected for the “Field”, Equals selected for the “Operator”, and POST selected for the “Value”.
  • The second condition has IP address selected for the “Field”, Is in list selected for the “Operator”, and the Developer IPs (IP) list selected for the “Value”.

A signal exclusion rule designed to prevent POST requests originating from a list of known internal developer IP addresses from being tagged with the 'No Content Type' signal, as described above.

Signal exclusion fields

Signal exclusion rules have the same fields as request rules as well as additional fields specific to the particular signal that’s being excluded.

Field Type Properties
Parameter name String Text or wildcard
Parameter value String Text or wildcard

Rate limit rules

See Rate Limit Rules for information about using rate limit rules.

Templated rules

See Templated Rules for information about using templated rules.

Converting Requests to Rules

Individual requests in the Requests page can be converted into pre-populated rules, enabling you to easily allow, block, and tag similar requests.

How to convert a request to a rule

  1. In the console, go to Requests.

  2. Locate or search for the request you want to convert into a rule.

  3. Click View request detail.

  4. Click Convert to rule in the upper-right corner.

  5. Select the type of rule you want to make (e.g., request, rate limit, or signal exclusion).

  6. Select which characteristics of the request you want to convert into rule conditions. For example, selecting “IP Address” and “Path” will create conditions in the rule that look for the IP address and path featured in the request.

  7. Click Continue.

  8. You will be taken to a pre-built rule with conditions featuring the request characteristics you selected. Modify the rule as needed, such as by adding and editing rule conditions.

  9. Finish setting up the rule by setting:

    • What action it should take (e.g., block, allow, or tag requests).
    • Whether it should be enabled or disabled.
    • If it should automatically be disabled after a set period of time.
    • A description of the rule.
  10. Click Create site rule.

Operators

When creating rules, operators (“equals”, “is in list”, etc.) are used to specify the logic of your rule when matching conditions. For example, the “equals” operator is used to check if a value in the request matches the value in the rule condition exactly—for example, to match a specific IP address or path.

Operator Function Example Match
Equals Checks if the request value matches the rule condition value exactly 203.0.113.169 Equals 203.0.113.169
Does not equal Checks if the request value does not match the rule condition value exactly 203.0.113.169 Does not equal 192.0.2.191
Contains Checks if the rule condition value being checked is contained within the request value; for example, to check if a substring is found within a larger string thisisanexamplestring Contains example
Does not contain Checks if the rule condition value being checked is not contained within the request value; for example, to check if a substring is not found within a larger string thisisanexamplestring Does not contain elephant
Like (wildcard) Allows the use of wildcard characters in matching; checks if the request value matches the rule condition value bats Like (wildcard) [bcr]ats
Not like (wildcard) Allows the use of wildcard characters in matching; checks if the request value does not match the rule condition value bats Not like (wildcard) [hps]ats
Matches (regexp) Allows the use of regular expressions in matching; checks if the request value matches the rule condition value bats Matches (regexp) (b|c|r)ats
Does not match (regexp) Allows the use of regular expressions in matching; checks if the request value does not match the rule condition value bats Does not match (regexp) (h|p|s)ats
Is in list Checks if the request value matches any of the values in a specific list 203.0.113.169 Is in list “Known IP Addresses”
Is not in list Checks if the request value does not match any of the values in a specific list 192.0.2.191 Is not in list “Known IP Addresses”

Wildcards

The “Like (wildcard)” operator supports 0-or-many wildcards (*), single-character wildcards (?), character-lists ([abc]), character-ranges ([a-c], [0-9]), alternatives ({cat,bat,[fr]at}), and exclusions ([!abc], [!0-9]).

If you need to match a literal *, ?, [, or ] character, escape them with the \ character. For example: \*.

The “Like (wildcard)” operator requires a full string match. If you’re trying to match part of a string, you may need to include the * wildcard at the beginning or end to include the rest of the string for correct matching.

Regular expressions are not supported with the “Like (wildcard)” operator. If you want to use regular expressions, you must use the “Matches (regexp)” operator.

Case sensitivity

All fields in rules are case sensitive with the exception of header names.

For example, if you create a rule that looks for a header named X-Custom-Header, it will match on requests with headers named X-Custom-Header and x-custom-header because header names aren’t case sensitive. However, if the rule looks for the value Example-Value, it will only match on Example-Value and not example-value because all other rule fields—such as header values in this example—are case sensitive.

Path syntax best practices

Always use leading slashes

For a URL like https://example.com/some-path, the correct path syntax to use would be /some-path.

Use relative paths instead of absolute URLs

For example, if the absolute URL to the login page on your site is https://example.com/login, then /login is the correct path syntax to enter when configuring your login signals.

Take care when using trailing characters in your paths

Since our path syntax uses exact matching, trailing characters can sometimes return zero matches. Consider an example where the path to your login page is https://example.com/login/:

  • /login/ will return a match
  • /login with not return a match

JSON POST body

When creating rules that inspect the JSON body of POST requests, Post Parameter names require a leading /. For example, if the JSON payload is:

{ 
  "foo": "bar" 
}

Then the name of the Post Parameter will need to be /foo in the rule.

An example rule using the condition of a Post Parameter with the name '/foo/ and the value of 'bar'.

The leading / on of Post Parameter name facilitates nested values. For example, /foo/bar for a payload such as:

{
  "foo": {
    "bar": ["value1", "value2"]
  }
}

Geolocation

Geolocation allows you to specify conditions that match against a particular country to block or allow traffic. Geolocation can be combined with other conditions like path or domain.

Where does the geodata come from?

We license MaxMind’s Geolite2 data and distribute it within our agent. This data is updated periodically and included with newer agent releases as well as dynamically updated similar to rule updates as of agent version 3.21.0.

How often is geodata updated?

We update our geodata and release an agent monthly (typically the second week of the month). At the same time as the agent release, the new geodata is deployed to our cloud tagging so that the latest country information is present. This will be a minor agent increment, such as 3.0.0 to 3.1.0. As of agent version 3.21.0, this data is also dynamically updated similar to rule updates and these agents will download and cache the updated geolocation data.

What happens if my agent is out-of-date?

If your agent is out-of-date or is not version 3.21.0 or later which will dynamically update, then an IP may be blocked or allowed based on outdated geo information. Or requests may display in the console that would have been blocked with newer geodata. The country displayed in the console will reflect the latest available geodata.

How do dynamic geolocation data updates work?

As of agent 3.21.0 the geolocation data is packaged up for the agent to download whenever there is an update. This data is cached locally on the agent machine. The cache location is under the shared-cache-dir directory which defaults to {$TMPDIR|%TMP%|%TEMP%}/sigsci-agent.cache/). The geolocation data is only downloaded if it does not exist locally or the data is not up-to-date.

Requirements for this functionality:

  • The filesystem where this cache directory resides must be:
    • Writeable by the user running the agent
    • Have at least 5MB of free space
    • While auto-detection of the cache directory normally works fine, you may need to configure shared-cache-dir on some systems where a TEMP space is not defined (e.g., where $TMPDIR or %TMP% or %TEMP% environment vars are not set properly)
  • The network must be capable of:
    • Downloading from the base download-url (this is the same base URL as normal rule updates)
    • Downloading the data (currently about 2MB) within the timeout limit (currently 60 seconds)

If the dynamic geolocation data cannot be downloaded, then the agent will default to the geolocation data packaged with the agent, reverting to functionality from agents prior to 3.21.0 as if the dynamic update feature was disabled.

How do I update my agent?

See Upgrading the Agent for documentation on how to upgrade the agent.