Redacting data

To maintain data privacy, Signal Sciences redacts sensitive data from requests before they reach the platform backend.

Selective data transfer and redaction

The Signal Sciences agent filters requests locally to determine if they contain an attack. Only requests that are marked as attacks or anomalies are then sent to the Signal Sciences backend after additional filtering and sanitizing are done. Once the agent identifies a potential attack or anomaly in a request, the agent sends only the individual parameter of the request which contains the attack payload, as well as a few other non-sensitive or benign portions of the request (e.g., client IP, user agent, or URI). The entire request is never sent to the Signal Sciences backend. Additionally, specific portions of the request are automatically redacted and never sent to the backend, including tokens, credentials, and known patterns such as credit card and social security numbers.

JSON API payloads

Signal Sciences automatically parses JSON key-value pairs and treats them like request parameters. The following sample requests demonstrate how redactions work within the context of a request.

The initial request:

1POST /request HTTP/1.1
2Content-Length: 72
3Content-Type: application/json
4Host: api.example.com
5{"user":"user@api.example.com","password":"<script>alert(1)</script>mypassword","zip":94089}

What's sent to Signal Sciences:

1POST /request HTTP/1.1
2Host: api.example.com
3
4password=

The initial request:

1POST /request HTTP/1.1
2Content-Length: 72
3Content-Type: application/json
4Host: api.example.com
5
6{"user":"user@api.example.com","password":"mypassword","zip":"<script>alert(1)</script>94089"}

What's sent to Signal Sciences:

1POST /request HTTP/1.1
2Host: api.example.com
3
4zip=<script>alert(1)</script>

Sensitive headers

Signal Sciences redacts the following from requests:

• Explicit names: authorization, x-auth-token, cookie, set-cookie
• Any names that contain: -token, -auth, -key, -sess, -pass, -secret
• Query strings from referer and location

The initial request:

1POST /example?sort=ascending HTTP/1.1
2Host: example.com
3User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:35.0)
4Accept: text/html, application/xhtml+xml
5Content-Length: 57
6Cookie: foo=bar
7
8sensitive=hunter2&foobar=<script>alert(1)</script>&page=3

What’s sent to Signal Sciences:

1POST /example HTTP/1.1
2Host: example.com
3User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:35.0)
4
5foobar=<script>alert(1)</script>

Sensitive parameters

If a request contains an attack or anomaly, and also contains sensitive data in commonly-used parameter names, Signal Sciences will redact the entire contents of the sensitive parameter. These parameters include:

• api_key
• password
• passwd
• pass
• pw
• user
• login
• loginid
• username
• email
• key
• id
• sid
• token
• request_token
• access_token
• csrfmiddlewaretoken
• oauth_verifier
• confirm_password
• password_confirmation

The initial request:

1POST /example HTTP/1.1
2
3username=<script>alert("jsmith")</script>

What’s sent to Signal Sciences:

1POST /example HTTP/1.1
2
3username=[redacted]

The console clearly displays which parameters have been redacted. Redacted parameters are replaced with the word REDACTED highlighted in yellow.

Sensitive patterns

Signal Sciences automatically redacts known patterns of sensitive information, which includes the following:

• Credit card numbers: values like 4111-1111-1111-1111 become 0000-0000-0000-0000
• Social security numbers: values like 078-05-1120 become 000-00-0000
• GUIDs: values like 3F2504E0-4F89-41D3-9A0C-0305E82C3301 become 0000000-0000-0000-0000-000000000000
• Bank account (IBAN) numbers: values like DE75512108001245126199 become AA00aaaa0000000

The initial request:

1POST /example HTTP/1.1
2
3credit_card_example=<script>alert("4111-1111-1111-1111")</script>

What’s sent to Signal Sciences:

1POST /example HTTP/1.1
2
3credit_card_example=<script>alert("0000-0000-0000-0000")</script>

Within the console we clearly display which patterns have been redacted. Redacted patterns are replaced with the word REDACTED highlighted in yellow.

Custom redactions

In addition to the redactions listed above, you can also specify additional fields to redact from requests. For example, if your password field is named foobar instead of password, that field can be specified for redaction.

Creating custom redactions

When you have a sensitive field that is not filtered out by default, you can create a custom field redaction:

1. From the Rules menu, select Redactions.
2. Click Add redaction.
3. In the Field name field, enter the name of the field to be redacted.
4. From the Field type menu, select the type of field to be redacted. Options include Request parameter, Request header, or Response header.
5. Click Create redaction.

Editing custom redactions

To edit a custom redaction, complete the following steps:

1. From the Rules menu, select Redactions.
2. Click View to the right of the custom redaction you want to edit.
3. Click Edit redaction.
4. Change the Field name and Field type as needed.
5. Click Update redaction.

Deleting custom redactions

1. From the Site Rules menu, select Redactions.
2. Click View to the right of the custom redaction you want to delete.
3. Click Remove redaction.
4. Click Delete to delete the redaction.

Transparency

To allow for easy verification of what the agent sends to the backend, Signal Sciences provides a way to view all agent to backend communication.

Verifying in the console

To verify our agents are correctly filtering and sanitizing requests, we provide a raw log of data that’s sent from our agents:

1. Log in to the Signal Sciences console.
2. From the Sites menu, select a site if you have more than one site.
3. Click on Agents.
4. Click on the Agent ID.
5. Click the Requests tab.
6. Review the requests and verify that data is correctly redacted.

Verifying with the agent

You can also verify directly from the agent itself by setting the debug-log-uploads agent configuration option. For example, if you want to log all agent uploads in formatted JSON, add the following line to your agent configuration file (by default at /etc/sigsci/agent.conf):

debug-log-uploads = 2

Additional information about agent configuration options can be found here.