Log streaming: Shape Log Analysis

  Last updated 2021-06-09

Fastly's Real-Time Log Streaming feature can send log files to Shape Security. Shape Log Analysis uses anonymized attack data to analyze HTTP and application logs for insight into fraudulent activity and various types of attack that attempt to bypass security measures protecting origin servers.

NOTE

Fastly does not provide direct support for third-party services. Read Fastly's Terms of Service for more information.

Prerequisites

Before adding Shape Log Analysis as a logging endpoint, send an email to fastly@f5.com with the subject line "Fastly Log Streaming Setup" to request a secure S3 bucket from Shape. In return, you will receive an email with details to help you complete the configuration of the logging endpoint including:

  1. a note about the Log format value
  2. a Bucket name
  3. an Access key
  4. a Secret key
  5. a Path
  6. a Domain

Each item will be specifically numbered in the email and the images in the configuration details below reflect those numbers.

Adding Shape Log Analysis as a logging endpoint

NOTE

This logging endpoint is only available for Fastly's Deliver (VCL-based) services, not for Compute services.

IMPORTANT

Shape Log Analysis setup and configuration uses Fastly's Amazon S3 log streaming endpoint to enable logs for storage and analysis. It places them in a secure S3 bucket managed by Shape that only accepts traffic from Fastly IP addresses.

After you've contacted fastly@f5.com and received the prerequisite information email, complete the following steps:

  1. Deliver services
  1. Review the information in our guide to setting up remote log streaming.

  2. In the Amazon Web Services S3 area, click Create endpoint.

  3. Fill out the Create an Amazon S3 endpoint fields as follows:

    • In the Name field, enter a human-readable name for the endpoint.
    • In the Placement area, select where the logging call should be placed in the generated VCL. Valid values are Format Version Default, waf_debug (waf_debug_log), and None. Read our guide on changing log placement for more information.

    • In the Log format field, copy and paste the following log format value to send log data to Shape's secure S3 bucket:

      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      {
        "timestamp": "%{begin:%Y-%m-%dT%H:%M:%S%z}t",
        "ts": "%{time.start.sec}V",
        "id.orig_h": "%h", "status_code": "%>s",
        "method": "%m",
        "host": "%{Host}i",
        "uri": "%U%q",
        "accept_encoding": "%{Accept-Encoding}i",
        "request_body_len": "%{req.body_bytes_read}V",
        "response_body_len": "%{resp.body_bytes_written}V",
        "location": "%{Location}i",
        "x_forwarded_for": "%{X-Forwarded-For}i",
        "user_agent": "%{User-Agent}i",
        "referer": "%{Referer}i",
        "accept": "%{Accept}i",
        "accept_language": "%{Accept-Language}i",
        "content_type": "%{Content-Type}o",
        "geo_city": "%{client.geo.city}V",
        "geo_country_code": "%{client.geo.country_code}V",
        "is_tls": %{if(req.is_ssl, "true", "false")}V,
        "tls_version": "%{tls.client.protocol}V",
        "tls_cipher_request": "%{tls.client.cipher}V",
        "tls_cipher_req_hash": "%{tls.client.ciphers_sha}V",
        "tls_extension_identifiers_hash": "%{tls.client.tlsexts_sha}V"
      }

    • In the Access method area, select User Credentials.

    • In the Bucket name field, enter the name of value #2 “Bucket Name” received in the Shape email response.

    • In the Access key field, enter the name of value #3 “Access Key” received in the Shape email response.

    • In the Secret key field, enter the name of value #4 “Secret Key” received in the Shape email response.

    NOTE

    Password management software may mistakenly treat the Secret Key field as a password field because of the way your web browser works. As such, that software may try to auto-fill this field with your Fastly account password. If this happens to you, the integration with Fastly services won't work and you will need to enter Secret Key manually instead.

  4. Click Advanced options and fill out the fields as follows:

    • In the Path field, enter the name of value #5 “Path” received in the email response.
    • In the Domain field, enter the name of value #6 “Domain” received in the email response.
    • In the Select a log line format area, select Blank to prevent prefixes from being added to log line messages and only report details in JSON log format. Our guide on changing log line formats provides more information.
  5. Click Create to create the new logging endpoint.
  6. Click Activate to deploy your configuration changes.

Shape data analysis

Once Fastly logging configuration is complete, logs will be sent to Shape's secure S3 bucket for analysis. Typically, Shape collects approximately two weeks worth of log data to provide an analysis of attack traffic. After analysis is complete, Shape will send you a report with data on topics like Malicious Automation, Attack Surface (URLs), Account Takeover Bots, Suspicious Manual Fraud, and Top Bot Campaigns.

Shape Log Analysis Dashboard Animated

Was this guide helpful?

Do not use this form to send sensitive information. If you need assistance, contact support. This form is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Fastly
© Fastly 2024