Log streaming: Shape Log Analysis

      Last updated July 29, 2020

    Fastly's Real-Time Log Streaming feature can send log files to Shape Security. Shape Log Analysis uses anonymized attack data to analyze HTTP and application logs for insight into fraudulent activity and various types of attack that attempt to bypass security measures protecting origin servers.

    Prerequisites

    Before adding Shape Log Analysis as a logging endpoint, send an email to fastly@f5.com with the subject line "Fastly Log Streaming Setup" to request a secure S3 bucket from Shape. In return, you will receive an email with details to help you complete the configuration of the logging endpoint including:

    1. a note about the Log format value
    2. a Bucket name
    3. an Access key
    4. a Secret key
    5. a Path
    6. a Domain

    Each item will be specifically numbered in the email and the images in the configuration details below reflect those numbers.

    Adding Shape Log Analysis as a logging endpoint

    After you've contacted fastly@f5.com and received the prerequisite information email, complete the following steps:

    1. Review the information in our Setting Up Remote Log Streaming guide.
    2. Click the Amazon Web Services S3 Create endpoint button. The Create an Amazon S3 endpoint page appears.

      the create a Shape Log Analysis endpoint page

    3. Fill out the Create an Amazon S3 endpoint fields as follows:
      • In the Name field, enter a human-readable name for the endpoint.
      • In the Log format field, copy and paste the following log format value to send log data to Shape's secure S3 bucket:

        1
        2
        3
        4
        5
        6
        7
        8
        9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        23
        24
        25
        
        {
          "timestamp": "%{begin:%Y-%m-%dT%H:%M:%S%z}t",
          "ts": "%{time.start.sec}V",
          "id.orig_h": "%h", "status_code": "%>s",
          "method": "%m",
          "host": "%{Host}i",
          "uri": "%U%q",
          "accept_encoding": "%{Accept-Encoding}i",
          "request_body_len": "%{req.body_bytes_read}V",
          "response_body_len": "%{resp.body_bytes_written}V",
          "location": "%{Location}i",
          "x_forwarded_for": "%{X-Forwarded-For}i",
          "user_agent": "%{User-Agent}i",
          "referer": "%{Referer}i",
          "accept": "%{Accept}i",
          "accept_language": "%{Accept-Language}i",
          "content_type": "%{Content-Type}o",
          "geo_city": "%{client.geo.city}V",
          "geo_country_code": "%{client.geo.country_code}V",
          "is_tls": %{if(req.is_ssl, "true", "false")}V,
          "tls_version": "%{tls.client.protocol}V",
          "tls_cipher_request": "%{tls.client.cipher}V",
          "tls_cipher_req_hash": "%{tls.client.ciphers_sha}V",
          "tls_extension_identifiers_hash": "%{tls.client.tlsexts_sha}V"
        }
        
      • In the Bucket name field, enter the name of value #2 “Bucket Name” received in the Shape email response.
      • In the Access key field, enter the name of value #3 “Access Key” received in the Shape email response.
      • In the Secret key field, enter the name of value #4 “Secret Key” received in the Shape email response.

    4. Click the Advanced options link of the Create a new S3 endpoint page and fill in the following:

      the advanced options on the create a new Shape Log Analysis endpoint page

    5. Fill out the Advanced options of the Create an Amazon S3 endpoint page as follows:
      • In the Path field, enter the name of value #5 “Path” received in the email response.
      • In the Domain field, enter the name of value #6 “Domain” received in the email response.
      • In the Select a log line format area, select Blank to prevent prefixes from being added to log line messages and only report details in JSON log format. Our guide on changing log line formats provides more information.
    6. Click the Create button to create the new logging endpoint.
    7. Click the Activate button to deploy your configuration changes.

    Shape data analysis

    Once Fastly logging configuration is complete, logs will be sent to Shape's secure S3 bucket for analysis. Typically, Shape collects approximately two weeks worth of log data to provide an analysis of attack traffic. After analysis is complete, Shape will send you a report with data on topics like Malicious Automation, Attack Surface (URLs), Account Takeover Bots, Suspicious Manual Fraud, and Top Bot Campaigns.

    Shape Log Analysis Dashboard Animated

    Back to Top