Getting started
Basics
Domains & Origins
Performance

Configuration
Basics
Conditions
Dictionaries
Domains & Origins
Request settings
Cache settings
Headers
Responses
Performance
Custom VCL
Image optimization
Video

Security
Access Control Lists
Monitoring and testing
Securing communications
Security measures
TLS
Web Application Firewall

Integrations
Logging endpoints
Non-Fastly services

Diagnostics
Streaming logs
Debugging techniques
Common errors

Account info
Account management
Billing
User access and control

Reference

    Log streaming: Splunk

      Last updated June 03, 2019

    Fastly's Real-Time Log Streaming feature can send log files to Splunk. Splunk is a web-based log analytics platform used by developers and IT teams.

    Prerequisites

    To use Splunk as a logging endpoint, you'll need to enable the HTTP Event Collector (HEC), create a token, and enable it. Follow the instructions on Splunk's website:

    1. Enable HEC.
    2. Create an HEC token.
    3. Enable the HEC token.

    You'll need to remember the HEC token and find the URL for your collector. The URL structure depends on the type of Splunk instance you're using. Use the table below to find the URL structure for your Splunk instance.

    Type URL
    Self hosted https://<hostname>:8088/services/collector/event
    Self-service Splunk Cloud plans https://input-<hostname>:8088/services/collector/event
    All other Splunk Cloud plans https://http-inputs-<hostname>:8088/services/collector/event

    While logged in to Splunk, you can find the hostname for the URL in your web browser's address bar.

    Adding Splunk as a logging endpoint

    After you've created a Splunk account and obtained your customer token, follow these instructions to add Splunk as a logging endpoint for Fastly services:

    1. Review the information in our Setting Up Remote Log Streaming guide.
    2. Click the Splunk logo. The Create a Splunk endpoint page appears.

      the create a Splunk endpoint page

    3. Fill out the Create a Splunk endpoint fields as follows:
      • In the Name field, type a human-readable name for the endpoint.
      • In the Log format field, type an Apache-style string or VCL variables to use for log formatting. You can use our recommended log format.
      • In the URL field, type the URL to send data to (e.g., https://<splunk_host>:8088/services/collector/event/1.0).
      • In the TLS hostname field, type the hostname used to verify the server's certificate. If you're using Splunk Enterprise, type SplunkServerDefaultCert.
      • In the TLS CA certificate field, type the CA certificate used to verify that the origin's certificate is valid. It must be in PEM format. This is not required if your origin-side TLS certificate is signed by a well-known CA. See the using TLS CA certificates section for more information.
      • In the Token field, type the token for the HEC.
    4. Click the Advanced options link of the Create a Splunk endpoint page. The Advanced options appear.

      the advanced options on the create a new Splunk endpoint page

    5. In the Placement area, select where the logging call should be placed in the generated VCL. Valid values are Format Version Default, None, and waf_debug (waf_debug_log). Selecting None creates a logging object that can only be used in custom VCL. See our guide on WAF logging for more information about waf_debug_log.
    6. Click the Create button to create the new logging endpoint.
    7. Click the Activate button to deploy your configuration changes.

    We recommend using the following log format to send data to Splunk.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    
    {
      "time":%{time.start.sec}V,
      "event":  {
        "service_id":"%{req.service_id}V",
        "time_start":"%{begin:%Y-%m-%dT%H:%M:%S%Z}t",
        "time_end":"%{end:%Y-%m-%dT%H:%M:%S%Z}t",
        "time_elapsed":%D,
        "client_ip":"%h",
        "client_as_name":"%{client.as.name}V",
        "client_as_number":"%{client.as.number}V",
        "client_connection_speed":"%{client.geo.conn_speed}V",
        "request":"%m",
        "protocol":"%H",
        "host":"%{Fastly-Orig-Host}i",
        "origin_host":"%v",
        "url":"%{cstr_escape(req.url)}V",
        "is_ipv6":%{if(req.is_ipv6, "true", "false")}V,
        "is_tls":%{if(req.is_ssl, "true", "false")}V,
        "tls_client_protocol":"%{cstr_escape(tls.client.protocol)}V",
        "tls_client_servername":"%{cstr_escape(tls.client.servername)}V",
        "tls_client_cipher":"%{cstr_escape(tls.client.cipher)}V",
        "tls_client_cipher_sha":"%{cstr_escape(tls.client.ciphers_sha )}V",
        "tls_client_tlsexts_sha":"%{cstr_escape(tls.client.tlsexts_sha)}V",
        "is_h2":%{if(fastly_info.is_h2, "true", "false")}V,
        "is_h2_push":%{if(fastly_info.h2.is_push, "true", "false")}V,
        "h2_stream_id":"%{fastly_info.h2.stream_id}V",
        "request_referer":"%{Referer}i",
        "request_user_agent":"%{User-Agent}i",
        "request_accept_content":"%{Accept}i",
        "request_accept_language":"%{Accept-Language}i",
        "request_accept_encoding":"%{Accept-Encoding}i",
        "request_accept_charset":"%{Accept-Charset}i",
        "request_connection":"%{Connection}i",
        "request_dnt":"%{DNT}i",
        "request_forwarded":"%{Forwarded}i",
        "request_via":"%{Via}i",
        "request_cache_control":"%{Cache-Control}i",
        "request_x_requested_with":"%{X-Requested-With}i",
        "request_x_att_device_id":"%{X-ATT-Device-Id}i",
        "request_x_forwarded_for":"%{X-Forwarded-For}i",
        "status":"%s",
        "content_type":"%{Content-Type}o",
        "cache_status":"%{regsub(fastly_info.state, "^(HIT-(SYNTH)|(HITPASS|HIT|MISS|PASS|ERROR|PIPE)).*", "\\2\\3")}V",
        "is_cacheable":%{if(fastly_info.state ~"^(HIT|MISS)$", "true", "false")}V,
        "response_age":"%{Age}o",
        "response_cache_control":"%{Cache-Control}o",
        "response_expires":"%{Expires}o",
        "response_last_modified":"%{Last-Modified}o",
        "response_tsv":"%{TSV}o",
        "server_datacenter":"%{server.datacenter}V",
        "server_ip":"%A",
        "geo_city":"%{client.geo.city.utf8}V",
        "geo_country_code":"%{client.geo.country_code}V",
        "geo_continent_code":"%{client.geo.continent_code}V",
        "geo_region":"%{client.geo.region}V",
        "req_header_size":%{req.header_bytes_read}V,
        "req_body_size":%{req.body_bytes_read}V,
        "resp_header_size":%{resp.header_bytes_written}V,
        "resp_body_size":%B,
        "socket_cwnd":%{client.socket.cwnd}V,
        "socket_nexthop":"%{client.socket.nexthop}V",
        "socket_tcpi_rcv_mss":%{client.socket.tcpi_rcv_mss}V,
        "socket_tcpi_snd_mss":%{client.socket.tcpi_snd_mss}V,
        "socket_tcpi_rtt":%{client.socket.tcpi_rtt}V,
        "socket_tcpi_rttvar":%{client.socket.tcpi_rttvar}V,
        "socket_tcpi_rcv_rtt":%{client.socket.tcpi_rcv_rtt}V,
        "socket_tcpi_rcv_space":%{client.socket.tcpi_rcv_space}V,
        "socket_tcpi_last_data_sent":%{client.socket.tcpi_last_data_sent}V,
        "socket_tcpi_total_retrans":%{client.socket.tcpi_total_retrans}V,
        "socket_tcpi_delta_retrans":%{client.socket.tcpi_delta_retrans}V,
        "socket_ploss":%{client.socket.ploss}V
      }
    }
    

    Using TLS CA certificates

    If you've installed your own TLS certificate in Splunk Enterprise or Splunk Cloud, you'll need to provide the corresponding CA certificate.

    Splunk Cloud

    For Splunk Cloud, the default set up has the following CA certificate:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    
    -----BEGIN CERTIFICATE-----
    MIIB/DCCAaGgAwIBAgIBADAKBggqhkjOPQQDAjB+MSswKQYDVQQDEyJTcGx1bmsg
    Q2xvdWQgQ2VydGlmaWNhdGUgQXV0aG9yaXR5MRYwFAYDVQQHEw1TYW4gRnJhbmNp
    c2NvMRMwEQYDVQQKEwpTcGx1bmsgSW5jMQswCQYDVQQIEwJDQTEVMBMGA1UECxMM
    U3BsdW5rIENsb3VkMB4XDTE0MTExMDA3MDAxOFoXDTM0MTEwNTA3MDAxOFowfjEr
    MCkGA1UEAxMiU3BsdW5rIENsb3VkIENlcnRpZmljYXRlIEF1dGhvcml0eTEWMBQG
    A1UEBxMNU2FuIEZyYW5jaXNjbzETMBEGA1UEChMKU3BsdW5rIEluYzELMAkGA1UE
    CBMCQ0ExFTATBgNVBAsTDFNwbHVuayBDbG91ZDBZMBMGByqGSM49AgEGCCqGSM49
    AwEHA0IABPRRy9i3yQcxgMpvCSsI7Qe6YZMimUHOecPZWaGz5jEfB4+p5wT7dF3e
    QrgjDWshVJZvK6KGO7nDh97GnbVXrTCjEDAOMAwGA1UdEwQFMAMBAf8wCgYIKoZI
    zj0EAwIDSQAwRgIhALMUgLYPtICN9ci/ZOoXeZxUhn3i4wIo2mPKEWX0IcfpAiEA
    8Jid6bzwUqAdDZPSOtaEBXV9uRIrNua0Qxl1S55TlWY=
    -----END CERTIFICATE-----
    

    Splunk Enterprise

    In the Fastly web interface, type SplunkServerDefaultCert in the TLS hostname field.

    For Splunk Enterprise, the default set up has the following CA certificate.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    
    -----BEGIN CERTIFICATE-----
    MIIDejCCAmICCQCNHBN8tj/FwzANBgkqhkiG9w0BAQsFADB/MQswCQYDVQQGEwJV
    UzELMAkGA1UECAwCQ0ExFjAUBgNVBAcMDVNhbiBGcmFuY2lzY28xDzANBgNVBAoM
    BlNwbHVuazEXMBUGA1UEAwwOU3BsdW5rQ29tbW9uQ0ExITAfBgkqhkiG9w0BCQEW
    EnN1cHBvcnRAc3BsdW5rLmNvbTAeFw0xNzAxMzAyMDI2NTRaFw0yNzAxMjgyMDI2
    NTRaMH8xCzAJBgNVBAYTAlVTMQswCQYDVQQIDAJDQTEWMBQGA1UEBwwNU2FuIEZy
    YW5jaXNjbzEPMA0GA1UECgwGU3BsdW5rMRcwFQYDVQQDDA5TcGx1bmtDb21tb25D
    QTEhMB8GCSqGSIb3DQEJARYSc3VwcG9ydEBzcGx1bmsuY29tMIIBIjANBgkqhkiG
    9w0BAQEFAAOCAQ8AMIIBCgKCAQEAzB9ltVEGk73QvPlxXtA0qMW/SLDQlQMFJ/C/
    tXRVJdQsmcW4WsaETteeWZh8AgozO1LqOa3I6UmrWLcv4LmUAh/T3iZWXzHLIqFN
    WLSVU+2g0Xkn43xSgQEPSvEK1NqZRZv1SWvx3+oGHgu03AZrqTj0HyLujqUDARFX
    sRvBPW/VfDkomHj9b8IuK3qOUwQtIOUr+oKx1tM1J7VNN5NflLw9NdHtlfblw0Ys
    5xI5Qxu3rcCxkKQuwz9KRe4iijOIRMAKX28pbakxU9Nk38Ac3PNadgIk0s7R829k
    980sqGWkd06+C17OxgjpQbvLOR20FtmQybttUsXGR7Bp07YStwIDAQABMA0GCSqG
    SIb3DQEBCwUAA4IBAQCxhQd6KXP2VzK2cwAqdK74bGwl5WnvsyqdPWkdANiKksr4
    ZybJZNfdfRso3fA2oK1R8i5Ca8LK3V/UuAsXvG6/ikJtWsJ9jf+eYLou8lS6NVJO
    xDN/gxPcHrhToGqi1wfPwDQrNVofZcuQNklcdgZ1+XVuotfTCOXHrRoNmZX+HgkY
    gEtPG+r1VwSFowfYqyFXQ5CUeRa3JB7/ObF15WfGUYplbd3wQz/M3PLNKLvz5a1z
    LMNXDwN5Pvyb2epyO8LPJu4dGTB4jOGpYLUjG1UUqJo9Oa6D99rv6sId+8qjERtl
    ZZc1oaC0PKSzBmq+TpbR27B8Zra3gpoA+gavdRZj
    -----END CERTIFICATE-----
    
    Back to Top