Log streaming: HTTPS

Fastly's Real-Time Log Streaming feature can send log files to an HTTPS endpoint.

NOTE

Fastly does not provide direct support for third-party services. Read Fastly's Terms of Service for more information.

Prerequisites

When sending logs to a HTTPS endpoint, Fastly requires proof that you control the domain name specified in the URL field by using a HTTP challenge on a well-known path. If, for example, your URL field is foo.example.com/some/log/path, then the following challenge path must send a 200 response:

foo.example.com/.well-known/fastly/logging/challenge

Responses must include the hex representation of the SHA-256 of your Fastly service ID and it must appear on its own line in the response. For example:

$ sha256sum <SERVICEID>
ef537f25c895bfa782526529a9b63d97aa631564d5d789c2b765448c8635fb6c

If multiple service IDs are used, multiple hex(sha256) lines can be added to that challenge body. In addition, an asterisk (*) can be used on a line to allow any service to post to the HTTP endpoint. For example:

ef537f25c895bfa782526529a9b63d97aa631564d5d789c2b765448c8635fb6c
06ae6402e02a9dad74edc71aa69c77c5747e553b0840bfc56feb7e65b23f0f61
*

Adding HTTPS as a logging endpoint

Follow these instructions to add HTTPS as a logging endpoint:

  1. Deliver services
  2. Compute services
  1. Review the information in our guide to setting up remote log streaming.
  2. In the HTTPS area, click Create endpoint.
  3. Fill out the Create an HTTPS endpoint fields as follows:
    • In the Name field, enter a human-readable name for the endpoint.
    • In the Placement area, select where the logging call should be placed in the generated VCL. Valid values are Format Version Default, waf_debug (waf_debug_log), and None. Read our guide on changing log placement for more information.
    • In the Log format field, optionally enter an Apache-style string or VCL variables to use for log formatting. Consult the example format section for details.
    • In the URL field, enter the URL to which log data will be sent (e.g., https://logs.example.com/).
    • (Optional) In the Maximum logs field, enter the maximum number of logs to send as a batch.
    • (Optional) In the Maximum bytes field, enter the maximum size of a log batch.
  4. Click Advanced options and fill out the fields as follows:
    • (Optional) In the Content type field, enter the content type to use when sending logs (e.g., application/json).
    • (Optional) In the Custom header name field, enter a custom header to use when sending logs (e.g., Authorization).
    • (Optional) In the Custom header value field, enter a custom header value to use when sending logs (e.g., Bearer <token>).
    • (Optional) In the Method area, select the appropriate HTTP method to use.
    • In the JSON log entry format area, select the appropriate log entry format to use. The JSON log entry format enforces valid JSON formatting. Selecting Array of JSON wraps JSON log batches in an array. Selecting Newline delimited places each JSON log entry onto a new line in a batch.
    • In the Select a log line format area, select the log line format for your log messages. Our guide on changing log line formats provides more information.
  5. Fill out the Using your own certificate authority (CA) section of the Advanced options area as follows:
    • (Optional) In the TLS Hostname field, enter the hostname used to verify the logging endpoint server's certificate. This can be either the Common Name (CN) or Subject Alternative Name (SAN). This field only appears when you select Yes from the Use TLS menu.
    • (Optional) In the TLS CA certificate field, copy and paste the certification authority (CA) certificate used to verify the logging endpoint server's certificate is valid. The certificate you upload must be in PEM format. Consider uploading the certificate if it's not signed by a well-known certification authority. This value is not required if your TLS certificate is signed by a well-known authority. This field only appears when you select Yes from the Use TLS menu.
    • (Optional) In the TLS client certificate field, copy and paste the TLS client certificate used to authenticate Fastly to the logging endpoint server. The TLS client certificate you upload must be in PEM format and must be accompanied by a client key. A TLS client certificate allows your logging endpoint server to authenticate that Fastly is performing the connection. This field only appears when you select Yes from the Use TLS menu.
    • (Optional) In the TLS client key field, copy and paste the TLS client key used to authenticate to the logging endpoint server. The TLS client key you upload must be in PEM format and must be accompanied by a TLS client certificate. A TLS client key allows your logging endpoint server to authenticate that Fastly is performing the connection. This field only appears when you select Yes from the Use TLS menu.
  6. Click Create to create the new logging endpoint.
  7. Click Activate to deploy your configuration changes.

Example format

The following is an example format string for sending data to an HTTPS logging endpoint. Our discussion of format strings provides more information.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
{
"timestamp": "%{strftime(\{"%Y-%m-%dT%H:%M:%S%z"\}, time.start)}V",
"client_ip": "%{req.http.Fastly-Client-IP}V",
"geo_country": "%{client.geo.country_name}V",
"geo_city": "%{client.geo.city}V",
"host": "%{if(req.http.Fastly-Orig-Host, req.http.Fastly-Orig-Host, req.http.Host)}V",
"url": "%{json.escape(req.url)}V",
"request_method": "%{json.escape(req.method)}V",
"request_protocol": "%{json.escape(req.proto)}V",
"request_referer": "%{json.escape(req.http.referer)}V",
"request_user_agent": "%{json.escape(req.http.User-Agent)}V",
"response_state": "%{json.escape(fastly_info.state)}V",
"response_status": %{resp.status}V,
"response_reason": %{if(resp.response, "%22"+json.escape(resp.response)+"%22", "null")}V,
"response_body_size": %{resp.body_bytes_written}V,
"fastly_server": "%{json.escape(server.identity)}V",
"fastly_is_edge": %{if(fastly.ff.visits_this_service == 0, "true", "false")}V
}

Firewall considerations

Your HTTPS endpoint may have limited security features. For this reason, it's best to create a firewall for your HTTP endpoint server and only accept TCP traffic on your configured port from our address blocks. Our list of IP address blocks is dynamic, so we recommend programmatically obtaining the list whenever possible.

Was this guide helpful?

Do not use this form to send sensitive information. If you need assistance, contact support. This form is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.