Log streaming: Log Shuttle

Fastly's Real-Time Log Streaming feature can send log files to Log Shuttle. Log Shuttle is an open source application designed to provide simpler encrypted and authenticated log delivery.

Adding Log Shuttle as a logging endpoint

Follow these instructions to add Log Shuttle as a logging endpoint:

1. Review the information in our Setting Up Remote Log Streaming guide.
2. Click the Log Shuttle Create endpoint button. The Create a Log Shuttle endpoint page appears.
3. Fill out the Create a Log Shuttle endpoint fields as follows:
   • In the Name field, enter a human-readable name for the endpoint.
   • In the Placement area, select where the logging call should be placed in the generated VCL. Valid values are Format Version Default, waf_debug (waf_debug_log), and None. See our guide on changing log placement for more information.
   • In the Log format field, optionally enter an Apache-style string or VCL variables to use for log formatting. See the example format section for details.
   • In the Token field, enter the data authentication token. This is required for some endpoints like Heroku's Log Integration.
   • In the URL field, enter the URL to which log data will be sent (e.g., https://logs.example.com/).
4. Click the Create button to create the new logging endpoint.
5. Click the Activate button to deploy your configuration changes.

Example format

The following is an example format string for sending data to Log Shuttle. Our discussion of format strings provides more information.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
{
  "timestamp": "%{strftime(\{"%Y-%m-%dT%H:%M:%S%z"\}, time.start)}V",
  "client_ip": "%{req.http.Fastly-Client-IP}V",
  "geo_country": "%{client.geo.country_name}V",
  "geo_city": "%{client.geo.city}V",
  "host": "%{if(req.http.Fastly-Orig-Host, req.http.Fastly-Orig-Host, req.http.Host)}V",
  "url": "%{json.escape(req.url)}V",
  "request_method": "%{json.escape(req.method)}V",
  "request_protocol": "%{json.escape(req.proto)}V",
  "request_referer": "%{json.escape(req.http.referer)}V",
  "request_user_agent": "%{json.escape(req.http.User-Agent)}V",
  "response_state": "%{json.escape(fastly_info.state)}V",
  "response_status": %{resp.status}V,
  "response_reason": %{if(resp.response, "%22"+json.escape(resp.response)+"%22", "null")}V,
  "response_body_size": %{resp.body_bytes_written}V,
  "fastly_server": "%{json.escape(server.identity)}V",
  "fastly_is_edge": %{if(fastly.ff.visits_this_service == 0, "true", "false")}V
}