Log streaming: FTP

Fastly's Real-Time Log Streaming feature can send log files to password-protected and anonymous FTP servers.

Adding FTP as a logging endpoint

Follow these instructions to add FTP as a logging endpoint:

1. Review the information in our Setting Up Remote Log Streaming guide.
2. Click the FTP Create endpoint button. The Create a File Transfer Protocol (FTP) endpoint page appears.
3. Fill out the Create a File Transfer Protocol (FTP) endpoint fields as follows:
   • In the Name field, enter a human-readable name for the endpoint.
   • In the Placement area, select where the logging call should be placed in the generated VCL. Valid values are Format Version Default, waf_debug (waf_debug_log), and None. See our guide on changing log placement for more information.
   • In the Log format field, optionally enter an Apache-style string or VCL variables to use for log formatting. See the example format section for details.
   • In the Timestamp format field, optionally enter a timestamp format for log files. The default is an strftime compatible string. Our guide on changing where log files are written provides more information.
   • In the Address field, enter the hostname or IP address of the FTP server. In the port field, enter the port number you're using for FTP (the default is 21).
   • In the Path field, optionally enter the path within the bucket to store the files. The path ends with a trailing slash. If this field is left empty, the files will be saved in the bucket's root path. Our guide on changing where log files are written provides more information.
   • In the User field, enter the username used to authenticate to the FTP server. For anonymous access, use the username anonymous.
   • In the Password field, enter the password used to authenticate to the FTP server. For anonymous access, use an email address as the password.
   • In the PGP public key field, optionally enter a PGP public key that Fastly will use to encrypt your log files before writing them to disk. You will only be able to read the contents by decrypting them with your private key. The PGP key should be in PEM (Privacy-Enhanced Mail) format. See our guide on log encryption for more information.
   • In the Period field, optionally enter an interval (in seconds) to control how frequently your log files are rotated. This value defaults to 3600 seconds.
4. Click the Advanced options link of the Create a File Transfer Protocol (FTP) endpoint page and decide which of the optional fields to change, if any.
5. Fill out the Advanced options of the Create a File Transfer Protocol (FTP) endpoint page as follows:
   • In the Select a log line format area, select the log line format for your log messages. Our guide on changing log line formats provides more information.
   • In the Compression field, optionally select the compression format you want applied to the log files. Our guide on changing log compression options provides more information.
6. Click the Create button to create the new logging endpoint.
7. Click the Activate button to deploy your configuration changes.

Example format

The following is an example format string for sending data to an FTP logging endpoint. Our discussion of format strings provides more information.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
{
  "timestamp": "%{strftime(\{"%Y-%m-%dT%H:%M:%S%z"\}, time.start)}V",
  "client_ip": "%{req.http.Fastly-Client-IP}V",
  "geo_country": "%{client.geo.country_name}V",
  "geo_city": "%{client.geo.city}V",
  "host": "%{if(req.http.Fastly-Orig-Host, req.http.Fastly-Orig-Host, req.http.Host)}V",
  "url": "%{json.escape(req.url)}V",
  "request_method": "%{json.escape(req.method)}V",
  "request_protocol": "%{json.escape(req.proto)}V",
  "request_referer": "%{json.escape(req.http.referer)}V",
  "request_user_agent": "%{json.escape(req.http.User-Agent)}V",
  "response_state": "%{json.escape(fastly_info.state)}V",
  "response_status": %{resp.status}V,
  "response_reason": %{if(resp.response, "%22"+json.escape(resp.response)+"%22", "null")}V,
  "response_body_size": %{resp.body_bytes_written}V,
  "fastly_server": "%{json.escape(server.identity)}V",
  "fastly_is_edge": %{if(fastly.ff.visits_this_service == 0, "true", "false")}V
}