Changing log line formats

Fastly's Real-Time Log Streaming feature allows you to change the format that your log messages are delivered in. By default, we send log messages out in standard syslog format. The prefix for this format (as defined in RFC 3164) appears as follows:

1
 <134>2016-07-04T22:37:26Z cache-sjc3128 LogTest[62959]: <your log message>

The prefix begins with the message priority (always <134>, which means Facility=Local0, Severity=Informational), followed the date and time the log was sent (2016-07-04T22:37:26Z), the cache node it came from (in this case cache-sjc3128), the name of your log (LogTest) and the ID of the process sending it (62959).

Available message formats

Although the default message prefix works for most logging services and processors, we allow you to choose one of several formats:

• classic is the default prefix format. A standard syslog prefix as defined by RFC 3164.
• loggly is a structured syslog prefix format based on RFC 5424.
• logplex is a Heroku-style length prefixed syslog format.
• blank means no prefix. Just your log message. Useful when writing to JSON and CSV files.

Updating endpoints to use a different format

The following logging endpoints can be updated to use a message format other than the default:

• Amazon S3
• Microsoft Azure Blob Storage
• Cloud Files
• DigitalOcean Spaces
• FTP
• Google Cloud Storage
• Apache Kafka
• OpenStack
• SFTP
• Sumo Logic
• Syslog

You can use the web interface or the API to update a logging endpoint.

Using the web interface

Follow these instructions to update a logging endpoint using the web interface:

1. Log in to the Fastly web interface and click the Configure link.
2. From the service menu, select the appropriate service.
3. Click the Configuration button and then select Clone active. The Domains page appears.
4. Click the Logging link. The Logging endpoints page appears.
5. Click the name of the logging endpoint you want to edit. The Edit this endpoint page appears.

6. Click the Advanced options link near the bottom of the page. The Advanced options appear.

   

7. In the Select a log line format section, select a log line format for the logging endpoint.
8. Click the Update button.
9. Click the Activate button to deploy your configuration changes.

Using the API

Run the following command to update a logging endpoint using the API:

1
curl -X PUT -H 'Fastly-Key: FASTLY_API_TOKEN' -H 'Content-Type: application/json' 'https://api.fastly.com/service/<your Fastly service ID>/version/<version number/logging/<log type>/<log name>' --data-binary '{"message_type":"<type>"}'

where log type is one of the eligible endpoint types:

• s3
• cloudfiles
• digitalocean
• elasticsearch
• ftp
• gcs
• kafka
• openstack
• sftp
• sumologic
• syslog

Keep in mind that the message_type field is a per-object field. Updating it on one logging object will not change it on any other objects. For example, to update a Google Cloud Storage endpoint to the blank message type the cURL command would look something like this if the endpoint was named "GCS Test":

1
curl -X PUT -H 'Fastly-Key: FASTLY_API_TOKEN' -H 'Content-Type: application/json' 'https://api.fastly.com/service/SU1Z0isxPaozGVKXdv0eY/version/1/logging/gcs/GCS%20Test' --data-binary '{"message_type":"blank"}'