LOG IN SIGN UP
Documentation

Custom log formats

Fastly provides two versions of custom log formats. All logs use the version 1 custom log format by default, but you can upgrade to the version 2 custom log format and then make version 2 look like version 1 for the sake of continuity. We've described the key advantages of the version 2 custom log format below.

Version 1 log format

This table details version 1 of Fastly's custom log formats.

String Description
b The content size of the response, calculated using the Content-Length header rather than actually checking the length of the response (and may therefore be wrong).
h The remote IP address.
l The remote log name. Always returns the hardcoded value "-".
r The HTTP verb and request path (e.g., GET /index.html). Unlike Apache and version 2 log formats, the protocol version is not included.
>s The status of the last request.
t The time the request was received, in Unix ctime format (e.g., Thu, 01 Jan 1970 00:00:00 GMT) rather than Apache's Standard English format (e.g., 01/Jan/1970:00:00:00 -0700).
u The remote user. Always returns the hardcoded value "-".

Version 2 log format

This table details version 2 of Fastly's custom log formats.

String Description
% The percent sign.
a The client IP address of the request.
A The local IP address.
B The size of response in bytes, excluding HTTP headers.
b The size of response in bytes, excluding HTTP headers. In Common Log Format (CLF), that means a "-" rather than a 0 when no bytes are sent.
{Foobar}C The contents of cookie Foobar in the request sent to the server.
D The time taken to serve the request, in microseconds.
{FOOBAR}e The contents of the environment variable FOOBAR. Always returns NIL.
f The filename.
h The remote hostname.
H The request protocol.
{Foobar}i The contents of Foobar: header lines in the request sent to the server.
I Bytes received, including request and headers. Cannot be zero.
k The number of keepalive requests handled on this connection. Always returns 0.
l The remote logname (from identd, if supplied). This will return a dash unless mod_ident is present and IdentityCheck is set On. Always returns "-".
m The request method.
{Foobar}n The contents of note Foobar from another module. Always returns NIL.
{Foobar}o The contents of Foobar: header lines in the reply.
O Bytes sent, including headers. Cannot be zero.
p The canonical port of the server serving the request. Always returns 80.
{format}p The canonical port of the server serving the request. Valid formats are canonical, local, or remote. Returns 80 for HTTP requests and 443 for HTTPS requests.
P The process ID of the child that serviced the request. Always returns NIL.
{format}P The process ID or thread ID of the child that serviced the request. Valid formats are pid, tid, and hextid. Always returns NIL.
q The query string (prepended with a ? if a query string exists, otherwise an empty string).
r The first line of the request.
R The handler generating the response (if any).
s The status. For requests that got internally redirected, this is the status of the original request. Use %>s for the final status.
t The time the request was received, in Standard English format (e.g., 01/Jan/1970:00:00:00 -0700). The last number indicates the timezone offset from GMT.
{format}t The time, in the form given by format, which should be in strftime(3) format (potentially localized). If the format starts with begin: (the default) the time is taken at the beginning of the request processing. If it starts with end: it is the time when the log entry gets written, close to the end of the request processing. In addition to the formats supported by strftime(3), the following format tokens are supported: sec (number of seconds since the Epoch), msec (number of milliseconds since the Epoch), usec (number of microseconds since the Epoch), msec_frac (millisecond fraction), and usec_frac (microsecond fraction).
T The time taken to serve the request, in seconds.
u The remote user if the request was authenticated. May be bogus if return status (%s) is 401 (unauthorized). Always returns "-".
U The URL path requested, not including any query string.
v The canonical ServerName of the server serving the request.
V The server name according to the UseCanonicalName setting.
{vcl}V The literal VCL to include without quoting (a Fastly extension).
X The connection status when response is completed. Statuses include X (connection aborted before the response completed), + (connection may be kept alive after the response is sent), and - (connection will be closed after the response is sent).

Upgrading endpoints to use version 2 log format

To upgrade a logging endpoints, run the following command:

curl -X PUT -H 'Fastly-Key: <your Fastly API key>' -H 'Content-Type: application/json' 'https://api.fastly.com/service/<your Fastly service ID>/version/<version number/logging/<log type>/<log name>' --data-binary '{"format_version":"2"}'

Where log type is type of the endpoint you want to upgrade i.e one of

Keep in mind that the format_version field is a per-object field. Updating it on one logging object will not change it on any other objects. For example, to upgrade a Google Cloud Storage endpoint the cURL command would look something like this if the endpoint was named "GCS Test":

curl -X PUT -H 'Fastly-Key: d3cafb4dde4dbeef' -H 'Content-Type: application/json' 'https://api.fastly.com/service/SU1Z0isxPaozGVKXdv0eY/version/1/logging/gcs/GCS%20Test' --data-binary '{"format_version":"2"}'

Determining which logging version is being used

To determine which logging version your service currently uses, issue the following cURL command:

curl -X GET -H 'Fastly-Key: <your Fastly API key>'  'https://api.fastly.com/service/<your Fastly service ID>/version/<version number>/logging/<log type>/<log name>'

where <log type> is ftp, heroku, logentries, loggly, logshuttle, papertrail, s3, scalyr, sftp, sumologic, or syslog. The cURL command will produce JSON output detailing the configuration of your service version. For example:

{
    "address": "logs.papertrailapp.com",
    "created_at": "2016-04-01T15:37:30+00:00",
    "deleted_at": null,
    "format": "time.start.msec time.to_first_byte time.elapsed req.body_bytes_read req.bytes_read resp.http.content-length server.region client.ip %>s \"req.request req.url req.proto\" \"req.http.referer\" \"req.http.user-agent\"",
    "format_version": "2",
    "hostname": "logs.papertrailapp.com",
    "name": "fastly",
    "port": "11111",
    "public_key": null,
    "response_condition": "LOG /",
    "service_id": "1a2b3c4d5e6f7g8h9j0k",
    "updated_at": "2016-04-01T19:47:47+00:00",
    "version": "123"
}

The format_version field displays either a 1 or a 2 as appropriate for the custom log format being used.

Advantages of using the version 2 custom log format

The key advantages of using the version 2 custom log format include the following:

Making version 2 logs look like version 1

The default logging format for version 1 is as follows:

%h %l %u %t %r %>s

Most of the directives in version 2 are exactly the same - only %t and %r are different. After you upgrade to version 2 log formats, you can recreate the appearance of version 1 logs using the new %{...}V directive, which allows you to specifically include VCL in logging directives:

%h %l %u %{now}V %{req.request}V %{req.url}V %>s

In addition, if you are using the %b directive in version 1, then you can use this directive instead:

%{resp.http.Content-Length}V


Additional resources:


Back to Top