LOG IN SIGN UP
Documentation

Log streaming: Microsoft Azure Blob Storage

  Last updated October 31, 2018

Fastly's Real-Time Log Streaming feature can send log files to Microsoft Azure Blob Storage (Blob Storage). Blob Storage is a static file storage service used to control arbitrarily large amounts of unstructured data and serve them to users over HTTP and HTTPS.

Prerequisites

Before adding Blob Storage as a logging endpoint for Fastly services, create an Azure storage account in the Azure portal. For help creating the account, see Microsoft's account creation documentation.

We recommend creating a Shared Access Signature (SAS) user specifically for Fastly. For more information, see Microsoft's shared access signatures (SAS) documentation, paying specific attention to the Account SAS URI example.

Here is an example of a SAS token that provides write permissions to a blob:

sv=2018-04-05&st=2018-04-29T22%3A18%3A26Z&sr=b&se=2020-04-30T02%3A23%3A26Z&sp=w&sig=Z%2FRHIX5Xcg0Mq2rqI3OlWTjEg2tYkboXr1P9ZUXDtkk%3D

The table breaks down each part of the token to understand how it contributes to the SAS:

Element Example Description
sv sv=2018-04-05 Storage services version.
sr sr=b Store resources for which this token has access. We require blob (b).
st st=2018-04-29T22%3A18%3A26Z The start time of the token, specified in UTC.
se se=2020-04-30T02%3A23%3A26Z The expiry time of the token, specified in UTC.
Ensure you update your token before it expires or the logging functionality will not work.
sp sp=w The permissions granted by the SAS token. We require write (w).
sig sig=Z%2FRHIX5Xcg0Mq2... The signature to authorize access to the blob.

Adding Blob Storage as a logging endpoint

After you've registered for an Azure account and created a SAS token, follow these instructions to add Blob Storage as a logging endpoint:

  1. Review the information in our Setting Up Remote Log Streaming guide.
  2. Click the Azure Blob Storage logo. The Create a Microsoft Azure Blob Storage endpoint page appears.

    the create a Microsoft Azure Blob Storage endpoint page

  3. Fill out the Create a Microsoft Azure Blob Storage endpoint fields as follows:
    • In the Name field, type a human-readable name for the endpoint.
    • In the Storage account name field, type the unique Azure namespace in which your data objects will be stored.
    • In the Container field, type the name of the Blob Storage container to store logs in. See Microsoft's Blob storage page for more information.
    • In the SAS token field, type the token associated with the container.

    • In the Period field, optionally type an interval (in seconds) to control how frequently your log files are rotated. This value defaults to 3600 seconds.
    • In the Log format field, optionally type an Apache-style string or VCL variables to use for log formatting. The Apache Common Log format string appears in this field by default. See our guidance on format strings for more information.
    • In the Timestamp format field, optionally type a timestamp format for log files. The default is an strftime compatible string. Our guide on changing where log files are written provides more information.
  4. Click the Advanced options link of the Create a Microsoft Azure Blob Storage endpoint page and decide which of the optional fields to change, if any.

    the advanced options on the create a Microsoft Azure Blob Storage endpoint page

  5. Fill out the Advanced options of the Create a Microsoft Azure Blob Storage endpoint page as follows:
    • In the Path field, optionally type the path within the container to store the files. The path ends with a trailing slash. If this field is left empty, the files will be saved in the container's root path. Our guide on changing where log files are written provides more information.
    • In the PGP public key field, optionally type a PGP public key that Fastly will use to encrypt your log files before writing them to disk. You will only be able to read the contents by decrypting them with your private key. The PGP key should be in PEM (Privacy-Enhanced Mail) format. See our guide on log encryption for more information.
    • In the Select a log line format area, select the log line format for your log messages. Our guide on changing log line formats provides more information.
    • In the Gzip level field, optionally type the level of gzip compression you want applied to the log files. You can specify any whole number from 1 (fastest and least compressed) to 9 (slowest and most compressed). This value defaults to 0 (no compression).
  6. Click the Create button to create the new logging endpoint.
  7. Click the Activate button to deploy your configuration changes.

Ingesting data for Azure Data Explorer

Azure Data Explorer is a data exploration service for log and telemetry data. To ingest your data correctly, Data Explorer requires your logs to be formatted as comma-separated values (CSVs). When creating your logging endpoint:

Our guide on changing log line formats provides more information.


Additional resources:


Back to Top