- About the web interface controls
- Always-on DDoS mitigation
- Browser recommendations when using the Fastly web interface
- Content and its delivery
- Fastly POP locations
- Getting started with Fastly
- How caching and CDNs work
- How Fastly's CDN Service works
- HTTP status codes cached by default
- Self-provisioned Fastly services
- Sign up and create your first service
- Working with services
Domains & Origins
Domains & Origins
- Changing origins based on user location
- Connecting to origins
- Enabling global POPs
- Failover configuration
- IPv6 support
- Maintaining separate HTTP and HTTPS requests to origin servers
- Routing assets to different origins
- Setting up redundant origin servers
- Specifying an override host
- Using Fastly with apex domains
- About Dynamic Servers
- Authenticating URL purge requests via API
- Cache control tutorial
- Caching configuration best practices
- Controlling caching
- Creating and using pools with Dynamic Servers
- Creating and using server entries with Dynamic Servers
- Enabling API caching
- Enabling automatic gzipping
- Failure modes with large files
- Getting started with surrogate keys
- HTTP/2 server push
- Implementing API cache control
- Large File Support
- Logging purge requests
- Making query strings agnostic
- Purging API cache with surrogate keys
- Request collapsing
- Serving stale content
- Setting Surrogate-Key headers based on a URL
- Setting Surrogate-Key headers for Amazon S3 origins
- Single purges
- Soft purges
- Streaming Miss
- Wildcard purges
- Accept-Language header VCL features
- Authenticating before returning a request
- Basic authentication
- Creating location-based tagging
- Custom responses that don't hit origin servers
- Delivering different content to different devices
- Enabling URL token validation
- Guide to VCL
- Isolating header values without regular expressions
- Manipulating the cache key
- IP geolocation variables: Migrating to the new dataset
- Overriding which IP address the geolocation features use
- Response Cookie handling
- Support for the Edge-Control header
- Understanding the different PASS action behaviors
- Using edge side includes (ESI)
- VCL regular expression cheat sheet
Access Control Lists
Monitoring and testing
Web Application Firewall
- Log streaming: Amazon S3
- Log streaming: Microsoft Azure Blob Storage
- Log streaming: Cloud Files
- Log streaming: Datadog
- Log streaming: DigitalOcean Spaces
- Log streaming: Elasticsearch
- Log streaming: FTP
- Log streaming: Google BigQuery
- Log streaming: Google Cloud Storage
- Log streaming: Honeycomb
- Log streaming: Kafka
- Log streaming: Log Shuttle
- Log streaming: LogDNA
- Log streaming: Logentries
- Log streaming: Loggly
- Log streaming: Heroku's Logplex
- Log streaming: OpenStack
- Log streaming: Papertrail
- Log streaming: Scalyr
- Log streaming: SFTP
- Log streaming: Splunk
- Log streaming: Sumo Logic
- Log streaming: Syslog
User access and control
Log streaming: Kafka
Last updated June 18, 2019
IMPORTANT: This feature is part of a limited availability release. For more information, see our product and feature lifecycle descriptions.
NOTE: This logging endpoint is disabled by default. To enable this endpoint for your account, contact firstname.lastname@example.org and request it.
NOTE: Fastly does not provide direct support for third-party services. See Fastly's Terms of Service for more information.
Before adding Apache Kafka as a logging endpoint for Fastly services, ensure Kafka is running on a remote server. You’ll need to know the hostname or IP address of one or more servers (Brokers) and the category or feed name to which messages will be stored (Topic). For more information on setting up Kafka see the Apache Kafka Quickstart guide.
Adding Kafka as a logging endpoint
Follow these instructions to add Kafka as a logging endpoint:
- Review the information in our Setting Up Remote Log Streaming guide.
Click the Apache Kafka logo. The Create an Apache Kafka endpoint page appears.
- Fill out the Create an Apache Kafka endpoint fields as follows:
- In the Name field, type a human-readable name for the endpoint.
- In the Log format field, optionally type an Apache-style string or VCL variables to use for log formatting. The Apache Common Log format string appears in this field by default. See our guidance on format strings for more information.
- In the Brokers field, type the hostname or IP address of one or more servers (Kafka brokers). Specify multiple servers using a comma-separated string.
- In the Topic field, type the name of the topic to send logs to.
- In the Write acknowledgement area, select the appropriate write acknowledgement a leader must receive before a write is considered successful.
- In the Compression codec area, select the appropriate codec to use for compression of your logs.
- From the TLS menu, select No to disable encryption for the Kafka endpoint, or Yes to enable it. When you select Yes, additional TLS fields appear.
- In the TLS Hostname field, optionally type the hostname used to verify the server's certificate. This can be either the Common Name (CN) or Subject Alternate Name (SAN). If the hostname is not specified, the hostname of the first broker in the Brokers field will be used. This field only appears when you select Yes from the Use TLS menu.
- In the TLS CA certificate field, optionally copy and paste the Certificate Authority (CA) certificate used to verify that the origin server's certificate is valid. The certificate you upload must be in PEM format. Consider uploading the certificate if it's not signed by a well-known certificate authority. This value is not required if your TLS certificate is signed by a well-known authority. This field only appears when you select Yes from the Use TLS menu.
- In the TLS client certificate field, optionally copy and paste the TLS client certificate used to authenticate to the origin server. The TLS client certificate you upload must be in PEM format and must be accompanied by a client certificate. A TLS client certificate allows your server to authenticate that Fastly is performing the connection. This field only appears when you select Yes from the Use TLS menu.
- In the TLS client key field, optionally copy and paste the TLS client key used to authenticate to the backend server. The TLS client key you upload must be in PEM format and must be accompanied by a TLS client certificate. A TLS client key allows your server to authenticate that Fastly is performing the connection. This field only appears when you select Yes from the Use TLS menu.
Click the Advanced options link of the Create an Apache Kafka endpoint page. The Advanced options appear.
- In the Placement area, select where the logging call should be placed in the generated VCL. Valid values are Format Version Default, None, and waf_debug (waf_debug_log). Selecting None creates a logging object that can only be used in custom VCL. See our guide on WAF logging for more information about
- Click the Create button to create the new logging endpoint.
- Click the Activate button to deploy your configuration changes.