- About the web interface controls
- Always-on DDoS mitigation
- Browser recommendations when using the Fastly web interface
- Content and its delivery
- Fastly POP locations
- Getting started with Fastly
- How caching and CDNs work
- How Fastly's CDN Service works
- HTTP status codes cached by default
- Self-provisioned Fastly services
- Sign up and create your first service
- Working with services
Domains & Origins
Domains & Origins
- Changing origins based on user location
- Connecting to origins
- Enabling global POPs
- Failover configuration
- IPv6 support
- Maintaining separate HTTP and HTTPS requests to origin servers
- Routing assets to different origins
- Setting up redundant origin servers
- Specifying an override host
- Using Fastly with apex domains
- About Dynamic Servers
- Authenticating URL purge requests via API
- Cache control tutorial
- Caching configuration best practices
- Controlling caching
- Creating and using pools with Dynamic Servers
- Creating and using server entries with Dynamic Servers
- Enabling API caching
- Enabling automatic gzipping
- Failure modes with large files
- Getting started with surrogate keys
- HTTP/2 server push
- Implementing API cache control
- Large File Support
- Logging purge requests
- Making query strings agnostic
- Purging API cache with surrogate keys
- Request collapsing
- Serving stale content
- Setting Surrogate-Key headers based on a URL
- Setting Surrogate-Key headers for Amazon S3 origins
- Single purges
- Soft purges
- Streaming Miss
- Wildcard purges
- Accept-Language header VCL features
- Authenticating before returning a request
- Basic authentication
- Creating location-based tagging
- Custom responses that don't hit origin servers
- Delivering different content to different devices
- Enabling URL token validation
- Guide to VCL
- Isolating header values without regular expressions
- Manipulating the cache key
- IP geolocation variables: Migrating to the new dataset
- Overriding which IP address the geolocation features use
- Response Cookie handling
- Support for the Edge-Control header
- Understanding the different PASS action behaviors
- Using edge side includes (ESI)
- VCL regular expression cheat sheet
Access Control Lists
Monitoring and testing
Web Application Firewall
- Log streaming: Amazon S3
- Log streaming: Microsoft Azure Blob Storage
- Log streaming: Cloud Files
- Log streaming: Datadog
- Log streaming: DigitalOcean Spaces
- Log streaming: FTP
- Log streaming: Google BigQuery
- Log streaming: Google Cloud Storage
- Log streaming: Honeycomb
- Log streaming: Kafka
- Log streaming: Log Shuttle
- Log streaming: LogDNA
- Log streaming: Logentries
- Log streaming: Loggly
- Log streaming: Heroku's Logplex
- Log streaming: OpenStack
- Log streaming: Papertrail
- Log streaming: Scalyr
- Log streaming: SFTP
- Log streaming: Splunk
- Log streaming: Sumo Logic
- Log streaming: Syslog
User access and control
How caching and CDNs work
Last updated April 22, 2019
Fastly is a content delivery network, or CDN. CDNs work on the principle that once a piece of content has been generated it doesn't need to be generated again for a while so a copy can be kept around in a cache. Cache machines are optimized to serve small files very very quickly. CDNs typically have caches placed in datacenters all around the world. When a user requests information from a customer's site they're actually redirected to the set of cache machines closest to them instead of the customer's actual servers. This means that a European user going to an American site gets their content anywhere from 200-500ms faster. CDNs also minimize the effects of a cache miss. A cache miss occurs when a user requests a bit of content and it is not in the cache at that moment (because it's expired, because no-one has asked for it before, or because the cache got too full and old content was thrown out).
What can be cached?
At Fastly, our architecture (known as a reverse proxy) is designed to enable customers to go a step further and cache entire web pages for even more efficient handling of your traffic.
TIP: Static files + media objects + web pages = your whole site. With the right service configuration (which we can assist you in setting up) Fastly can reduce your backend traffic by orders of magnitude with no loss in control over the content your users see.
Managing the Cache
Caching serves as a powerful weapon in your make-the-site-faster arsenal. However, most objects in your cache aren't going to stay there permanently. They'll need to expire so that fresh content can be served. How long that content should stay in the cache might be mere seconds or a number of minutes or even a year or more.
How can you manage which of your content is cached, where, and for how long? By setting policies that control the cached data. Most caching policies are implemented as a set of HTTP headers sent with your content by the web server (as specified in the configuration or the application). These headers were designed with the client (browser) in mind but CDNs like Fastly will also use those headers as a guide on caching policy.
Expires header is the original cache-related HTTP header and tells the cache (typically a browser cache) how long to hang onto a piece of content. Thereafter, the browser will re-request the content from its source. The downside is that it's a static date and if you don't update it later, the date will pass and the browser will start requesting that resource from the source every time it sees it.
Fastly will respect the
Expires header value only if the
Cache-Control headers are not found in the request.
Cache-Control headers (introduced in the HTTP 1.1 specification) cover browser caches and in most cases, intermediate caches as well:
Cache-Control: public- Any cache can store a copy of the content.
Cache-Control: private- Don't store, this is for a single user.
Cache-Control: no-cache- Re-validate before serving this content.
Cache-Control: no-store- Don't ever store this content.
Cache-Control: public, max-age=[seconds]- Caches can store this content for n seconds.
Cache-Control: s-maxage=[seconds]- Same as max-age but applies specifically to proxy caches.
private Cache-Control headers will influence Fastly's caching. All other Cache-Control headers will not, but will be passed through to the browser. For more in-depth information about how Fastly responds to these Cache-Control headers and how these headers interact with Expires and Surrogate-Control, check out our Cache Control Tutorial .
NOTE: For more information on the rest of the Cache-Control headers, see the relevant section in Mark Nottingham's Caching Tutorial.
Surrogate headers are a relatively new addition to the cache management vocabulary (described in this W3C tech note). These headers provide a specific cache policy for proxy caches in the processing path.
Surrogate-Control accepts many of the same values as
Cache-Control, plus some other more esoteric ones (read the tech note for all the options).
One use of this technique is to provide conservative cache interactions to the browser (for example,
Cache-Control: no-cache). This causes the browser to re-validate with the source on every request for the content. This makes sure that the user is getting the freshest possible content. Simultaneously, a
Surrogate-Control header can be sent with a longer
max-age that lets a proxy cache in front of the source handle most of the browser traffic, only passing requests to the source when the proxy's cache expires.
With Fastly, one of the most useful
Surrogate headers is
Surrogate-Key. When Fastly processes a request and sees a
Surrogate-Key header, it uses the space-separated value as a list of tags to associate with the request URL in the cache. Combined with Fastly's Purge API an entire collection of URLs can be expired from the cache in one API call (and typically happens in around 1ms).
Surrogate-Control is the most specific.
Fastly and Cache Control Headers
Fastly looks for caching information in each of these headers as described in our Cache-Control docs. In order of preference:
When an object or collection of objects in the cache expires, the next time any of those objects are requested, the request is going to get passed through to your application. Generally, with a good caching strategy, this won't break things. However, when a popular object or collection of objects expires from the cache, your backend can be hit with a large influx of traffic as the cache nodes refetch the objects from the source.
In most cases, the object being fetched is not going to differ between requests, so why should every cache node have to get its own copy from the backend? With Shield Nodes, they don't have to. Shielding configured through the Fastly web interface allows you to select a specific datacenter (most efficiently, one geographically close to your application) to act as a shield node. When objects in the cache expire, the shield node is the only node to get the content from your source application. All other cache nodes will fetch from the shield node, reducing source traffic dramatically.
- Wikipedia: Reverse Proxy
- Fastly's Cache-Control docs
- Mark Nottingham's Caching Tutorial
- Surrogate header W3C tech note