Getting started with surrogate keys
Last updated 2020-04-27
Efficient cache invalidation is an essential part of keeping your website fast. Purging too much cache using purge all may increase your website's load time while the cache rebuilds. If you find yourself purging all cache on more than a weekly basis, consider using surrogate keys for more targeted purging.
Surrogate keys are unique identifiers that you assign to groups of content for processing. They allow you to selectively purge related content. Using the
Surrogate-Key header, you can tag a group of objects with a key term, a string of any characters you want, and then use that term to purge multiple pieces of content at once. This process can occur automatically within your application, making it easier to cache and purge content that changes rapidly and unpredictably.
Understanding surrogate keys
After you've signed up for Fastly and added one or more services, you can start examining how your origin server responds to requests. When your origin server responds to an HTTP request for content, it's because Fastly hasn't yet cached that content or the cache has expired. Your server's response to the request will resemble the example shown below. (Note that you can use the curl command to inspect any of your server's responses.)
1 2 3 4 HTTP/1.1 200 OK Content-Type: text/html Connection: keep-alive ...
To control how your content is served to users and cached by Fastly, you can add to or modify the headers that are included in your origin server's response. The
Surrogate-Key header is one of the headers that you can add to the response. It allows you to "tag" an object, such as an image or a blog post, with one or more keys. When the object changes, you can reference the key in a purge request to remove the object from the cache.
You can add space-delimited strings to the
Surrogate-Key header, like this:
1 2 3 4 HTTP/1.1 200 OK Surrogate-Key: key1 key2 key3 Content-Type: text/html ...
This response contains three surrogate keys:
key3. When Fastly receives a response like this, we use the surrogate keys to create a mapping from each key to the cached content, then we strip out the
Surrogate-Key header so it's not included in the response to your readers.
You can't use duplicate surrogate keys. For example, if you tried to use
foo, the two
foos would be collapsed into a single instance of the term.
Creating relationships between keys and objects
One of the major advantages of surrogate keys is that they allow for a many-to-many relationship between keys and objects. An origin server's response can associate multiple keys with the object and the same key can be provided in different responses. Take a look at these two requests and responses:
1 2 3 4 5 6 GET /blog/ HTTP/1.1 Host: www.example.com HTTP/1.1 200 OK Content-Type: text/html Content-Length: 1234 Surrogate-Key: mainpage template-a
1 2 3 4 5 6 7 GET /blog/article/fastly-rocks HTTP/1.1 Host: www.example.com HTTP/1.1 200 OK Content-Type: text/html Content-Length: 2345 Surrogate-Key: template-a article-fastly-rocks
In this example, there are two objects (
/blog/article/fastly-rocks) with three keys (
article-fastly-rocks). Two of the keys (
article-fastly-rocks) are associated with a single object and a third key (
template-a) is associated with both objects.
Purging objects with surrogate keys
By using the
Surrogate-Key header to associate keys with one or more objects, you can precisely control which objects are removed from cache during a purge. Consider the example presented above. Purging the
mainpage key would remove only the
/blog object from the cache. On the other hand, purging the
template-a key would remove both the
/blog/article/fastly-rocks objects from the cache.
You can use the Fastly web interface to manually purge objects via key, or you can use our Purge API. If you're using Fastly to cache your API, check out the guide on purging API cache with surrogate keys to learn how surrogate keys can help you.
Generating and setting surrogate keys
There are two ways to set the
Surrogate-Key header: by adding the header in the Fastly web interface, or by generating the keys with your own application. We describe how to use the Fastly web interface in our guide to generating Surrogate-Key headers based on URLs (we have a separate guide for Amazon S3 origins).
However, it is more useful to make your surrogate key associations on your own application server and include them in the HTTP response that you send to Fastly. This way, you can assign exactly the keys that you want on every response.
You can check the surrogate keys for a URL by using the
Fastly-Debug: 1 header. See the instructions on using a Fastly-Debug header with curl for more information.
The surrogate keys sent by your origin server can be as simple or complex as you need, subject to format limitations. Surrogate keys must be formatted as a single string without spaces. Spaces separate keys.
Surrogate keys are subject to size limitations. Individual surrogate keys may not exceed 1,024 bytes in length and
Surrogate-Key header values (comprising one or more space-separated keys) may not exceed 16,384 bytes in length. If either of the key or key header value limits are reached while parsing a
Surrogate-Key header, the key currently being parsed and all keys following it within the same header will be ignored.