LOG IN SIGN UP
Documentation

Getting started with surrogate keys

  Last updated December 05, 2018

Efficient cache invalidation is an essential part of keeping your website fast. Purging too much cache using purge all may increase your website's load time while the cache rebuilds. If you find yourself purging all cache on more than a weekly basis, consider using surrogate keys for more targeted purging.

Surrogate keys allow you to selectively purge related content. Using the Surrogate-Key header, you can tag a group of objects with a key and then use it to purge multiple pieces of content at once. This process can occur automatically within your application, making it easier to cache and purge content that changes rapidly and unpredictably.

Understanding surrogate keys

After you've signed up for Fastly and added one or more services, you can start examining how your origin server responds to requests. When your origin server responds to an HTTP request for content, it's because Fastly hasn't yet cached that content or the cache has expired. Your server's response to the request will resemble the example shown below. (Note that you can use the curl command to inspect any of your server's responses.)

1
2
3
4
HTTP/1.1 200 OK
Content-Type: text/html
Connection: keep-alive
...

To control how your content is served to users and cached by Fastly, you can add to or modify the headers that are included in your origin server's response. The Surrogate-Key header is one of the headers that you can add to the response. It allows you to "tag" an object, such as an image or a blog post, with one or more keys. When the object changes, you can reference the key in a purge request to remove the object from the cache.

You can add space-delimited strings to the Surrogate-Key header, like this:

1
2
3
4
HTTP/1.1 200 OK
Surrogate-Key: key1 key2 key3
Content-Type: text/html
...

This response contains three surrogate keys: key1, key2, and key3. When Fastly receives a response like this, we use the surrogate keys to create a mapping from each key to the cached content, then we strip out the Surrogate-Key header so it's not included in the response to your readers.

Creating relationships between keys and objects

One of the major advantages of surrogate keys is that they allow for a many-to-many relationship between keys and objects. An origin server's response can associate multiple keys with the object and the same key can be provided in different responses. Take a look at these two requests and responses:

1
2
3
4
5
6
GET /blog/ HTTP/1.1
Host: www.example.com

HTTP/1.1 200 OK Content-Type: text/html
Content-Length: 1234
Surrogate-Key: mainpage template-a
1
2
3
4
5
6
7
GET /blog/article/fastly-rocks HTTP/1.1
Host: www.example.com

HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 2345
Surrogate-Key: template-a article-fastly-rocks

In this example, there are two objects (/blog and /blog/article/fastly-rocks) with three keys (mainpage, template-a, and article-fastly-rocks). Two of the keys (mainpage and article-fastly-rocks) are associated with a single object and a third key (template-a) is associated with both objects.

Purging objects with surrogate keys

By using the Surrogate-Key header to associate keys with one or more objects, you can precisely control which objects are removed from cache during a purge. Consider the example presented above. Purging the mainpage key would remove only the /blog object from the cache. On the other hand, purging the template-a key would remove both the /blog and /blog/article/fastly-rocks objects from the cache.

You can use the Fastly web interface to manually purge objects via key, or you can use our Purge API. If you're using Fastly to cache your API, check out the guide on purging API cache with surrogate keys to learn how surrogate keys can help you.

Generating and setting surrogate keys

There are two ways to set the Surrogate-Key header: by adding the header in the Fastly web interface, or by generating the keys with your own application. We describe how to use the Fastly web interface in our guide to generating Surrogate-Key headers based on URLs (we have a separate guide for Amazon S3 origins).

However, it is more useful to make your surrogate key associations on your own application server and include them in the HTTP response that you send to Fastly. This way, you can assign exactly the keys that you want on every response.

Troubleshooting

You can check the surrogate keys for a URL by using the Fastly-Debug: 1 header. See the instructions on using a Fastly-Debug header with curl for more information.

Limitations

The surrogate keys sent by your origin server can be as simple or complex as you need, subject to size limitations. Individual surrogate keys may not exceed 1,024 bytes in length, and a Surrogate-Key header value (comprising one or more space-separated keys) must not exceed 16,384 bytes in length. If either of these limits is reached while parsing a Surrogate-Key header, the key currently being parsed and all keys following it within the same header will be ignored.


Back to Top