Wildcard purges

Wildcard purging allows you to flush the cache of all pages under a directory branch or URL path; for example, you want to empty the cache of all pages under your "/service" path. Having to purge each URL one by one using the Fastly API or via the Fastly app is not very efficient.

Although Fastly does not have a specific wildcard purge function, you can implement the same behavior by making a small configuration change using surrogate keys. Surrogate keys allow you to tag a group of objects with a keyword (key) and then purge multiple pieces of content at once with it via the web interface or via custom VCL.

To purge content based on wildcard paths, follow the steps below.

Via the web interface

To purge content based on wildcard paths via the web interface, follow the steps below.

Create a default wildcard header

We set a default wildcard so that we have the flexibility to append other surrogate keys to a URL path.

1. Log in to the Fastly web interface.
2. From the Home page, select the appropriate service. You can use the search box to search by ID, name, or domain.
3. Click the Edit configuration button and then select the option to clone the active version. The Domains page appears.
4. Click the Content link. The Content page appears.

5. Click the Create header button. The Create a header page appears.

   

6. Fill out the Create a header fields as follows:

   • In the Name field, enter Default Wildcard. This name is displayed in the Fastly web interface.
   • From the Type menu, select Cache and from the Action menu, select Set.
   • In the Destination field, enter http.Surrogate-Key.
   • In the Source field, enter "".
   • From the Ignore if set menu, select No.
   • In the Priority field, enter 10.

7. Click the Create button. A new header appears in the Headers area of the Content page.

Create headers for each wildcard path being purged

Next, create a header for each of the wildcard paths you need the ability to purge. For instance, you want to purge the wildcard path /*/foo.

1. Click the Create header button to create another new header.

   

2. Fill out the Create a header fields as follows:

   • In the Description field, enter /*/foo Wildcard. This name is displayed in the Fastly web interface.
   • From the Type menu, select Cache, and from the Action menu, select Append.
   • In the Destination field, enter http.Surrogate-Key.
   • In the Source field, enter " */foo". There is a space before the asterisk in the Source field, which is important when appending multiple surrogate keys to a URL.
   • From the Ignore if set menu, select No.
   • In the Priority field, enter 20.

3. Click the Create button. A new header appears in the Headers area of the Content page.

Notice the Action is set to Append to add to the default wildcard surrogate key. The Priority is set to 20 so that the Default Wildcard header is executed first and then the wildcard path appends.

Create conditions for each wildcard path being purged

Finally, create a condition for each of the wildcard paths you need the ability to purge.

1. Click the Attach a condition link next to the wildcard path header name. The Create a new cache condition window appears.

   

2. Fill out the Create a new cache condition fields as follows:

   • In the Name field, enter /*/foo Wildcard Condition.
   • In the Apply if field, enter req.url ~ "^/[^/]*/foo$".

3. Click the Save and apply to button to create the new condition.

What does the condition mean? In the Apply if field above, the first "^" and "$" tells Fastly to look for the following pattern:

• Start from the first slash after the request Host header.
• There should be one directory.
• It should be followed by the path /foo ending the URL.

Some examples would be /a/foo, /bar/foo, and /c/foo. You could also remove the first "^" and ">"$" to allow the condition to be more general so that the pattern can occur in the middle of a URL path.

Some other examples for URL wildcard conditions:

Purge the wildcard

Ready to purge that wildcard? You can do this through the web interface using the steps below.

1. Log in to the Fastly web interface.

2. From the Purge menu, select Purge Key.

   

   The Purge Key window appears.

3. In the Keys field, enter the surrogate key you want to purge. Continuing with our example, you would enter */foo without the quotes that were entered in the Source field of the New Header window above.

   

4. Click the Purge button.

Via custom VCL

To purge content based on wildcard paths via custom VCL, follow the steps below.

1. Add the following code to the VCL template:

   1sub construct_skey {
   2  if (req.url.path ~ "^(((((/[^/]+)?/[^/]+)?/[^/]+)?/[^/]+)?/[^/]+)") {
   3    # This prevents us from doing this twice when shielding
   4    if (std.strstr(beresp.http.Surrogate-Key, re.group.1)) {
   5      return;
   6    }
   7
   8    if (!re.group.2) {
   9      set beresp.http.Surrogate-Key = if(beresp.http.Surrogate-Key, beresp.http.Surrogate-Key " ", "")
   10        + re.group.1;
   11      return;
   12    }
   13
   14    if (!re.group.3) {
   15      set beresp.http.Surrogate-Key = if(beresp.http.Surrogate-Key, beresp.http.Surrogate-Key " ", "")
   16        + re.group.1 + " " + re.group.2;
   17      return;
   18    }
   19
   20    if (!re.group.4) {
   21      set beresp.http.Surrogate-Key = if(beresp.http.Surrogate-Key, beresp.http.Surrogate-Key " ", "")
   22        + re.group.1 + " " + re.group.2 + " " + re.group.3;
   23      return;
   24    }
   25
   26    if (!re.group.5) {
   27      set beresp.http.Surrogate-Key = if(beresp.http.Surrogate-Key, beresp.http.Surrogate-Key " ", "")
   28        + re.group.1 + " " + re.group.2 + " " + re.group.3 + " " + re.group.4;
   29      return;
   30    }
   31    set beresp.http.Surrogate-Key = if(beresp.http.Surrogate-Key, beresp.http.Surrogate-Key " ", "")
   32      + re.group.1 + " " + re.group.2 + " " + re.group.3 + " " + re.group.4 + " " + re.group.5;
   33  }
   34}

2. Call the subroutine in vcl_fetch:

   1sub vcl_fetch {
   2call construct_skey;
   3}

3. Check your success by curling an object not already in cache with the Fastly-Debug:1 header to expose the surrogate keys. For example:

   1$ curl -svo /dev/null http://www.example.com/test/test2/file3.txt -H Fastly-Debug:1
   2* Trying 192.0.2.0...
   3* Connected to www.example.com (192.0.2.0) port 80 (#0)
   4> Host: www.example.com
   5> User-Agent: curl/7.43.0
   6> Accept: */*
   7> Fastly-Debug:1
   8>
   9< HTTP/1.1 200 OK
   10< Server: Apache
   11< Content-Type: text/plain
   12< Surrogate-Key: /test /test/test2 /test/test2/file3.txt
   13< Via: 1.1 varnish
   14< X-Backend-IP: 203.0.113.0
   15< Cache-Control: max-age=31536000, stale-while-revalidate=31536000, stale-if-error=31536000
   16< Content-Length: 19
   17< Accept-Ranges: bytes
   18< Date: Fri, 29 Jan 2016 21:30:08 GMT
   19< Via: 1.1 varnish
   20< Age: 1035
   21< Connection: keep-alive
   22< Fastly-Debug-Path: (D cache-sjc3123-SJC 1454103008) (F cache-sjc3134-SJC 1454101973) (D cache-den6026-DEN 1454101973) (F cache-den6027-DEN 1454101973)
   23< Fastly-Debug-TTL: (H cache-sjc3123-SJC - - 1035) (M cache-den6026-DEN - - 0)
   24< Fastly-Debug-Digest: b43bd38cf940e1669c2927c8662660e5170758053dda42e772ce3fc34ee57fc1
   25< X-Served-By: cache-den6026-DEN, cache-sjc3123-SJC
   26< X-Cache: MISS, HIT
   27< X-Cache-Hits: 0, 1
   28< Vary: Accept-Encoding
   29<
   30{ [19 bytes data]
   31* Connection #0 to host www.example.com left intact

   In the above example, the < Surrogate-Key: /test /test/test2 /test/test2/file3.txt headers show the addition of the three surrogate keys.

Via the API

You can also use our key-based purging via the API to perform wildcard purging using an HTTP request:

POST /service/<Fastly Service ID>/purge/*/foo
Fastly-Key: FASTLY_API_TOKEN

This will purge any content that was associated with the "*/foo" surrogate key according to the setup in your header rules. Additional syntax for purging a service through the API can be found in the Purging section of the API documentation.