Wildcard purges

Wildcard purging allows you to flush the cache of all pages under a directory branch or URL path; for example, you want to empty the cache of all pages under your "/service" path. Having to purge each URL one by one using the Fastly API or via the Fastly app is not very efficient.

Although Fastly does not have a specific wildcard purge function, you can implement the same behavior by making a small configuration change using surrogate keys. Surrogate keys allow you to tag a group of objects with a keyword (key) and then purge multiple pieces of content at once with it via the web interface or via custom VCL.

IMPORTANT

Purging will only apply to new objects as they're being put into the cache after you set up configuration changes. It will not apply to objects already in the cache when this configuration is being applied.

To purge content based on wildcard paths, follow the steps below.

Via the web interface

To purge content based on wildcard paths via the web interface, follow the steps below.

Create a default wildcard header

We set a default wildcard so that we have the flexibility to append other surrogate keys to a URL path.

  1. Log in to the Fastly web interface.
  2. From the Home page, select the appropriate service. You can use the search box to search by ID, name, or domain.
  3. Click the Edit configuration button and then select the option to clone the active version. The Domains page appears.
  4. Click the Content link. The Content page appears.
  5. Click the Create header button. The Create a header page appears.

    settings for creating a default wildcard header

  6. Fill out the Create a header fields as follows:

    • In the Name field, enter Default Wildcard. This name is displayed in the Fastly web interface.
    • From the Type menu, select Cache and from the Action menu, select Set.
    • In the Destination field, enter http.Surrogate-Key.
    • In the Source field, enter "".
    • From the Ignore if set menu, select No.
    • In the Priority field, enter 10.
  7. Click the Create button. A new header appears in the Headers area of the Content page.

Create headers for each wildcard path being purged

Next, create a header for each of the wildcard paths you need the ability to purge. For instance, you want to purge the wildcard path /*/foo.

  1. Click the Create header button to create another new header.

    a wildcard header for the path being purged

  2. Fill out the Create a header fields as follows:

    • In the Description field, enter /*/foo Wildcard. This name is displayed in the Fastly web interface.
    • From the Type menu, select Cache, and from the Action menu, select Append.
    • In the Destination field, enter http.Surrogate-Key.
    • In the Source field, enter " */foo". There is a space before the asterisk in the Source field, which is important when appending multiple surrogate keys to a URL.
    • From the Ignore if set menu, select No.
    • In the Priority field, enter 20.
  3. Click the Create button. A new header appears in the Headers area of the Content page.

Notice the Action is set to Append to add to the default wildcard surrogate key. The Priority is set to 20 so that the Default Wildcard header is executed first and then the wildcard path appends.

Create conditions for each wildcard path being purged

Finally, create a condition for each of the wildcard paths you need the ability to purge.

  1. Click the Attach a condition link next to the wildcard path header name. The Create a new cache condition window appears.

    the new condition window

  2. Fill out the Create a new cache condition fields as follows:

    • In the Name field, enter /*/foo Wildcard Condition.
    • In the Apply if field, enter req.url ~ "^/[^/]*/foo$".
  3. Click the Save and apply to button to create the new condition.

What does the condition mean? In the Apply if field above, the first "^" and "$" tells Fastly to look for the following pattern:

  • Start from the first slash after the request Host header.
  • There should be one directory.
  • It should be followed by the path /foo ending the URL.

Some examples would be /a/foo, /bar/foo, and /c/foo. You could also remove the first "^" and ">"$" to allow the condition to be more general so that the pattern can occur in the middle of a URL path.

Some other examples for URL wildcard conditions:

Apply if fieldMatched pattern
req.url ~ "/[^/]*/foo"/delta/wow/a/foo/neat/cool/img.gif
req.url ~ "^/.*/foo$"/a/b/c/d/e/f/foo

Purge the wildcard

Ready to purge that wildcard? You can do this through the web interface using the steps below.

  1. Log in to the Fastly web interface.
  2. From the Purge menu, select Purge Key.

    the purge menu

    The Purge Key window appears.

  3. In the Keys field, enter the surrogate key you want to purge. Continuing with our example, you would enter */foo without the quotes that were entered in the Source field of the New Header window above.

    the purge key window

  4. Click the Purge button.

Via custom VCL

To purge content based on wildcard paths via custom VCL, follow the steps below.

  1. Add the following code to the VCL template:

    1sub construct_skey {
    2 if (req.url.path ~ "^(((((/[^/]+)?/[^/]+)?/[^/]+)?/[^/]+)?/[^/]+)") {
    3 # This prevents us from doing this twice when shielding
    4 if (std.strstr(beresp.http.Surrogate-Key, re.group.1)) {
    5 return;
    6 }
    7
    8 if (!re.group.2) {
    9 set beresp.http.Surrogate-Key = if(beresp.http.Surrogate-Key, beresp.http.Surrogate-Key " ", "")
    10 + re.group.1;
    11 return;
    12 }
    13
    14 if (!re.group.3) {
    15 set beresp.http.Surrogate-Key = if(beresp.http.Surrogate-Key, beresp.http.Surrogate-Key " ", "")
    16 + re.group.1 + " " + re.group.2;
    17 return;
    18 }
    19
    20 if (!re.group.4) {
    21 set beresp.http.Surrogate-Key = if(beresp.http.Surrogate-Key, beresp.http.Surrogate-Key " ", "")
    22 + re.group.1 + " " + re.group.2 + " " + re.group.3;
    23 return;
    24 }
    25
    26 if (!re.group.5) {
    27 set beresp.http.Surrogate-Key = if(beresp.http.Surrogate-Key, beresp.http.Surrogate-Key " ", "")
    28 + re.group.1 + " " + re.group.2 + " " + re.group.3 + " " + re.group.4;
    29 return;
    30 }
    31 set beresp.http.Surrogate-Key = if(beresp.http.Surrogate-Key, beresp.http.Surrogate-Key " ", "")
    32 + re.group.1 + " " + re.group.2 + " " + re.group.3 + " " + re.group.4 + " " + re.group.5;
    33 }
    34}
  2. Call the subroutine in vcl_fetch:

    1sub vcl_fetch {
    2call construct_skey;
    3}
  3. Check your success by curling an object not already in cache with the Fastly-Debug:1 header to expose the surrogate keys. For example:

    1$ curl -svo /dev/null http://www.example.com/test/test2/file3.txt -H Fastly-Debug:1
    2* Trying 192.0.2.0...
    3* Connected to www.example.com (192.0.2.0) port 80 (#0)
    4> Host: www.example.com
    5> User-Agent: curl/7.43.0
    6> Accept: */*
    7> Fastly-Debug:1
    8>
    9< HTTP/1.1 200 OK
    10< Server: Apache
    11< Content-Type: text/plain
    12< Surrogate-Key: /test /test/test2 /test/test2/file3.txt
    13< Via: 1.1 varnish
    14< X-Backend-IP: 203.0.113.0
    15< Cache-Control: max-age=31536000, stale-while-revalidate=31536000, stale-if-error=31536000
    16< Content-Length: 19
    17< Accept-Ranges: bytes
    18< Date: Fri, 29 Jan 2016 21:30:08 GMT
    19< Via: 1.1 varnish
    20< Age: 1035
    21< Connection: keep-alive
    22< Fastly-Debug-Path: (D cache-sjc3123-SJC 1454103008) (F cache-sjc3134-SJC 1454101973) (D cache-den6026-DEN 1454101973) (F cache-den6027-DEN 1454101973)
    23< Fastly-Debug-TTL: (H cache-sjc3123-SJC - - 1035) (M cache-den6026-DEN - - 0)
    24< Fastly-Debug-Digest: b43bd38cf940e1669c2927c8662660e5170758053dda42e772ce3fc34ee57fc1
    25< X-Served-By: cache-den6026-DEN, cache-sjc3123-SJC
    26< X-Cache: MISS, HIT
    27< X-Cache-Hits: 0, 1
    28< Vary: Accept-Encoding
    29<
    30{ [19 bytes data]
    31* Connection #0 to host www.example.com left intact

    In the above example, the < Surrogate-Key: /test /test/test2 /test/test2/file3.txt headers show the addition of the three surrogate keys.

Via the API

You can also use our key-based purging via the API to perform wildcard purging using an HTTP request:

POST /service/<Fastly Service ID>/purge/*/foo
Fastly-Key: FASTLY_API_TOKEN

This will purge any content that was associated with the "*/foo" surrogate key according to the setup in your header rules. Additional syntax for purging a service through the API can be found in the Purging section of the API documentation.

Was this guide helpful?

Do not use this form to send sensitive information. If you need assistance, contact support.