LOG IN SIGN UP
Documentation

Wildcard purges

Wildcard purging allows you to flush the cache of all pages under a directory branch or URL path; for example, you want to empty the cache of all pages under your "/service" path. Having to purge each URL one by one using the Fastly API or via the Fastly app is not very efficient.

Although Fastly does not have a specific wildcard purge function, you can implement the same behavior by making a small configuration change using surrogate keys. Surrogate keys allow you to tag a group of objects with a keyword (key) and then purge multiple pieces of content at once with it via the web interface or via custom VCL.

To purge content based on wildcard paths, follow the steps below.

Via the web interface

To purge content based on wildcard paths via the web interface, follow the steps below.

Create a default wildcard header

We set a default wildcard so that we have the flexibility to append other surrogate keys to a URL path.

  1. Log in to the Fastly web interface and click the Configure link.
  2. From the service menu, select the appropriate service.
  3. Click the Edit configuration button and then select Clone active. The service version page appears.
  4. Click the Content tab. The Content page appears.

    the Content page

  5. Click the Create header button. The Create a new header page appears.

    settings for creating a default wildcard header

  6. Fill out the Create a new header fields as follows:

    • From the Type menu, select Cache and from the Action menu, select Set.
    • In the Destination field, type http.Surrogate-Key.
    • In the Source field, type "".
    • From the Ignore if set menu, select No.
    • In the Priority field, type 10.
    • In the Description field, type Default Wildcard. This name is displayed in the Fastly web interface.
  7. Click the Create button. A new header appears in the Headers area of the Content page.

Create headers for each wildcard path being purged

Next, create a header for each of the wildcard paths you need the ability to purge. For instance, you want to purge the wildcard path /*/foo.

  1. Click the Create header button to create another new header.

    a wildcard header for the path being purged

  2. Fill out the Create a new header fields as follows:

    • From the Type menu, select Cache, and from the Action menu, select Append.
    • In the Destination field, type http.Surrogate-Key.
    • In the Source field, type " */foo". There is a space before the asterisk in the Source field, which is important when appending multiple surrogate keys to a URL.
    • From the Ignore if set menu, select No.
    • In the Priority field, type 20.
    • In the Description field, type /*/foo Wildcard. This name is displayed in the Fastly web interface.
  3. Click the Create button. A new header appears in the Headers area of the Content page.

Notice the Action is set to Append to add to the default wildcard surrogate key. The Priority is set to 20 so that the Default Wildcard header is executed first and then the wildcard path appends.

Create conditions for each wildcard path being purged

Finally, create a condition for each of the wildcard paths you need the ability to purge.

  1. Click the Attach a condition link next to the wildcard path header name. The Create a new cache condition window appears.

    the new condition window

  2. Fill out the Create a new cache condition fields as follows:

    • In the Name field, type /*/foo Wildcard Condition.
    • In the Apply if field, type req.url ~ "^/[^/]*/foo$".
  3. Click the Save and apply to button to create the new condition.

What does the condition mean? In the Apply if field above, the first "^" and "$" tells Fastly to look for the following pattern:

Some examples would be /a/foo, /bar/foo, and /c/foo. You could also remove the first "^" and ">"$" to allow the condition to be more general so that the pattern can occur in the middle of a URL path.

Some other examples for URL wildcard conditions:

Apply if field Matched pattern
req.url ~ "/[^/]*/foo" /delta/wow/a/foo/neat/cool/img.gif
req.url ~ "^/.*/foo$" /a/b/c/d/e/f/foo

Purge the wildcard

Ready to purge that wildcard? You can do this through the UI using the steps below.

  1. Log in to the Fastly web interface and click the Configure link.
  2. From the Purge menu, select Purge Key.

    the purge drop down

    The Purge Key window appears.

  3. In the Keys field, type the surrogate key you want to purge. Continuing with our example, you would type */foo without the quotes that were entered in the Source field of the New Header window above.

    the purge key window

  4. In the Password field, type your Fastly password.

  5. Click the Purge button.

Via custom VCL

To purge content based on wildcard paths via custom VCL, follow the steps below.

  1. Add the following code to the VCL template:
   sub construct_skey {
     if (req.url ~ "^(/[^/\?]*)(/[^/\?]*)?(/[^/\?]*)?(/[^/\?]*)?(/[^/\?]*)?" && !std.strstr(beresp.http.Surrogate-Key,re.group.1)) {
       if (!re.group.1) {
         return;
       }
       set req.http.skey = re.group.1;
       set beresp.http.Surrogate-Key = if(beresp.http.Surrogate-Key, beresp.http.Surrogate-Key " ", "") req.http.skey;

       if (!re.group.2) {
         return;
       }
       set req.http.skey = req.http.skey re.group.2;
       set beresp.http.Surrogate-Key = beresp.http.Surrogate-Key " " req.http.skey;

       if (!re.group.3) {
         return;
       }
       set req.http.skey = req.http.skey re.group.3;
       set beresp.http.Surrogate-Key = beresp.http.Surrogate-Key " " req.http.skey;

       if (!re.group.4) {
         return;
       }
       set req.http.skey = req.http.skey re.group.4;
       set beresp.http.Surrogate-Key = beresp.http.Surrogate-Key " " req.http.skey;

       if (!re.group.5) {
         return;
       }
       set req.http.skey = req.http.skey re.group.5;
       set beresp.http.Surrogate-Key = beresp.http.Surrogate-Key " " req.http.skey;
     }
   }
  1. Call the subroutine in vcl_fetch:
   sub vcl_fetch {
   call construct_skey;
   }
  1. Check your success by curling an object not already in cache with the Fastly-Debug:1 header to expose the surrogate keys. For example:
   $ curl -svo /dev/null http://www.example.com/test/test2/file3.txt -HFastly-Debug:1
   * Trying 192.0.2.0...
   * Connected to www.example.com (192.0.2.0) port 80 (#0)
   > Host: www.example.com
   > User-Agent: curl/7.43.0
   > Accept: */*
   > Fastly-Debug:1
   >
   < HTTP/1.1 200 OK
   < Server: Apache
   < Content-Type: text/plain
   < Surrogate-Key: /test /test/test2 /test/test2/file3.txt
   < Via: 1.1 varnish
   < X-Backend-IP: 203.0.113.0
   < Cache-Control: max-age=31536000, stale-while-revalidate=31536000, stale-if-error=31536000
   < Content-Length: 19
   < Accept-Ranges: bytes
   < Date: Fri, 29 Jan 2016 21:30:08 GMT
   < Via: 1.1 varnish
   < Age: 1035
   < Connection: keep-alive
   < Fastly-Debug-Path: (D cache-sjc3123-SJC 1454103008) (F cache-sjc3134-SJC 1454101973) (D cache-den6026-DEN 1454101973) (F cache-den6027-DEN 1454101973)
   < Fastly-Debug-TTL: (H cache-sjc3123-SJC - - 1035) (M cache-den6026-DEN - - 0)
   < Fastly-Debug-Digest: b43bd38cf940e1669c2927c8662660e5170758053dda42e772ce3fc34ee57fc1
   < X-Served-By: cache-den6026-DEN, cache-sjc3123-SJC
   < X-Cache: MISS, HIT
   < X-Cache-Hits: 0, 1
   < Vary: Accept-Encoding
   <
   { [19 bytes data]
   * Connection #0 to host www.example.com left intact

In the above example, the < Surrogate-Key: /test /test/test2 /test/test2/file3.txt headers show the addition of the three surrogate keys.

Via the API

You can also use our key-based purging via the API to perform wildcard purging using an HTTP request:

POST /service/<Fastly Service ID>/purge/*/foo
Fastly-Key: <Fastly Key>

This will purge any content that was associated with the "*/foo" surrogate key according to the setup in your header rules. Additional syntax for purging a service through the API can be found in the Purging section of the API documentation.


Back to Top