Wildcard purges
Last updated 2019-07-15
Wildcard purging allows you to flush the cache of all pages under a directory branch or URL path; for example, you want to empty the cache of all pages under your "/service" path. Having to purge each URL one by one using the Fastly API or via the Fastly app is not very efficient.
Although Fastly does not have a specific wildcard purge function, you can implement the same behavior by making a small configuration change using surrogate keys. Surrogate keys allow you to tag a group of objects with a keyword (key) and then purge multiple pieces of content at once with it via the web interface or via custom VCL.
IMPORTANT
Purging will only apply to new objects as they're being put into the cache after you set up configuration changes. It will not apply to objects already in the cache when this configuration is being applied.
To purge content based on wildcard paths, follow the steps below.
Via the web interface
To purge content based on wildcard paths via the web interface, follow the steps below.
Create a default wildcard header
We set a default wildcard so that we have the flexibility to append other surrogate keys to a URL path.
- Log in to the Fastly web interface.
- From the Home page, select the appropriate service. You can use the search box to search by ID, name, or domain.
- Click the Edit configuration button and then select the option to clone the active version. The Domains page appears.
- Click the Content link. The Content page appears.
Click the Create header button. The Create a header page appears.
Fill out the Create a header fields as follows:
- In the Name field, enter
Default Wildcard
. This name is displayed in the Fastly web interface. - From the Type menu, select Cache and from the Action menu, select Set.
- In the Destination field, enter
http.Surrogate-Key
. - In the Source field, enter
""
. - From the Ignore if set menu, select No.
- In the Priority field, enter
10
.
- In the Name field, enter
Click the Create button. A new header appears in the Headers area of the Content page.
Create headers for each wildcard path being purged
Next, create a header for each of the wildcard paths you need the ability to purge. For instance, you want to purge the wildcard path /*/foo
.
Click the Create header button to create another new header.
Fill out the Create a header fields as follows:
- In the Description field, enter
/*/foo Wildcard
. This name is displayed in the Fastly web interface. - From the Type menu, select Cache, and from the Action menu, select Append.
- In the Destination field, enter
http.Surrogate-Key
. - In the Source field, enter
" */foo"
. There is a space before the asterisk in the Source field, which is important when appending multiple surrogate keys to a URL. - From the Ignore if set menu, select No.
- In the Priority field, enter
20
.
- In the Description field, enter
Click the Create button. A new header appears in the Headers area of the Content page.
Notice the Action is set to Append to add to the default wildcard surrogate key. The Priority is set to 20 so that the Default Wildcard header is executed first and then the wildcard path appends.
Create conditions for each wildcard path being purged
Finally, create a condition for each of the wildcard paths you need the ability to purge.
Click the Attach a condition link next to the wildcard path header name. The Create a new cache condition window appears.
Fill out the Create a new cache condition fields as follows:
- In the Name field, enter
/*/foo Wildcard Condition
. - In the Apply if field, enter
req.url ~ "^/[^/]*/foo$"
.
- In the Name field, enter
Click the Save and apply to button to create the new condition.
What does the condition mean? In the Apply if field above, the first "^"
and "$"
tells Fastly to look for the following pattern:
- Start from the first slash after the request Host header.
- There should be one directory.
- It should be followed by the path
/foo
ending the URL.
Some examples would be /a/foo
, /bar/foo
, and /c/foo
. You could also remove the first "^"
and ">"$"
to allow the condition to be more general so that the pattern can occur in the middle of a URL path.
Some other examples for URL wildcard conditions:
Apply if field | Matched pattern |
---|---|
req.url ~ "/[^/]*/foo" | /delta/wow/a/foo/neat/cool/img.gif |
req.url ~ "^/.*/foo$" | /a/b/c/d/e/f/foo |
Purge the wildcard
Ready to purge that wildcard? You can do this through the web interface using the steps below.
- Log in to the Fastly web interface.
From the Purge menu, select Purge Key.
The Purge Key window appears.
In the Keys field, enter the surrogate key you want to purge. Continuing with our example, you would enter
*/foo
without the quotes that were entered in the Source field of the New Header window above.Click the Purge button.
Via custom VCL
To purge content based on wildcard paths via custom VCL, follow the steps below.
Add the following code to the VCL template:
1sub construct_skey {2 if (req.url.path ~ "^(((((/[^/]+)?/[^/]+)?/[^/]+)?/[^/]+)?/[^/]+)") {3 # This prevents us from doing this twice when shielding4 if (std.strstr(beresp.http.Surrogate-Key, re.group.1)) {5 return;6 }78 if (!re.group.2) {9 set beresp.http.Surrogate-Key = if(beresp.http.Surrogate-Key, beresp.http.Surrogate-Key " ", "")10 + re.group.1;11 return;12 }1314 if (!re.group.3) {15 set beresp.http.Surrogate-Key = if(beresp.http.Surrogate-Key, beresp.http.Surrogate-Key " ", "")16 + re.group.1 + " " + re.group.2;17 return;18 }1920 if (!re.group.4) {21 set beresp.http.Surrogate-Key = if(beresp.http.Surrogate-Key, beresp.http.Surrogate-Key " ", "")22 + re.group.1 + " " + re.group.2 + " " + re.group.3;23 return;24 }2526 if (!re.group.5) {27 set beresp.http.Surrogate-Key = if(beresp.http.Surrogate-Key, beresp.http.Surrogate-Key " ", "")28 + re.group.1 + " " + re.group.2 + " " + re.group.3 + " " + re.group.4;29 return;30 }31 set beresp.http.Surrogate-Key = if(beresp.http.Surrogate-Key, beresp.http.Surrogate-Key " ", "")32 + re.group.1 + " " + re.group.2 + " " + re.group.3 + " " + re.group.4 + " " + re.group.5;33 }34}Call the subroutine in
vcl_fetch
:1sub vcl_fetch {2call construct_skey;3}Check your success by curling an object not already in cache with the
Fastly-Debug:1
header to expose the surrogate keys. For example:1$ curl -svo /dev/null http://www.example.com/test/test2/file3.txt -H Fastly-Debug:12* Trying 192.0.2.0...3* Connected to www.example.com (192.0.2.0) port 80 (#0)4> Host: www.example.com5> User-Agent: curl/7.43.06> Accept: */*7> Fastly-Debug:18>9< HTTP/1.1 200 OK10< Server: Apache11< Content-Type: text/plain12< Surrogate-Key: /test /test/test2 /test/test2/file3.txt13< Via: 1.1 varnish14< X-Backend-IP: 203.0.113.015< Cache-Control: max-age=31536000, stale-while-revalidate=31536000, stale-if-error=3153600016< Content-Length: 1917< Accept-Ranges: bytes18< Date: Fri, 29 Jan 2016 21:30:08 GMT19< Via: 1.1 varnish20< Age: 103521< Connection: keep-alive22< Fastly-Debug-Path: (D cache-sjc3123-SJC 1454103008) (F cache-sjc3134-SJC 1454101973) (D cache-den6026-DEN 1454101973) (F cache-den6027-DEN 1454101973)23< Fastly-Debug-TTL: (H cache-sjc3123-SJC - - 1035) (M cache-den6026-DEN - - 0)24< Fastly-Debug-Digest: b43bd38cf940e1669c2927c8662660e5170758053dda42e772ce3fc34ee57fc125< X-Served-By: cache-den6026-DEN, cache-sjc3123-SJC26< X-Cache: MISS, HIT27< X-Cache-Hits: 0, 128< Vary: Accept-Encoding29<30{ [19 bytes data]31* Connection #0 to host www.example.com left intactIn the above example, the
< Surrogate-Key: /test /test/test2 /test/test2/file3.txt
headers show the addition of the three surrogate keys.
Via the API
You can also use our key-based purging via the API to perform wildcard purging using an HTTP request:
POST /service/<Fastly Service ID>/purge/*/fooFastly-Key: FASTLY_API_TOKEN
This will purge any content that was associated with the "*/foo"
surrogate key according to the setup in your header rules. Additional syntax for purging a service through the API can be found in the Purging section of the API documentation.
Do not use this form to send sensitive information. If you need assistance, contact support.