Amazon S3

Amazon S3 public and private buckets can be used as origins with Fastly.

Using Amazon S3 as an origin

To make your S3 data buckets available through Fastly, follow the steps below.

Creating a new service

Follow the instructions for creating a new service.

1. When you create the new domain and the new Host:

   • In the Domain Name field on the Create a domain page, enter the hostname you want to use as the URL (e.g., cdn.example.com).
   • In the Hosts field on the Origins page, enter the appropriate address for your Host using the format <BUCKET>.s3.<REGION>.amazonaws.com. Use the table in the Amazon S3 endpoints of the AWS general reference documentation as a guide. For example, if your bucket name is fastlytestbucket and your region is us-east-2, your hostname would be fastlytestbucket.s3.us-east-2.amazonaws.com.
   TIP

   Most customers select a region close to the interconnect location they specify for shielding.

2. When you edit the Host details on the Edit this host page:

   • In the Name field, enter any descriptive name for your service if you haven't already done so.
   • In the Address field, ensure you've entered the appropriate address for your region (e.g., fastlytestbucket.s3.us-east-2.amazonaws.com). You entered this information during Host creation.

3. When you edit the Transport Layer Security (TLS) area information for your Host:

   • Leave the Enable TLS? default set to Yes to secure the connection between Fastly and your origin.

     TIP

     If you're using Amazon S3 to host a static website, Enable TLS? should be set to No instead of relying on the default. Amazon doesn't support TLS connections to S3 buckets with the static website hosting feature enabled. You can still use one of Fastly's TLS service options to secure connections between Fastly and clients.

   • Under the SNI hostname field, select the checkbox to Match the SNI hostname to the Certificate hostname. The address you entered during Host creation appears.

   • In the Certificate hostname field, enter fastlytestbucket.s3.us-east-2.amazonaws.com.

4. In the Override host field, enter an appropriate address for your Host (e.g., fastlytestbucket.s3.us-east-2.amazonaws.com). You entered this information during Host creation. If you're using an Amazon S3 private bucket leave this field blank.

Validating the DNS mapping

By default, we create a DNS mapping called yourdomain.global.prod.fastly.net. In the example above, it would be cdn.example.com.global.prod.fastly.net. Enter this URL in a browser to confirm it's working.

Creating a DNS alias

Create a DNS alias for the domain name you specified (e.g., CNAME cdn.example.com to global-nossl.fastly.net).

Verifying your results

Fastly will cache any content without an explicit Cache-Control header for 1 hour. You can verify whether you are sending any cache headers using curl. For example:

$ curl -I opscode-full-stack.s3.amazonaws.com

HTTP/1.1 200 OK
x-amz-id-2: ZpzRp7IWc6MJ8NtDEFGH12QBdk2CM1+RzVOngQbhMp2f2ZyalkFsZd4qPaLMkSlh
x-amz-request-id: ABV5032583242618
Date: Fri, 18 Mar 2012 17:15:38 GMT
Content-Type: application/xml
Transfer-Encoding: chunked
Server: AmazonS3

In this example, no Cache-Control headers are set so the default Time to Live (TTL) will be applied.

Enhanced cache control

If you need more control over how different types of assets are cached (e.g., JavaScript files, images), check out our documentation on cache freshness.

Using an Amazon S3 private bucket

To use an Amazon S3 private bucket with Fastly, you must implement version 4 of Amazon’s header-based authentication. You can do this using custom VCL. Start by obtaining the following information from AWS:

Once you have this information, you can configure your Fastly service to authenticate against your S3 bucket using header authentication by calculating the appropriate header value in VCL.

Start by creating a regular VCL snippet. Give it a meaningful name, such as AWS protected origin. When you create the snippet, select within subroutine to specify its placement and choose miss as the subroutine type. Then, populate the VCL field with the following code (be sure to change specific values as noted to ones relevant to your own AWS bucket):

1declare local var.awsAccessKey STRING;
2declare local var.awsSecretKey STRING;
3declare local var.awsS3Bucket STRING;
4declare local var.awsRegion STRING;
5declare local var.awsS3Host STRING;
6declare local var.canonicalHeaders STRING;
7declare local var.signedHeaders STRING;
8declare local var.canonicalRequest STRING;
9declare local var.canonicalQuery STRING;
10declare local var.stringToSign STRING;
11declare local var.dateStamp STRING;
12declare local var.signature STRING;
13declare local var.scope STRING;
14
15set var.awsAccessKey = "YOUR_AWS_ACCESS_KEY";   # Change this value to your own data
16set var.awsSecretKey = "YOUR_AWS_SECRET_KEY";   # Change this value to your own data
17set var.awsS3Bucket = "YOUR_AWS_BUCKET_NAME";   # Change this value to your own data
18set var.awsRegion = "YOUR_AWS_BUCKET_REGION";   # Change this value to your own data
19set var.awsS3Host = var.awsS3Bucket ".s3." var.awsRegion ".amazonaws.com";
20
21if (req.method == "GET" && !req.backend.is_shield) {
22
23  set bereq.http.x-amz-content-sha256 = digest.hash_sha256("");
24  set bereq.http.x-amz-date = strftime({"%Y%m%dT%H%M%SZ"}, now);
25  set bereq.http.host = var.awsS3Host;
26  set bereq.url = querystring.remove(bereq.url);
27  set bereq.url = regsuball(urlencode(urldecode(bereq.url.path)), {"%2F"}, "/");
28  set var.dateStamp = strftime({"%Y%m%d"}, now);
29  set var.canonicalHeaders = ""
30    "host:" bereq.http.host LF
31    "x-amz-content-sha256:" bereq.http.x-amz-content-sha256 LF
32    "x-amz-date:" bereq.http.x-amz-date LF
33  ;
34  set var.canonicalQuery = "";
35  set var.signedHeaders = "host;x-amz-content-sha256;x-amz-date";
36  set var.canonicalRequest = ""
37    "GET" LF
38    bereq.url.path LF
39    var.canonicalQuery LF
40    var.canonicalHeaders LF
41    var.signedHeaders LF
42    digest.hash_sha256("")
43  ;
44
45  set var.scope = var.dateStamp "/" var.awsRegion "/s3/aws4_request";
46
47  set var.stringToSign = ""
48    "AWS4-HMAC-SHA256" LF
49    bereq.http.x-amz-date LF
50    var.scope LF
51    regsub(digest.hash_sha256(var.canonicalRequest),"^0x", "")
52  ;
53
54  set var.signature = digest.awsv4_hmac(
55    var.awsSecretKey,
56    var.dateStamp,
57    var.awsRegion,
58    "s3",
59    var.stringToSign
60  );
61
62  set bereq.http.Authorization = "AWS4-HMAC-SHA256 "
63    "Credential=" var.awsAccessKey "/" var.scope ", "
64    "SignedHeaders=" var.signedHeaders ", "
65    "Signature=" + regsub(var.signature,"^0x", "")
66  ;
67  unset bereq.http.Accept;
68  unset bereq.http.Accept-Language;
69  unset bereq.http.User-Agent;
70  unset bereq.http.Fastly-Client-IP;
71}

You may also remove the headers that AWS adds to the response. Do this by creating another VCL snippet. Give it a meaningful name, such as Strip AWS response headers. When you create the snippet, select within subroutine to specify its placement and choose fetch as the subroutine type. Then, place the following code in the VCL field:

1unset beresp.http.x-amz-id-2;
2unset beresp.http.x-amz-request-id;
3unset beresp.http.x-amz-delete-marker;
4unset beresp.http.x-amz-version-id;

Following redirects to S3 objects and caching S3 responses

Using VCL Snippets, Fastly can follow redirects to S3 objects and cache the response.

To configure Fastly to follow redirects to S3 objects, follow the steps below:

1. Log in to the Fastly web interface.
2. From the Home page, select the appropriate service. You can use the search box to search by ID, name, or domain.
3. Click Edit configuration and then select the option to clone the active version.

4. Click VCL Snippets.

5. Click Create snippet.

   

6. In the Name field, enter an appropriate name (e.g., S3 redirect - recv).

7. From the Type (placement of the snippet) controls, select within subroutine.

8. From the Select subroutine menu, select recv (vcl_recv).

9. In the VCL field, add the following condition:

   1if (req.http.redir != "true") {
   2  set req.backend = Main_Origin;
   3} else {
   4  set req.backend = s3_backend;
   5  set req.http.host = "s3.amazonaws.com";
   6}

   IMPORTANT

   In the condition above, be sure to replace the Main_Origin and s3_backend placeholders with the actual names of your backends in the service to which you're applying these redirects. You can find the exact names by navigating to the Deliver page and clicking Show VCL, which appears just beneath your service name while viewing that service.

10. Click Create to create the snippet.

11. Click Create snippet again.

    

12. In the Name field, enter an appropriate name (e.g., S3 redirect - deliver).

13. From the Type (placement of the snippet) controls, select within subroutine.

14. From the Select subroutine menu, select deliver (vcl_deliver).

15. In the VCL field, add the following condition:

    1if (resp.status == 302 || resp.status == 301) {
    2  set req.http.redir = "true";
    3  set req.url = regsub(resp.http.Location, "http://s3.amazonaws.com/(.*)$", "/\1");
    4  set req.http.Fastly-Force-Shield = "yes";
    5  restart;
    6}

16. Click Create to create the snippet.

17. Click Activate to deploy your configuration changes.