LOG IN SIGN UP
Documentation

Amazon S3

Amazon S3 can be used as an origin or as a private bucket.

Using Amazon S3 as an origin

To make your S3 data buckets available through Fastly follow the steps below.

  1. Log in to the Fastly application.
  2. Click the configure tab to access the control panel.

    the configure tab

  3. Click the green New Service button at the top right of the window. The New Service window appears.

    the New Service window

  4. Fill out the New Service window as follows:

    • In the Name field, type any descriptive name for your service.
    • In the Origin Server Address field, type s3.amazonaws.com. If you are using a non-standard S3 region (anything other than us-east), you must include that region in the server address field (e.g., s3.us-west-2.amazonaws.com). In the port field, you can type either 80 for HTTP or 443 for HTTPS.
    • In the Domain Name field, type the hostname you want to use as the URL (e.g., cdn.example.com).
  5. Click the Create button. A new service appears in the list of services available.

Now that the service is created, you will need to set the Default Host to <yourbucket>.s3.amazonaws.com by following the steps below:

  1. Click the configure tab to access the control panel.
  2. Select the new service you just created from the Service menu.
  3. Click the blue Configure button to the right of the service name.
  4. Click the Settings pane from the list on the left. The Settings controls appear.

    the Default settings area

  5. In the Default Host field of the Default Settings area, type the hostname of your S3 bucket. For example, <yourbucket>.s3.amazonaws.com.

    Set Default Host in Service Configuration

  6. Click Save Settings and deploy your changes. Your service should be active within few seconds.

Testing your results

By default, we create DNS mapping called yourdomain.global.prod.fastly.net. In the example above, it would be cdn.example.com.global.prod.fastly.net. Please test, and if you are satisfied with the results, create a DNS alias for the domain name you specified (e.g., CNAME cdn.example.com to global-nossl.fastly.net).

Fastly will cache any content without an explicit Cache-Control header for 1 hour. You can verify whether you are sending any cache headers using curl. For example:

$ curl -I opscode-full-stack.s3.amazonaws.com

HTTP/1.1 200 OK
x-amz-id-2: ZpzRp7IWc6MJ8NtDEFGH12QBdk2CM1+RzVOngQbhMp2f2ZyalkFsZd4qPaLMkSlh
x-amz-request-id: ABV5032583242618
Date: Fri, 18 Mar 2012 17:15:38 GMT
Content-Type: application/xml
Transfer-Encoding: chunked
Server: AmazonS3

In this example no cache control headers are set so default TTL will be applied.

Enhanced cache control

If you need more control over how different types of assets are cached (e.g., Javascript files, images) media check out our Amazon S3 configuration in our Cache Control tutorial.

Using an Amazon S3 private bucket

To use an Amazon S3 private bucket with Fastly, follow the instructions below.

Before you begin

Be sure you've already made your S3 data buckets available to Fastly by pointing to the right S3 bucket and setting your origin to port 443. This needs to be done before authenticating.

Be sure you've got the AWS access key ID, AWS secret key ID, and AWS Bucket name on hand. The Amazon S3 Authorization header takes the following form:

  Authorization: AWS `_AWSAccessKeyId_`:`_Signature_`

From your developer Amazon account you will need the following information:

  1. The AWS access key ID and AWS secret access key. The AWS secret access key is issued when you register. If you do not have or remember your AWS secret access key, create a new AWS access key ID. The AWS secret access key will be displayed before disappearing again.
  2. Your AWS Bucket name.

Setting up Fastly to use an Amazon S3 private bucket

In order to use an Amazon S3 private bucket with Fastly, create two headers, a Date header (for use with the authorization Signature) and an Authorization header.

Create a Date header

  1. Log in to the Fastly application.
  2. Click the configure tab to access the control panel.

    the configure tab

  3. Select the appropriate service from the Service menu.

  4. Click the blue Configure button to the right of the service name.

  5. Click the Content pane from the list on the left.

  6. In the Headers area, click the New button to create a new header. The New Header window appears.

    creating a date header via the new header window

  7. Fill out the New Header window as follows:

    • In the Name field, type Date.
    • From the Type/Action menus, select Request and Set.
    • In the Destination field type http.Date.
    • In the Source field type now.
    • From the Ignore if Set menu, select No.
    • In the Priority field, type 10.
  8. Click the Create button. A new Date header appears in the Headers area of the Content section. You will use this later within the Signature of the Authorization header.

Create an Authorization header

Next create the Authorization header with the specifications listed below.

  1. Click the New button again to create another new header.

    creating and authorization header via the new header window

  2. Fill out the New Header window as follows:

    • In the Name field, type S3 Authorization.
    • From the Type/Action menus, select Request and Set.
    • In the Destination field type http.Authorization.
    • From the Ignore if Set menu, select No.
    • In the Priority field, type 20.
  3. In the Source field, type the header authorization information using the following format:

    "AWS <AWS access key ID>:" digest.hmac_sha1_base64("<AWS secret key ID>", if(req.request == "HEAD", "GET", req.request) LF LF LF req.http.Date LF "/<AWS Bucket name>" req.url.path)
    

    replacing <AWS access key ID>, <AWS secret key ID>, and <AWS Bucket name> with the information you gathered before you began. For example:

    "AWS JKCAUEFV2ONFFOFMSSLA:" digest.hmac_sha1_base64("P2WPSu68Bfl89j72vT+bXYZB7SjlOwhT4whqt27", if(req.request == "HEAD", "GET", req.request) LF LF LF req.http.Date LF "/test123" req.url.path)
    
  4. Click the Create button. A new Authorization header appears in the Headers area of the Content section.

A detailed look at the Source field

So what's going on in the Source field of the Authorization header? Here's the basic format:

AWS<Access Key><Signature Function><key><message>

It tells us the following:

Element Description
AWS A constant placed before the access key. It's always AWS.
access key The access key ID from your Amazon developer's account. We used JKCAUEFV2ONFFOFMSSLA in this example.
signature function The algorithm used to validate the key and message of the signature. We used digest.hmac_sha1_base64(<key>, <message>) in this example.
key The secret key ID from your Amazon developer's account. We used P2WPSu68BfI89j72vT+bXYZB7SjIOwhT4whqt27 in this example.
message The UTF-8 encoding of the StringToSign. See the table below for a break down of each portion of the message.

The message that's part of the Source field in the Authorization header takes on this basic format:

<HTTP-verb></n><Content-MD5>/n<Content-Type></n><Date></n><CanonicalizedAmzHeader></n><CanonicalizedResource>

It tells us the following:

Element Description
HTTP-verb The REST verb. We use req.request in this example. We rewrite HEAD to GET because Varnish does this internally before sending requests to origin.
/n A newline indicator constant. It's always /n.
Content-MD5 The content-md5 header value, used as a message integrity check. It's often left blank. We use LF (line feed) in this example.
Content-Type The content-type header value, used to specify the MIME-type. It's often left blank. We use LFin this example.
Date The date and time stamp. We use req.http.Date (which we created first as a separate header in the steps above).
CanonicalizedAmzHeader The x-amz headers, which customize your S3 implementation. It's often left blank. We use LF in this example.
CanonicalizedResource Your Amazon private bucket name. We use "/test123" in this example.

Following redirects to S3 objects and caching S3 responses

With custom VCL, Fastly can follow redirects to S3 objects and cache the s3 response as well as the 301 or 302 response separately.

Once the ability to upload custom VCL has been enabled, be sure to read our "How do I mix and match Fastly VCL with custom VCL?" instructions. It's important to include the entire VCL boilerplate if you do not intend to override the Fastly default settings.

To configure Fastly to follow redirects to S3 objects, insert the following VCL snippets in your custom VCL:

Within vcl_recv

sub vcl_recv {

  if (req.http.redir != "true") {
    set req.backend = Main_Origin;
  } else {
    set req.backend = s3_backend;
    set req.http.host = "s3.amazonaws.com";
  }

#FASTLY recv

  if (req.request != "HEAD" && req.request != "GET" && req.request != "FASTLYPURGE") {
    return(pass);
  }

  return(lookup);

}

Within vcl_deliver

sub vcl_deliver {

  if (resp.status == 302 || resp.status == 301) {
    set req.http.redir = "true";
    set req.url = regsub(resp.http.Location, "http://s3.amazonaws.com/(.*)$", "/\1");
    set req.http.Fastly-Force-Shield = "yes";
    restart;
  }

#FASTLY deliver

  return(deliver);
}

Be sure to set the Main_Origin and s3_backend to the actual name of your backends in the service to which you're applying these redirects. You can find the exact names by reviewing your VCL; simply click on the VCL button at the top of the page while viewing the service.

Once you added these VCL snippets to your custom VCL, upload the VCL file and then activate the new version of your service to apply the changes.