- English
- 日本語
Storj DCS Object Storage
Last updated 2021-05-04
Storj DCS can be used as an origin for public and private Storj buckets via the Storj DCS S3 Gateway. Built on the Storj Network, Storj DCS is a decentralized object storage service that is S3 compatible and end-to-end encrypted by default.
Prerequisites
Before adding Storj DCS as an origin for Fastly services, you will need to create a Storj DCS account, project and access credentials, and a bucket that will serve as your origin.
Using Storj DCS as an origin
To use Storj DCS as an origin and make your Storj bucket available through Fastly via the Storj DCS S3 Gateway, follow the steps below.
Creating a new service
Follow the instructions for creating a new service.
- When you create the new domain and the new host:
- In the Domain Name field on the Create a domain page, enter the hostname you want to use as the URL (e.g.,
cdn.example.com
). - In the Hosts field on the Origins page, enter the IP address or hostname of your Storj DCS Gateway Endpoint using the format
<BUCKET>.gateway.<REGION>.storjshare.io
including your bucket (e.g.,origin.gateway.us1.storjshare.io
).
- In the Domain Name field on the Create a domain page, enter the hostname you want to use as the URL (e.g.,
- When you edit the host details on the Edit this host page:
- In the Name field, enter any descriptive name for your service if you haven't already done so.
- In the Address field, ensure you've entered the IP address or hostname of your Storj DCS Gateway Endpoint. You entered this information during host creation.
- When you edit the Transport Layer Security (TLS) area information for your host:
- If you've set up TLS for your Storj DCS S3 Gateway, leave the Enable TLS? default set to Yes to secure the connection between Fastly and your origin.
- Under the SNI hostname field, select the checkbox to Match the SNI hostname to the Certificate hostname. The address you entered during host creation appears.
- In the Certificate hostname field, enter the IP address or hostname of your Storj DCS S3 Gateway.
Testing your results
By default, we create a DNS mapping called yourdomain.global.prod.fastly.net
. In the example above, it would be cdn.example.com.global.prod.fastly.net
. Create a DNS alias for the domain name you specified (e.g., CNAME cdn.example.com
to global-nossl.fastly.net
).
Fastly will cache any content without an explicit Cache-Control
header for 1 hour. You can verify whether you are sending any cache headers using curl. For example:
$ curl -I https://cdn.example.com
Accept-Ranges: bytesContent-Length: 250Content-Type: application/xmlServer: MinIO/DEVELOPMENT.GOGETVary: OriginDate: Wed, 07 Oct 2020 02:31:27 GMT
In this example, no Cache-Control headers are set so the default TTL will be applied.
Enhanced cache control
If you need more control over how different types of assets are cached (e.g., JavaScript files, images), check out our documentation on cache freshness.
Using a Storj DCS bucket for origin hosting
To use a Storj DCS S3 Gateway as an origin with Fastly, you must implement version 4 of Amazon’s header-based authentication. You can do this using custom VCL. Start by obtaining the following information from AWS:
Item | Description |
---|---|
Bucket name | The name of your private bucket. When you download items from your bucket, this is the string listed in the URL path or hostname of each object. |
Access key | The access key string associated with a Storj DCS Access Grant that has at least read permissions on the bucket. |
Secret key | The secret access key paired with the access key above. |
Once you have this information, you can configure your Fastly service to authenticate against your private bucket using header authentication by calculating the appropriate header value in VCL.
IMPORTANT
Consider leaving the Override host field for the origin blank in your service settings. This setting will override the host header from the snippets shown here and may invalidate the signature that authenticates the information being sent.
Start by creating a regular VCL snippet. Give it a meaningful name, such as Storj DCS Origin
. When you create the snippet, select within subroutine to specify its placement and choose miss as the subroutine type. Then, populate the VCL field with the following code (be sure to change specific values as noted to ones relevant to your own bucket):
1declare local var.accessKey STRING;2declare local var.secretKey STRING;3declare local var.storjBucket STRING;4declare local var.storjGateway STRING;5declare local var.region STRING;6declare local var.canonicalHeaders STRING;7declare local var.signedHeaders STRING;8declare local var.canonicalRequest STRING;9declare local var.canonicalQuery STRING;10declare local var.stringToSign STRING;11declare local var.dateStamp STRING;12declare local var.signature STRING;13declare local var.scope STRING;14
15
16set var.accessKey = "YOUR_ACCESS_KEY"; # Change this value to your own data17set var.secretKey = "YOUR_SECRET_KEY"; # Change this value to your own data18set var.storjBucket = "YOUR_BUCKET_NAME"; # Change this value to your own data19set var.storjGateway = "STORJ-DCS_GATEWAY"; # Change this value to your own data20set var.region = "decentralized";21
22
23if (req.method == "GET" && !req.backend.is_shield) {24
25 set bereq.http.x-amz-content-sha256 = digest.hash_sha256("");26 set bereq.http.x-amz-date = strftime({"%Y%m%dT%H%M%SZ"}, now);27 set bereq.http.host = var.storjBucket "." var.storjGateway;28 set bereq.url = querystring.remove(bereq.url);29 set bereq.url = regsuball(urlencode(urldecode(bereq.url.path)), {"%2F"}, "/");30 set var.dateStamp = strftime({"%Y%m%d"}, now);31 set var.canonicalHeaders = ""32 "host:" bereq.http.host LF33 "x-amz-content-sha256:" bereq.http.x-amz-content-sha256 LF34 "x-amz-date:" bereq.http.x-amz-date LF35 ;36 set var.canonicalQuery = "";37 set var.signedHeaders = "host;x-amz-content-sha256;x-amz-date";38 set var.canonicalRequest = ""39 "GET" LF40 bereq.url.path LF41 var.canonicalQuery LF42 var.canonicalHeaders LF43 var.signedHeaders LF44 digest.hash_sha256("")45 ;46
47 set var.scope = var.dateStamp "/" var.region "/s3/aws4_request";48
49
50 set var.stringToSign = ""51 "AWS4-HMAC-SHA256" LF52 bereq.http.x-amz-date LF53 var.scope LF54 regsub(digest.hash_sha256(var.canonicalRequest),"^0x", "")55 ;56
57 set var.signature = digest.awsv4_hmac(58 var.secretKey,59 var.dateStamp,60 var.region,61 "s3",62 var.stringToSign63 );64
65
66 set bereq.http.Authorization = "AWS4-HMAC-SHA256 "67 "Credential=" var.accessKey "/" var.scope ", "68 "SignedHeaders=" var.signedHeaders ", "69 "Signature=" + regsub(var.signature,"^0x", "")70 ;71
72 unset bereq.http.Accept;73 unset bereq.http.Accept-Language;74 unset bereq.http.User-Agent;75 unset bereq.http.Fastly-Client-IP;76}
Do not use this form to send sensitive information. If you need assistance, contact support. This form is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.