Failure modes with large files
Last updated 2020-12-18
If you haven't enabled Segmented Caching, you may encounter the following failure modes when working with large objects.
Large objects can cause inefficiencies in content distribution networks. The Segmented Caching feature can help resolve these large object inefficiencies. Fastly recommends enabling Segmented Caching on services that will be serving large objects. Without Segmented Caching enabled, object size limits for your account depend on when you become a Fastly customer:
- If you created your account on or after June 17, 2020 and haven't enabled Segmented Caching, your Fastly services have a maximum object size of 20 MB.
- If you created your account prior to June 17, 2020 and haven't enabled Segmented Caching, your Fastly services have a maximum cacheable object size of 2 GB for requests without Streaming Miss or 5 GB for requests with Streaming Miss.
If the response from the origin has a
Content-Length header field that exceeds the maximum object size, Fastly will generate a
503 Response object too large response to the client. If no
Content-Length header field is returned, Fastly will start to fetch the response body. If, while fetching the response body, we determine that the object exceeds the maximum object size, we will generate a status
503 Response object too large response to the client.
Content-Length header field is present and the Streaming Miss feature is enabled, Fastly will stream the content back to the client. However, if while streaming the response body Fastly determines that the object exceeds the maximum cacheable object size, it will terminate the client connection abruptly. The client will detect a protocol violation, because it will see its connection close without a properly terminating 0-length chunk.
If reading the response body from the origin fails or times out, the problem will be reported differently depending on whether or not you’ve enabled Streaming Miss to act when the object is fetched. Without Streaming Miss, a 503 response will be generated. With Streaming Miss, however, it is already too late to send an error response since the header will already have been sent. In this case, Fastly will abruptly terminate the client connection and the client will detect a protocol violation. If the response was chunked, the client will see its connection close without a properly terminating 0-length chunk. If Content-Length was known, the client will see the connection close before the number of bytes given.