Start here

Getting started
Domains & Origins

Domains & Origins
Request settings
Load balancing
Custom VCL
Image optimization

Access Control Lists
Monitoring and testing
Securing communications
Security measures
Web Application Firewall

Logging endpoints
Non-Fastly services

Streaming logs
Debugging techniques
Common errors

Account info
Account management
User access and control


    Failure modes with large files

      Last updated July 15, 2019

    Varnish is excellent at caching, managing, and delivering small files but historically the handling of large files has been challenging. By default, Fastly limits cached file sizes to 2GB. Using streaming miss increases that limit for files up to 5GB. Fastly’s also allows you to enable support for files larger than the 5GB caching limit using VCL snippets. Regardless of which feature you use, there are a few failure modes you may encounter when working with large files, especially those larger than 5GB in size.

    Maximum object size limits

    If the response from the origin has a Content-Length header field that exceeds the maximum object size, Fastly will immediately generate a 503 response to the client unless you tell Fastly to specifically pass those objects directly to the user from the origin If no Content-Length header field is returned, Fastly will start to fetch the response body. If, while fetching the response body, we determine that the object exceeds maximum object size, we will generate a status 503 response to the client (again, unless you specifically tell Fastly to do otherwise as previously mentioned).

    If no Content-Length header field is present and Streaming Miss is in effect, Fastly will stream the content back to the client. However, if while streaming the response body Fastly determines that the object exceeds the maximum cacheable object size, it will terminate the client connection abruptly. The client will detect a protocol violation, because it will see its connection close without a properly terminating 0-length chunk.

    Origin read failures

    If reading the response body from the origin fails or times out, the problem will be reported differently depending on whether or not you’ve enabled Streaming Miss to act when the object is fetched. Without Streaming Miss, a 503 response will be generated. With Streaming Miss, however, it is already too late to send an error response since the header will already have been sent. In this case, Fastly will abruptly terminate the client connection and the client will detect a protocol violation. If the response was chunked, the client will see its connection close without a properly terminating 0-length chunk. If Content-Length was known, the client will see the connection close before the number of bytes given.

    Back to Top