Agent Configuration

For most installations, accesskeyid and secretaccesskey will be the only fields that require configuring; the default agent configuration will suffice for everything else. However, some environments will want to use additional options to better suit their environment.

The agent configuration is flexible enough to work in all environments. Most configuration options are available in three forms: config file, command line, and by setting environment variables.

IMPORTANT: Agent configuration options listed as “experimental” are not fully developed and are subject to change. Use caution when building automated processes involving these options as their functionality may change as they mature.

Configuration Options

The following are the current configuration options (as of v4.41.0 on the linux platform). You can view these options on the installed Agent version by running with the --usage command line option.

Agent Configuration Options
`accesskeyid`=string Set access key ID, required in most cases
`anonymous-ip-secret-key`=string Set anonymous IP secret key. Default is to use `secretaccesskey` when generating anonymous IP addresses
`bypass-egress-proxy-for-upstreams`[=true\|false] [EXPERIMENTAL] Exclude all upstream traffic from using the egress proxy Default: “false”
`cleaner-interval`=time-duration How often to run cleanup routine Default: “10s”
`client-ip-header`=string Specify the request header containing the client IP address Default: “X-Forwarded-For”
`config`=string Specify the configuration file Default: "/etc/sigsci/agent.conf"
`context-expiration`=time-duration How long to keep request context to match with response before cleanup Default: “10s”
`custom-request-headers`=string [EXPERIMENTAL] Add custom headers to the RPC response, which will be added to the HTTP request by the module [format is CSV if name:val pairs with $AgentResponse, $RequestID, $TagList dynamic values]
`debug-log-all-the-things`[=true\|false] [EXPERIMENTAL] Log all the things Default: “false”
`debug-log-blocked-requests`[=true\|false] [EXPERIMENTAL] Log when a request is blocked Default: “false”
`debug-log-config-updates`=integer [EXPERIMENTAL] Log when config updated or checked, 0=off, 1=updated, 2=more details Default: “0”
`debug-log-connection-errors`=integer [EXPERIMENTAL] Log when connections are dropped due an error. 0=off,1=on Default: “0”
`debug-log-engine-errors`=integer [EXPERIMENTAL] Log WAF engine errors: 0=off, 1=on, 2=verbose Default: “1”
`debug-log-proxy-requests`[=true\|false] [EXPERIMENTAL] Generates debug output of proxied requests Default: “false”
`debug-log-rpc-data`=string [EXPERIMENTAL] Log (hexdump) raw RPC data to the given file
`debug-log-uploads`=integer [EXPERIMENTAL] Log what is being sent to Signal Sciences: 0=off, 1=json, 2=json-pretty Default: “0”
`debug-log-web-inputs`=integer [EXPERIMENTAL] Log web inputs coming from the module: 0=off, 1=json, 2=json-pretty Default: “0”
`debug-log-web-outputs`=integer [EXPERIMENTAL] Log web outputs going back to the module: 0=off,1=json,2=json-pretty Default: “0”
`debug-standalone`=integer [EXPERIMENTAL] Bitfield: 0=normal, 1=no upload, 2=no download, 3=no networking, 4=use empty rules, 7=no net+empty rules Default: “0”
`download-cdn-url`=string [EXPERIMENTAL] CDN URL to check and download new configurations before checking download-url, empty string disables CDN fetch Default: “https://wafconf.signalsciences.net”
`download-config-cache`=string Filename to cache latest downloaded config (if relative, then base it on shared-cache-dir)
`download-config-version`=integer [EXPERIMENTAL] Force the downloader to download a specific config version: 0=auto versioning Default: “0”
`download-failover-url`=string [EXPERIMENTAL] URL to check and download new configurations if download-url is not available Default: “https://sigsci-agent-wafconf-us-west-2.s3.amazonaws.com”
`download-interval`=time-duration [EXPERIMENTAL] How often to check for a new configuration Default: “30s”
`download-url`=string [EXPERIMENTAL] URL to check and download new configurations Default: “https://sigsci-agent-wafconf.s3.amazonaws.com”
`envoy-expect-response-data`=integer [EXPERIMENTAL] Expect response data from envoy: 0=response data is not expected and some dependent product features will not be available, 1=agent will wait for response data via http_grpc_access_log gRPC API Default: “0”
`envoy-grpc-address`=string [EXPERIMENTAL] Envoy gRPC address to listen on (unix domain socket path or host:port)
`envoy-grpc-cert`=string [EXPERIMENTAL] Envoy gRPC optional TLS cert file (PEM format)
`envoy-grpc-key`=string [EXPERIMENTAL] Envoy gRPC optional TLS key file (PEM format)
`haproxy-spoa-address`=string [EXPERIMENTAL] Haproxy SPOA address to listen on (unix domain socket path or host:port) Default: “unix:/var/run/sigsci-ha.sock”
`--help` (commandline only option) Dump basic help text
`inspection-alt-response-codes`=csv-integer [DEPRECATED] DO NOT USE: the alternative response code concept is deprecated - all codes 300-599 are now considered blocking codes and this option will be removed
`inspection-anomaly-duration`=time-duration [EXPERIMENTAL] Envoy/revproxy global duration after which the request will be considered an anomaly and the response will be inspected even if nothing else was found in the request during inspection Default: “1s”
`inspection-anomaly-size`=integer [EXPERIMENTAL] Envoy/revproxy global response size limit which the request will be considered an anomaly and the response will be inspected even if nothing else was found in the request during inspection Default: “524288”
`inspection-debug`[=true\|false] [EXPERIMENTAL] Envoy/revproxy global enable/disable inspection debug logging Default: “false”
`inspection-max-content-length`=integer [EXPERIMENTAL] Envoy/revproxy global max request content length that is allowed to be inspected Default: “307200”
`inspection-timeout`=time-duration [EXPERIMENTAL] Envoy/revproxy global inspection timeout after which the system will fail open Default: “100ms”
`jaeger-tracing`[=true\|false] [EXPERIMENTAL] Enables jaeger tracing - configured with `JAEGER\_\` environment variables (currently for envoy only) Default: “false”*
`--legal` (commandline only option) Show legal information and exit
`local-networks`=string Set local networks for determining the real client IP (CSV of CIDR, ‘all’, ’none’, or ‘private’). These are the networks trusted to set the client IP header. Default: “all”
`log-out`=string Log output location, ‘stderr’, ‘stdout’, or file name (NOTE: on Windows, important logs will be sent to the eventlog)
`max-backlog`=integer Maximum RPC requests in queue (by default scaled with rpc-workers) Default: “0”
`max-connections`=integer Maximum in-flight RPC connections (by default scaled with rpc-workers) Default: “0”
`max-inspecting`=integer [DEPRECATED] Reverse proxy only - maximum in-flight transactions that the engine can be inspecting, 0=unlimited Default: “0”
`max-logs`=integer Maximum number of log lines held while waiting to send upstream Default: “1000”
`max-procs`=string Maximum number or percentage of CPUs (cores) to use e.g max-procs=4 or max-procs=“100%”. See https://docs.fastly.com/signalsciences/how-it-works/performance-reliability/#how-much-cpu-does-signal-sciences-consume for defaults.
`max-records`=integer Maximum number of records held while waiting to send (by default scaled with rpc-workers) Default: “0”
`reverse-proxy`[=true\|false] [DEPRECATED] Enable the reverse proxy, which requires setting a listener and upstream Default: “false”
`reverse-proxy-accesslog`=string [DEPRECATED] Reverse proxy access log filename
`reverse-proxy-conn-idle-max`=integer [DEPRECATED] Reverse proxy max idle connections Default: “100”
`reverse-proxy-conn-idle-timeout`=time-duration [DEPRECATED] Reverse proxy idle connection timeout Default: “1m30s”
`reverse-proxy-conn-keepalive`=time-duration [DEPRECATED] Reverse proxy connection TCP keepalive interval Default: “30s”
`reverse-proxy-conn-timeout`=time-duration [DEPRECATED] Reverse proxy connection (TCP handshake) timeout Default: “30s”
`reverse-proxy-expect-continue-timeout`=time-duration [DEPRECATED] Reverse proxy timeout waiting for continue after expect Default: “1s”
`reverse-proxy-idle-timeout`=time-duration [DEPRECATED] Reverse proxy idle timeout Default: “0s”
`reverse-proxy-listener`=string [DEPRECATED] Reverse proxy listener address:port
`reverse-proxy-pass-host-header`[=true\|false] [DEPRECATED] Pass the client supplied host header through to the upstream (including the upstream TLS handshake for use with SNI and certificate validation) Default: “true”
`reverse-proxy-read-timeout`=time-duration [DEPRECATED] Reverse proxy read timeout Default: “0s”
`reverse-proxy-shutdown-timeout`=time-duration [DEPRECATED] Reverse proxy shutdown timeout for transactions to complete Default: “30s”
`reverse-proxy-tls`[=true\|false] [DEPRECATED] Enable the TLS reverse proxy, which requires setting a listener and upstream Default: “false”
`reverse-proxy-tls-cert`=string [DEPRECATED] Reverse proxy TLS certificate file (PEM format)
`reverse-proxy-tls-cipher-suites`=csv-string [DEPRECATED] Reverse proxy TLS listener cipher suites [use --show-tls-cipher-suites for a list] Default: “TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA”
`reverse-proxy-tls-handshake-timeout`=time-duration [DEPRECATED] Reverse proxy TLS handshake timeout Default: “10s”
`reverse-proxy-tls-insecure-skip-verify`[=true\|false] [DEPRECATED] Insecurely skip reverse proxy upstream TLS verification Default: “false”
`reverse-proxy-tls-key`=string [DEPRECATED] Reverse proxy TLS private key file (PEM format)
`reverse-proxy-tls-listener`=string [DEPRECATED] Reverse proxy TLS listener address:port
`reverse-proxy-tls-min-version`=string [DEPRECATED] Reverse proxy TLS listener min version Default: “1.0”
`reverse-proxy-tls-upstream`=csv-string [DEPRECATED] Reverse proxy TLS upstream, comma separated address:port[:scheme] with default scheme=https
`reverse-proxy-trust-proxy-headers`[=true\|false] [DEPRECATED] Trust the incoming proxy (X-Forwarded-For) header values Default: “true”*
`reverse-proxy-upstream`=csv-string [DEPRECATED] Reverse proxy upstream, comma separated address:port
`reverse-proxy-write-timeout`=time-duration [DEPRECATED] Reverse proxy write timeout Default: “0s”
`revproxy-reload-on-update`[=true\|false] [EXPERIMENTAL] Reload the reverse proxy service config on agent config updates to support dynamic reconfiguration (only functions on OSes that support zero downtime restarts such as Linux >= 3.9 kernel) Default: “false”
`rpc-address`=string RPC address to listen on and serve modules from Default: “unix:/var/run/sigsci.sock”
`rpc-version`=integer [DEPRECATED] RPC protocol version Default: “0”
`rpc-workers`=integer [EXPERIMENTAL] RPC workers to use. If unset, then the `max-procs` value will be used Default: “0”
`sample-percent`=integer Sample input, 100=process everything, 0=ignore everything Default: “100”
`secretaccesskey`=string Set secretaccesskey, required along with accesskeyid in most cases
`server-flavor`=string [EXPERIMENTAL] Server-flavor, allow distinguishing this revproxy install as a buildpack or other flavor.
`server-hostname`=string Server hostname, default is to ask OS
`service-shutdown-timeout`=time-duration Timeout waiting for pending transactions to complete during service shutdown Default: “2s”
`shared-cache-dir`=string [EXPERIMENTAL] Base directory for any cache files Default: "/tmp/sigsci-agent.cache"
`--show-tls-cipher-suites` (commandline only option) Show available TLS cipher suites and exit
`statsd-address`=string Set the statsd address to send metrics to (e.g., `hostname:port` or `unix:///path/socket`)
`statsd-metrics`=csv-string [EXPERIMENTAL] Set the statsd metrics filter (glob patterns allowed - assumed prefix if no patterns used) Default: ""*
`statsd-type`=string [EXPERIMENTAL] Set the statsd server type to enable advanced features (e.g., `statsd` or `dogstatsd`) Default: “statsd”
`upload-log`=string [EXPERIMENTAL] Log filename to write agent event data
`upload-log-header-map`[=true\|false] [EXPERIMENTAL] HTTP request,response header data in map format Default: “false”
`upload-syslog`[=true\|false] [EXPERIMENTAL] Write agent event data to syslog Default: “false”
`upload-url`=string [EXPERIMENTAL] URL to upload agent data Default: “https://c.signalsciences.net/0/push”
`--usage` (commandline only option) Dump full usage text
`validate-config`[=true\|false] Validate the config entries provided to warn against potentially invalid entries Default: “true”
`--version` (commandline only option) Show version information and exit
`waf-data-log`=string [EXPERIMENTAL] Filename to log WAF inspection data (currently JSON format). Using “eventlog” on Windows will send these to the eventlog. Only requests containing a signal will be logged.
`windows-eventlog-level`=integer [EXPERIMENTAL] Set the windows eventlog level (use names that will be converted to integers: `debug`, `info`, `warning`, `error`, or `none`). Default: “3”
Block Based Options The following block based options are only available as such in a configuration file. In the configuration file, they must be after all other regular options in the file. As an alternative to a configuration file these can be configured from a command-line option or environment variable in the following format: --option=‘name1:{opt=val,…};name2:{opt=val,…}’ OR SIGSCI_OPTION=‘name1:{opt=val,…};name2:{opt=val,…}’
`[revproxy-listener.NAME]` Define named reverse proxy listener(s) with options (block or revproxy-listener=“name1:{opt=val,…};name2:{opt=val,…};…”) revproxy-listener options: `access-log`=string Access log filename `close-conn-on-request-smuggling`[=true\|false] [DEPRECATED] ‘Connection: close’ header will be added to requests that appear to be HTTP Request Smuggling attacks Default: “false” `conn-idle-max`=integer Max idle connections in the upstream connection pool (0 will disable connection pooling) Default: “100” `conn-idle-timeout`=time-duration Idle connection timeout for the upstream connection pool Default: “1m30s” `conn-keepalive`=time-duration Connection keepalive interval for upstream connections Default: “30s” `conn-max-per-host`=integer Maximum total number of upstream connections in any state per host (0 is unlimited). Connections over the limit will block until more are available Default: “0” `conn-timeout`=time-duration Connection timeout for upstream connections Default: “30s” `enabled`[=true\|false] Enable/disable the reverse proxy listener Default: “true” `expect-continue-timeout`=time-duration Timeout waiting for ‘continue’ after ’expect’ for upstream traffic Default: “1s” `expose-raw-headers`[=true\|false] [DEPRECATED] This experimental option replaces ‘close-conn-on-request-smuggling’ functionality. The option will need to be enabled per each reverse proxy listener. Default: “true” `http2`[=true\|false] Enable HTTP/2 support for the listener Default: “true” `http2-upstreams`[=true\|false] Prefer HTTP/2 for the upstreams Default: “true” `idle-timeout`=time-duration Network idle timeout for the listener Default: “0s” `inspect-websocket`[=true\|false] Enable/disable websocket inspection Default: “false” `inspection-alt-response-codes`=csv-integer [DEPRECATED] DO NOT USE: the alternative response code concept is deprecated - all codes 300-599 are now considered blocking codes and this option will be removed `inspection-anomaly-duration`=time-duration Duration after which the request will be considered an anomaly and the response will be inspected even if nothing else was found in the request during inspection Default: “1s” `inspection-anomaly-size`=integer Response size limit which the request will be considered an anomaly and the response will be inspected even if nothing else was found in the request during inspection Default: “524288” `inspection-debug`[=true\|false] Enable/disable inspection debug logging Default: “false” `inspection-max-content-length`=integer Max request content length that is allowed to be inspected Default: “307200” `inspection-timeout`=time-duration Inspection timeout after which the system will fail open Default: “100ms” `listener`=string Listener URL [scheme://address:port] `log-all-errors`[=true\|false] Log all errors, not just common Default: “false” `minimal-header-rewriting`[=true\|false] Minimal header rewriting. If enabled, then only hop-by-hop headers will be removed as required by RFC-2616 sec 13.5.1. No proxy headers will be added/modified, though they will be passed through if trust-proxy-headers is set Default: “false” `pass-host-header`[=true\|false] Pass the client supplied host header through to the upstream (including the upstream TLS handshake for use with SNI and certificate validation) Default: “true” `read-timeout`=time-duration Network read timeout for the listener Default: “0s” `remove-hop-header`[=true\|false] Unused hop headers will be removed from forwarded requests Default: “true” `request-timeout`=time-duration Overall request timeout (will enable buffering, which may cause issues with streaming services) Default: “0s” `response-flush-interval`=time-duration Interval to flush any buffered/streaming response data (0 disables forced flushes; -1 forces flushes after every write; interval values force flushes on a fixed time interval) Default: “0s” `response-header-timeout`=time-duration Response header timeout waiting for upstream responses Default: “0s” `shutdown-timeout`=time-duration Timeout waiting for pending transactions to complete during server shutdown Default: “30s” `tls-ca-roots`=string TLS trusted certificate authority certificates file (PEM format) `tls-cert`=string TLS certificate file (PEM format) `tls-cipher-suites`=csv-string TLS listener cipher suites [use --show-tls-cipher-suites for a list] Default: “TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA” `tls-handshake-timeout`=time-duration TLS handshake timeout for upstream connections Default: “10s” `tls-insecure-skip-verify`[=true\|false] Insecurely skip upstream TLS verification (for self signed certs, etc.) Default: “false” `tls-key`=string TLS private key file (PEM format) `tls-key-passphrase`=string TLS private key passphrase in the format `type:data`, where type is one of: `pass` or `file` (EX: `pass:mypassword` or `file:/etc/secrets/tls-key-passphrase`) `tls-min-version`=string TLS listener min version Default: “1.0” `tls-verify-servername`=string Force the servername used in upstream TLS verification; consider using pass-host-header first, but this may be required if neither the hostname used by the downstream client nor the hostname/ip used in the upstream URL is listed in the upstream TLS certificate `trust-proxy-headers`[=true\|false] Trust the incoming proxy (X-Forwarded-For) header values. If not trusted, then incoming proxy headers are removed before any additions are made Default: “true”* `upstreams`=csv-string Upstream, comma separated upstream URLs [scheme://address:port] `write-timeout`=time-duration Network write timeout for the listener Default: “0s”

System Environment Options

These system level environment variable based options will also affect processing.

Environment Variables
`HTTP_PROXY` or `http_proxy`=url [DEPRECATED] Proxy outbound HTTP requests through the proxy at the defined URL
`HTTPS_PROXY` or `https_proxy`=url Proxy outbound HTTPS requests through the proxy at the defined URL (takes precedence over HTTP_PROXY for HTTPS requests)
`NO_PROXY` or `no_proxy`=csv-url Comma separated list of URLs NOT to proxy or ‘*’ for all URLs

The options are generally available in three forms, overridden in the following order:

In the configuration file (default: /etc/sigsci/agent.conf)
On the command line, prefixed with a double dash (--) (e.g., --help)
As an environment variable, all capitalized, prefixed with SIGSCI_ and dashes changed to underscores (_) (e.g., the max-procs option would become the SIGSCI_MAX_PROCS environment variable)

There are a few exceptions:

Informational options such as --help, --legal, and --version only make sense as command line options and are noted above.
As of agent v4.22.0, the HTTP_PROXY environment variable is deprecated and will no longer be honored for https connections, HTTPS_PROXY must be used.
The agent will honor the system HTTPS_PROXY environment variable allowing configuration of an egress HTTPS proxy URL for those sites where outbound access must be through a proxy (e.g., HTTPS_PROXY=http://10.0.0.1:8080).

Configuring a HTTPS Proxy for the Agent

If the system the agent is running on does not have direct internet access, it may need to be configured to access the internet via a HTTPS proxy. To do this, one or more of the HTTPS_PROXY, or NO_PROXY system environment variables will need to be configured. While on some systems this may be set system wide, it may be desireable to use the proxy for only the Signal Sciences agent.

Linux Package Based Systems (deb, rpm, etc)

On Linux and similar systems, the sigsci-agent service (systemd, upstart, init.d, etc.) will source in the /etc/default/sigsci-agent file containing var=value pairs. To set the proxy for the agent, add the environment variable configurations into this file, one per line.

For example, to use the HTTPS proxy at 10.0.0.1 on port 8080 add the following to /etc/default/sigsci-agent:

HTTPS_PROXY=http://10.0.0.1:8080

The sigsci-agent service will then need to be restarted.

Windows Based Systems

On Windows based system where the agent is run as a service, the environment variables can be set system wide, however this may require a system reboot for the services to see the change.

If the change only needs to be set for the Signal Sciences agent, then set the following registry entry to update the environment settings for only the sigsci-agent service:

Add a Multi-String Value (REG_MULTI_SZ) registry entry if it does not already exist:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\sigsci-agent\Environment

Edit the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\sigsci-agent\Environment value adding an environment variable and value in var=value form, one per line Example:

HTTPS_PROXY=http://10.0.0.1:8080

The sigsci-agent service will then need to be restarted

This can be done manually using the regedit.exe or similar utility, or via the commandline with something like the following, replacing the URLs with the correct proxy URLs:

reg add HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\sigsci-agent /v Environment /t REG_MULTI_SZ /d "HTTPS_PROXY=http://10.0.0.1:8080"

If more than one variable needs to be set, then separate each var=value with a NULL (\0) character in the above command, such as "HTTPS_PROXY=http://10.0.0.1:8080\0NO_PROXY=http://localhost"

Reverse Proxy Configuration

The agent may be configured to run as a reverse proxy. To learn more, see the reverse proxy configuration documentation.

Note: Updating the sigsci-agent will remove all environment settings from the registry, therefore if one wishes to preserve any settings, consider downloading the sigsci-agent_latest.zip from Windows Agent Installation and replacing the executables.

Next Steps

Explore our module options and install the Signal Sciences module.