Configuring the agent

For most installations, accesskeyid and secretaccesskey will be the only fields that require configuring; the default agent configuration will suffice for everything else. However, some environments will want to use additional options to better suit their environment.

The agent configuration is flexible enough to work in all environments. Most configuration options are available in three forms: config file, command line, and by setting environment variables.

IMPORTANT

Agent configuration options listed as "experimental" are not fully developed and are subject to change. Use caution when building automated processes involving these options as their functionality may change as they mature.

Configuration Options

The following are the current configuration options (as of v4.46.0 on the linux platform). You can view these options on the installed Agent version by running with the --usage command line option.

Agent Configuration Options
accesskeyid=string 
    Set access key ID, required in most cases
anonymous-ip-secret-key=string 
    Set anonymous IP secret key. Default is to use secretaccesskey when generating anonymous IP addresses
bypass-egress-proxy-for-upstreams[=true|false] [EXPERIMENTAL]
    Exclude all upstream traffic from using the egress proxy
      Default: "false"
cleaner-interval=time-duration 
    How often to run cleanup routine
      Default: "10s"
client-ip-header=string 
    Specify the request header containing the client IP address
      Default: "X-Forwarded-For"
config=string 
    Specify the configuration file
      Default: "/etc/sigsci/agent.conf"
context-expiration=time-duration 
    How long to keep request context to match with response before cleanup
      Default: "10s"
custom-request-headers=string [EXPERIMENTAL]
    Add custom headers to the RPC response, which will be added to the HTTP request by the module [format is CSV if name:val pairs with $AgentResponse, $RequestID, $TagList dynamic values]
debug-log-all-the-things[=true|false] [EXPERIMENTAL]
    Log all the things
      Default: "false"
debug-log-blocked-requests[=true|false] [EXPERIMENTAL]
    Log when a request is blocked
      Default: "false"
debug-log-config-updates=integer [EXPERIMENTAL]
    Log when config updated or checked, 0=off, 1=updated, 2=more details
      Default: "0"
debug-log-connection-errors=integer [EXPERIMENTAL]
    Log when connections are dropped due an error. 0=off,1=on
      Default: "0"
debug-log-engine-errors=integer [EXPERIMENTAL]
    Log WAF engine errors: 0=off, 1=on, 2=verbose
      Default: "1"
debug-log-proxy-requests[=true|false] [EXPERIMENTAL]
    Generates debug output of proxied requests
      Default: "false"
debug-log-rpc-data=string [EXPERIMENTAL]
    Log (hexdump) raw RPC data to the given file
debug-log-uploads=integer [EXPERIMENTAL]
    Log what is being sent to Signal Sciences: 0=off, 1=json, 2=json-pretty
      Default: "0"
debug-log-web-inputs=integer [EXPERIMENTAL]
    Log web inputs coming from the module: 0=off, 1=json, 2=json-pretty
      Default: "0"
debug-log-web-outputs=integer [EXPERIMENTAL]
    Log web outputs going back to the module: 0=off,1=json,2=json-pretty
      Default: "0"
debug-standalone=integer [EXPERIMENTAL]
    Bitfield: 0=normal, 1=no upload, 2=no download, 3=no networking, 4=use empty rules, 7=no net+empty rules
      Default: "0"
download-cdn-url=string [EXPERIMENTAL]
    CDN URL to check and download new configurations before checking download-url, empty string disables CDN fetch
      Default: "https://wafconf.signalsciences.net"
download-config-cache=string 
    Filename to cache latest downloaded config (if relative, then base it on shared-cache-dir)
download-config-version=integer [EXPERIMENTAL]
    Force the downloader to download a specific config version: 0=auto versioning
      Default: "0"
download-failover-url=string [EXPERIMENTAL]
    URL to check and download new configurations if download-url is not available
      Default: "https://sigsci-agent-wafconf-us-west-2.s3.amazonaws.com"
download-interval=time-duration [EXPERIMENTAL]
    How often to check for a new configuration
      Default: "30s"
download-url=string [EXPERIMENTAL]
    URL to check and download new configurations
      Default: "https://sigsci-agent-wafconf.s3.amazonaws.com"
envoy-expect-response-data=integer [EXPERIMENTAL]
    Expect response data from envoy: 0=response data is not expected and some dependent product features will not be available, 1=agent will wait for response data via http_grpc_access_log gRPC API
      Default: "0"
envoy-grpc-address=string [EXPERIMENTAL]
    Envoy gRPC address to listen on (unix domain socket path or host:port)
envoy-grpc-cert=string [EXPERIMENTAL]
    Envoy gRPC optional TLS cert file (PEM format)
envoy-grpc-key=string [EXPERIMENTAL]
    Envoy gRPC optional TLS key file (PEM format)
haproxy-spoa-address=string [EXPERIMENTAL]
    Haproxy SPOA address to listen on (unix domain socket path or host:port)
      Default: "unix:/var/run/sigsci-ha.sock"
--help (commandline only option) 
    Dump basic help text
inspection-alt-response-codes=csv-integer [DEPRECATED]
    DO NOT USE: the alternative response code concept is deprecated - all codes 300-599 are now considered blocking codes and this option will be removed
inspection-anomaly-duration=time-duration [EXPERIMENTAL]
    Envoy/revproxy global duration after which the request will be considered an anomaly and the response will be inspected even if nothing else was found in the request during inspection
      Default: "1s"
inspection-anomaly-size=integer [EXPERIMENTAL]
    Envoy/revproxy global response size limit which the request will be considered an anomaly and the response will be inspected even if nothing else was found in the request during inspection
      Default: "524288"
inspection-debug[=true|false] [EXPERIMENTAL]
    Envoy/revproxy global enable/disable inspection debug logging
      Default: "false"
inspection-max-content-length=integer [EXPERIMENTAL]
    Envoy/revproxy global max request content length that is allowed to be inspected
      Default: "307200"
inspection-timeout=time-duration [EXPERIMENTAL]
    Envoy/revproxy global inspection timeout after which the system will fail open
      Default: "100ms"
jaeger-tracing[=true|false] [EXPERIMENTAL]
    Enables jaeger tracing - configured with JAEGER\_\* environment variables (currently for envoy only)
      Default: "false"
--legal (commandline only option) 
    Show legal information and exit
local-networks=string 
    Set local networks for determining the real client IP (CSV of CIDR, 'all', 'none', or 'private'). These are the networks trusted to set the client IP header.
      Default: "all"
log-out=string 
    Log output location, 'stderr', 'stdout', or file name (NOTE: on Windows, important logs will be sent to the eventlog)
max-backlog=integer 
    Maximum RPC requests in queue (by default scaled with rpc-workers)
      Default: "0"
max-connections=integer 
    Maximum in-flight RPC connections (by default scaled with rpc-workers)
      Default: "0"
max-inspecting=integer [DEPRECATED]
    Reverse proxy only - maximum in-flight transactions that the engine can be inspecting, 0=unlimited
      Default: "0"
max-logs=integer 
    Maximum number of log lines held while waiting to send upstream
      Default: "1000"
max-procs=string 
    Maximum number or percentage of CPUs (cores) to use e.g max-procs=4 or max-procs="100%".
max-records=integer 
    Maximum number of records held while waiting to send (by default scaled with rpc-workers)
      Default: "0"
reverse-proxy[=true|false] [DEPRECATED]
    Enable the reverse proxy, which requires setting a listener and upstream
      Default: "false"
reverse-proxy-accesslog=string [DEPRECATED]
    Reverse proxy access log filename
reverse-proxy-conn-idle-max=integer [DEPRECATED]
    Reverse proxy max idle connections
      Default: "100"
reverse-proxy-conn-idle-timeout=time-duration [DEPRECATED]
    Reverse proxy idle connection timeout
      Default: "1m30s"
reverse-proxy-conn-keepalive=time-duration [DEPRECATED]
    Reverse proxy connection TCP keepalive interval
      Default: "30s"
reverse-proxy-conn-timeout=time-duration [DEPRECATED]
    Reverse proxy connection (TCP handshake) timeout
      Default: "30s"
reverse-proxy-expect-continue-timeout=time-duration [DEPRECATED]
    Reverse proxy timeout waiting for continue after expect
      Default: "1s"
reverse-proxy-idle-timeout=time-duration [DEPRECATED]
    Reverse proxy idle timeout
      Default: "0s"
reverse-proxy-listener=string [DEPRECATED]
    Reverse proxy listener address:port
reverse-proxy-pass-host-header[=true|false] [DEPRECATED]
    Pass the client supplied host header through to the upstream (including the upstream TLS handshake for use with SNI and certificate validation)
      Default: "true"
reverse-proxy-read-timeout=time-duration [DEPRECATED]
    Reverse proxy read timeout
      Default: "0s"
reverse-proxy-shutdown-timeout=time-duration [DEPRECATED]
    Reverse proxy shutdown timeout for transactions to complete
      Default: "30s"
reverse-proxy-tls[=true|false] [DEPRECATED]
    Enable the TLS reverse proxy, which requires setting a listener and upstream
      Default: "false"
reverse-proxy-tls-cert=string [DEPRECATED]
    Reverse proxy TLS certificate file (PEM format)
reverse-proxy-tls-cipher-suites=csv-string [DEPRECATED]
    Reverse proxy TLS listener cipher suites [use --show-tls-cipher-suites for a list]
      Default: "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA"
reverse-proxy-tls-handshake-timeout=time-duration [DEPRECATED]
    Reverse proxy TLS handshake timeout
      Default: "10s"
reverse-proxy-tls-insecure-skip-verify[=true|false] [DEPRECATED]
    Insecurely skip reverse proxy upstream TLS verification
      Default: "false"
reverse-proxy-tls-key=string [DEPRECATED]
    Reverse proxy TLS private key file (PEM format)
reverse-proxy-tls-listener=string [DEPRECATED]
    Reverse proxy TLS listener address:port
reverse-proxy-tls-min-version=string [DEPRECATED]
    Reverse proxy TLS listener min version
      Default: "1.0"
reverse-proxy-tls-upstream=csv-string [DEPRECATED]
    Reverse proxy TLS upstream, comma separated address:port[:scheme] with default scheme=https
reverse-proxy-trust-proxy-headers[=true|false] [DEPRECATED]
    Trust the incoming proxy (X-Forwarded-For*) header values
      Default: "true"
reverse-proxy-upstream=csv-string [DEPRECATED]
    Reverse proxy upstream, comma separated address:port
reverse-proxy-write-timeout=time-duration [DEPRECATED]
    Reverse proxy write timeout
      Default: "0s"
revproxy-reload-on-update[=true|false] [EXPERIMENTAL]
    Reload the reverse proxy service config on agent config updates to support dynamic reconfiguration (only functions on OSes that support zero downtime restarts such as Linux >= 3.9 kernel)
      Default: "false"
rpc-address=string 
    RPC address to listen on and serve modules from
      Default: "unix:/var/run/sigsci.sock"
rpc-version=integer [DEPRECATED]
    RPC protocol version
      Default: "0"
rpc-workers=integer [EXPERIMENTAL]
    RPC workers to use. If unset, then the max-procs value will be used
      Default: "0"
sample-percent=integer 
    Sample input, 100=process everything, 0=ignore everything
      Default: "100"
secretaccesskey=string 
    Set secretaccesskey, required along with accesskeyid in most cases
server-flavor=string [EXPERIMENTAL]
    Server-flavor, allow distinguishing this revproxy install as a buildpack or other flavor.
server-hostname=string 
    Server hostname, default is to ask OS
service-shutdown-timeout=time-duration 
    Timeout waiting for pending transactions to complete during service shutdown
      Default: "2s"
shared-cache-dir=string [EXPERIMENTAL]
    Base directory for any cache files
      Default: "/tmp/sigsci-agent.cache"
--show-tls-cipher-suites (commandline only option) 
    Show available TLS cipher suites and exit
statsd-address=string 
    Set the statsd address to send metrics to (e.g., hostname:port or unix:///path/socket)
statsd-metrics=csv-string [EXPERIMENTAL]
    Set the statsd metrics filter (glob patterns allowed - assumed prefix if no patterns used)
      Default: "*"
statsd-type=string [EXPERIMENTAL]
    Set the statsd server type to enable advanced features (e.g., statsd or dogstatsd)
      Default: "statsd"
upload-log=string [EXPERIMENTAL]
    Log filename to write agent event data
upload-log-header-map[=true|false] [EXPERIMENTAL]
    HTTP request,response header data in map format
      Default: "false"
upload-syslog[=true|false] [EXPERIMENTAL]
    Write agent event data to syslog
      Default: "false"
upload-url=string [EXPERIMENTAL]
    URL to upload agent data
      Default: "https://c.signalsciences.net/0/push"
--usage (commandline only option) 
    Dump full usage text
validate-config[=true|false] 
    Validate the config entries provided to warn against potentially invalid entries
      Default: "true"
--version (commandline only option) 
    Show version information and exit
waf-data-log=string [EXPERIMENTAL]
    Filename to log WAF inspection data (currently JSON format). Using "eventlog" on Windows will send these to the eventlog. Requests are logged to the provided location if they have at least one signal added by default inspectors or if they have at least one signal added by a request rule with a Request logging value of Sampled.
waf-data-log-all[=true|false] [EXPERIMENTAL]
    When logging WAF inspection data, log every request not just those with signals.
      Default: "false"
windows-eventlog-level=integer [EXPERIMENTAL]
    Set the windows eventlog level (use names that will be converted to integers: debug, info, warning, error, or none).
      Default: "3"
Block Based Options

The following block based options are only available
as such in a configuration file. In the configuration file,
they must be after all other regular options in the file.

As an alternative to a configuration file these can be
configured from a command-line option or environment variable
in the following format:

  --option='name1:{opt=val,...};name2:{opt=val,...}'
OR
  SIGSCI_OPTION='name1:{opt=val,...};name2:{opt=val,...}'
[revproxy-listener.NAME]  
    Define named reverse proxy listener(s) with options (block or revproxy-listener="name1:{opt=val,...};name2:{opt=val,...};...")

revproxy-listener options:
    access-log=string 
        Access log filename
    close-conn-on-request-smuggling[=true|false] [DEPRECATED]
        'Connection: close' header will be added to requests that appear to be HTTP Request Smuggling attacks
          Default: "false"
    conn-idle-max=integer 
        Max idle connections in the upstream connection pool (0 will disable connection pooling)
          Default: "100"
    conn-idle-timeout=time-duration 
        Idle connection timeout for the upstream connection pool
          Default: "1m30s"
    conn-keepalive=time-duration 
        Connection keepalive interval for upstream connections
          Default: "30s"
    conn-max-per-host=integer 
        Maximum total number of upstream connections in any state per host (0 is unlimited). Connections over the limit will block until more are available
          Default: "0"
    conn-timeout=time-duration 
        Connection timeout for upstream connections
          Default: "30s"
    enabled[=true|false] 
        Enable/disable the reverse proxy listener
          Default: "true"
    expect-continue-timeout=time-duration 
        Timeout waiting for 'continue' after 'expect' for upstream traffic
          Default: "1s"
    expose-raw-headers[=true|false] [DEPRECATED]
        This experimental option replaces 'close-conn-on-request-smuggling' functionality. The option will need to be enabled per each reverse proxy listener.
          Default: "true"
    extend-content-types[=true|false] 
        Enables extended content inspection while running in reverse proxy mode
          Default: "false"
    http2[=true|false] 
        Enable HTTP/2 support for the listener
          Default: "true"
    http2-upstreams[=true|false] 
        Prefer HTTP/2 for the upstreams
          Default: "true"
    idle-timeout=time-duration 
        Network idle timeout for the listener
          Default: "0s"
    inspect-websocket[=true|false] 
        Enable/disable websocket inspection
          Default: "false"
    inspection-alt-response-codes=csv-integer [DEPRECATED]
        DO NOT USE: the alternative response code concept is deprecated - all codes 300-599 are now considered blocking codes and this option will be removed
    inspection-anomaly-duration=time-duration 
        Duration after which the request will be considered an anomaly and the response will be inspected even if nothing else was found in the request during inspection
          Default: "1s"
    inspection-anomaly-size=integer 
        Response size limit which the request will be considered an anomaly and the response will be inspected even if nothing else was found in the request during inspection
          Default: "524288"
    inspection-debug[=true|false] 
        Enable/disable inspection debug logging
          Default: "false"
    inspection-max-content-length=integer 
        Max request content length that is allowed to be inspected
          Default: "307200"
    inspection-timeout=time-duration 
        Inspection timeout after which the system will fail open
          Default: "100ms"
    listener=string 
        Listener URL [scheme://address:port]
    log-all-errors[=true|false] 
        Log all errors, not just common
          Default: "false"
    minimal-header-rewriting[=true|false] 
        Minimal header rewriting. If enabled, then only hop-by-hop headers will be removed as required by RFC-2616 sec 13.5.1. No proxy headers will be added/modified, though they will be passed through if trust-proxy-headers is set
          Default: "false"
    pass-host-header[=true|false] 
        Pass the client supplied host header through to the upstream (including the upstream TLS handshake for use with SNI and certificate validation)
          Default: "true"
    read-timeout=time-duration 
        Network read timeout for the listener
          Default: "0s"
    remove-hop-header[=true|false] 
        Unused hop headers will be removed from forwarded requests
          Default: "true"
    request-timeout=time-duration 
        Overall request timeout (will enable buffering, which may cause issues with streaming services)
          Default: "0s"
    response-flush-interval=time-duration 
        Interval to flush any buffered/streaming response data (0 disables forced flushes; -1 forces flushes after every write; interval values force flushes on a fixed time interval)
          Default: "0s"
    response-header-timeout=time-duration 
        Response header timeout waiting for upstream responses
          Default: "0s"
    shutdown-timeout=time-duration 
        Timeout waiting for pending transactions to complete during server shutdown
          Default: "30s"
    tls-ca-roots=string 
        TLS trusted certificate authority certificates file (PEM format)
    tls-cert=string 
        TLS certificate file (PEM format)
    tls-cipher-suites=csv-string 
        TLS listener cipher suites [use --show-tls-cipher-suites for a list]
          Default: "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA"
    tls-handshake-timeout=time-duration 
        TLS handshake timeout for upstream connections
          Default: "10s"
    tls-insecure-skip-verify[=true|false] 
        Insecurely skip upstream TLS verification (for self signed certs, etc.)
          Default: "false"
    tls-key=string 
        TLS private key file (PEM format)
    tls-key-passphrase=string 
        TLS private key passphrase in the format type:data, where type is one of: pass or file (EX: pass:mypassword or file:/etc/secrets/tls-key-passphrase)
    tls-min-version=string 
        TLS listener min version
          Default: "1.0"
    tls-verify-servername=string 
        Force the servername used in upstream TLS verification; consider using pass-host-header first, but this may be required if neither the hostname used by the downstream client nor the hostname/ip used in the upstream URL is listed in the upstream TLS certificate
    trust-proxy-headers[=true|false] 
        Trust the incoming proxy (X-Forwarded-For*) header values. If not trusted, then incoming proxy headers are removed before any additions are made
          Default: "true"
    upstreams=csv-string 
        Upstream, comma separated upstream URLs [scheme://address:port]
    write-timeout=time-duration 
        Network write timeout for the listener
          Default: "0s"

System Environment Options

These system level environment variable based options will also affect processing.

Environment Variables
HTTP_PROXY or http_proxy=url [DEPRECATED]
    Proxy outbound HTTP requests through the proxy at the defined URL
HTTPS_PROXY or https_proxy=url
    Proxy outbound HTTPS requests through the proxy at the defined URL (takes precedence over HTTP_PROXY for HTTPS requests)
NO_PROXY or no_proxy=csv-url
    Comma separated list of URLs NOT to proxy or '*' for all URLs

The options are generally available in three forms, overridden in the following order:

  1. In the configuration file (default: /etc/sigsci/agent.conf)
  2. On the command line, prefixed with a double dash (--) (e.g., --help)
  3. As an environment variable, all capitalized, prefixed with SIGSCI_ and dashes changed to underscores (_) (e.g., the max-procs option would become the SIGSCI_MAX_PROCS environment variable)

There are a few exceptions:

  • Informational options such as --help, --legal, and --version only make sense as command line options as noted.
  • As of agent v4.22.0, the HTTP_PROXY environment variable is deprecated and will no longer be honored for https connections. HTTPS_PROXY must be used.
  • The agent will honor the system HTTPS_PROXY environment variable allowing configuration of an egress HTTPS proxy URL for those sites where outbound access must be through a proxy (e.g., HTTPS_PROXY=http://10.0.0.1:8080).

Configuring HTTPS proxy for the agent

If the system the agent is running on does not have direct internet access, it may need to be configured to access the internet via a HTTPS proxy. To do this, one or more of the HTTPS_PROXY, or NO_PROXY system environment variables will need to be configured.

While some systems may set this system wide, we recommend using the proxy for only the Signal Sciences agent.

Also, note that the package the agent relies on to determine the proxy URL from HTTPS_PROXY, NO_PROXY variables, ignores localhost or loopback addresses (e.g., 127.0.0.1, 0.0.0.0) with or without the port number. If you are relying on HTTPS_PROXY or NO_PROXY environment variables to be set with either of the aforementioned addresses, use a fully qualified domain name (FQDN) via the operating systems host file (e.g., c:\Windows\System32\Drivers\etc\hosts or /etc/hosts) and use it as the value.

Linux package-based systems

On Linux and similar systems, the sigsci-agent service (e.g., systemd, upstart, init.d) will source in the /etc/default/sigsci-agent file containing var=value pairs. To set the proxy for the agent, add the environment variable configurations into this file, one per line.

For example, to use the HTTPS proxy at 10.0.0.1 on port 8080, add the following to /etc/default/sigsci-agent:

HTTPS_PROXY=http://10.0.0.1:8080

On distributions using upstart, the export function needs to be prefixed:

$ export HTTPS_PROXY=http://10.0.0.1:8080

The sigsci-agent service will then need to be restarted.

Windows-based systems

On Windows-based system where the agent is run as a service, the environment variables can be set system wide. However, this may require a system reboot for the services to acknowledge the change.

If the change only needs to be set for the Signal Sciences agent, then set the following registry entry to update the environment settings for only the sigsci-agent service:

  1. Add a Multi-String Value (REG_MULTI_SZ) registry entry if it does not already exist:

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\sigsci-agent\Environment

  2. Edit the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\sigsci-agent\Environment value adding an environment variable and value in var=value form, one per line. For example:

    HTTPS_PROXY=http://10.0.0.1:8080

  3. Restart the sigsci-agent service. This can be done manually using regedit.exe or a similar utility or via the command line with something like the following, replacing the URLs with the correct proxy URLs:

    $ reg add HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\sigsci-agent /v Environment /t REG_MULTI_SZ /d "HTTPS_PROXY=http://10.0.0.1:8080"

    If more than one variable needs to be set, be sure to separate each var=value with a NULL (\0) character in the restart command (e.g., "HTTPS_PROXY=http://10.0.0.1:8080\0NO_PROXY=http://localhost").

Reverse proxy configuration

The agent may be configured to run as a reverse proxy. To learn more, check out the reverse proxy configuration documentation.

NOTE

Updating the sigsci-agent will remove all environment settings from the registry, therefore if one wishes to preserve any settings, consider downloading the sigsci-agent_latest.zip from Windows Agent Installation and replacing the executables.

Next steps

Explore our module options and install the Signal Sciences module.

Was this guide helpful?

Do not use this form to send sensitive information. If you need assistance, contact support. This form is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.