Apache module fails to load
(The following information has been confirmed for RHEL/CentOS deployments using the default yum module installation.)
The default install location for the SigSci Apache module is
/etc/httpd/modules but some systems may have Apache loading it’s config from a non-standard directory. When this happens the
yum installer will not install
/etc/httpd/modules but instead to the following path:
If Apache fails to restart after the module installation because it cannot locate
mod_signalsciences.so change the LoadModules line in
httpd.conf to reflect the correct location on the target system.
How do I configure the agent to use a proxy for egress traffic?
The agent can be configured to use a local proxy for egress traffic to the Signal Sciences cloud infrastructure by setting the
HTTPS_PROXY environment variable. Add the following line to
IP-OR-HOST-NAME with the IP address or hostname to proxy traffic to:
Restart the agent and verify the configuration.
How can I view requests that have been blocked or allowed by rules?
Normally, requests that are immediately blocked or allowed by rules will not be visible in the console. To add visibility to immediately blocked or allowed requests, configure the rule to add a custom signal to the requests. A representative sample of requests that have been tagged with a custom signal will be listed in the Requests page of the console and can be found by searching for the custom signal.
Changing hostname for web servers
By default, the agent asks the OS for the hostname configuration. The agent can be configured to instead use a custom hostname in one of two ways:
-server-hostname="HOSTNAME" flag when starting the
sigsci-agent process via command line:
Add the following line to your agent configuration file (by default at
server-hostname = "HOSTNAME"
Agent or module is not detected
When the module and agent have been successfully installed you will be able to see them reporting within the Agents page of the console. In many cases, customers first realize there may be a problem with their configuration when they have started the agent and everything appears to be running normally but the agent or module are not listed correctly.
Agent is not detected
Although the agent appears to be running, it’s possible for the agent to not be listed in the Agents page of the console. This is typically due to either the agent being misconfigured or a connection issue between the agent and our cloud-hosted backend. Run through the following troubleshooting steps:
Check if the agent is running:
ps -aef | grep sigsci-agent
Try restarting the agent with:
sudo restart sigsci-agent
If the agent is running, ensure communication between the agent and the cloud-hosted backend isn’t blocked by your firewall. The Signal Sciences agent communicates with the following endpoints outbound via port 443/TCP:
Additional information about firewall restrictions can be found in Architecture
Review any log files for error messages:
ls -l /var/log/sigsci-agent
tail -n 20 /var/log/sigsci-agent
If the agent is not starting and nothing is written to the log files, check what messages are displayed if you run the agent manually:
stop sigsci-agent /usr/sbin/sigsci-agent
Run the debug tool and send the output, along with a detailed description of the issue and all log files, to our Support team.
Module is not detected
Alternatively, although the console may show that the agent is reporting, the module may be listed as “undetected”. There are a few possible causes to this scenario and the following steps are intended to help troubleshoot this condition:
It is necessary to send a request through the system in order for the module to report to the agent. Generating a manual 404 to the server in question by requesting a page that doesn’t exist is the easiest way to start seeing traffic validated on the console. Allow up to 30 seconds from the time of the request for the module to report and the console to display the anomaly.
Confirm the steps for module installation specific to your web server, and any optional configuration changes, have been made correctly.
Restart the web server after module installation.
If the module is still not reporting and no data is showing in the console, check for issues related to domain socket permissions. By default, the agent and module are configured to use
/var/run/sigsci.sockas the local domain socket under Linux operating systems and will require sufficient privileges to run properly:
If using Red Hat/CentOS, check for SELinux:
If SE Linux is enabled refer to the SELinux support guide.
If using Ubuntu check for AppArmor and adjust security profiles if necessary:
If the module is still not reporting, reach out to our Support team with a detailed description of the issue and the following logs:
NGINX or Apache
error.log, IIS error logs (default
If NGINX is your web server, capture the output of:
Collect the configuration files
/etc/sigsci/agent.confand if running NGINX
/etc/nginx/nginx.confor if running Apache your
httpd.confnormally located in
Agent not receiving request data when integrated with Ambassador
The Ambassador configuration may not have AuthService defined, which is required for the Signal Sciences agent to receive request data. AuthService is enabled by default; if the agent is not receiving requests, run
kubectl get authservice to check on the status of this service.
What is a “499” status code?
You may occasionally see the Signal Sciences agent return a status code of “499”. A “499” status code indicates the client closed the connection mid-request.
Why are my F5 load balancer health checks failing when going through the Signal Sciences reverse proxy?
F5 load balancer health checks use HTTP/0.9 by default. However, the SigSci reverse proxy does not support HTTP/0.9 because Go—which the Signal Sciences agent is written in—does not support it. This results in the F5 healthchecks failing with 400 “Bad Request” response codes.
To resolve this, force the F5 health checks to use HTTP/1.0 or HTTP/1.1 instead. Specify the HTTP version in the Send String, which will force the monitor to send an HTTP/1.0 or 1.1 request instead.
Below is an example of an HTTP/0.9 GET request:
HTTP/1.0, it will instead become an HTTP/1.0 GET request:
GET /index.html HTTP/1.0
For additional information about altering the F5 health check requests, see F5’s official documentation.
What flags are available for configuring the agent?
The following options were derived from running the command
sigsci-agent -help and can be used as command line flags, set in
/etc/sigsci/agent.conf or set as ENV vars.
Refer to our Configuration Options to view all flags.
Generated environment variables:
SIGSCI_RPC_ADDRESS SIGSCI_RPC_VERSION SIGSCI_ACCESSKEYID SIGSCI_SECRETACCESSKEY SIGSCI_MAX_CONNECTIONS SIGSCI_MAX_BACKLOG SIGSCI_MAX_PROCS SIGSCI_MAX_RECORDS SIGSCI_SAMPLE_PERCENT SIGSCI_UPLOAD_URL SIGSCI_UPLOAD_INTERVAL SIGSCI_UPLOAD_SEND_EMPTY SIGSCI_DOWNLOAD_URL SIGSCI_DOWNLOAD_INTERVAL SIGSCI_SERVER_HOSTNAME SIGSCI_CLIENT_IP_HEADER SIGSCI_REVERSE_PROXY SIGSCI_REVERSE_PROXY_LISTENER SIGSCI_REVERSE_PROXY_UPSTREAM SIGSCI_DEBUG_LISTENER SIGSCI_DEBUG_RPC_SERIAL SIGSCI_DEBUG_GC_PERCENT SIGSCI_DEBUG_DELAY SIGSCI_DEBUG_ALWAYS_REPLY SIGSCI_DEBUG_RPC_TEST_HARNESS SIGSCI_DEBUG_LOG_BLOCKED_REQUESTS SIGSCI_DEBUG_LOG_RULE_UPDATES SIGSCI_DEBUG_LOG_WEB_INPUTS SIGSCI_DEBUG_LOG_WEB_OUTPUTS SIGSCI_DEBUG_LOG_UPLOADS SIGSCI_DEBUG_LOG_PROXY_REQUESTS SIGSCI_DEBUG_LOG_CONNECTION_ERRORS SIGSCI_DEBUG_LOG_RPC_DATA SIGSCI_DEBUG_STANDALONE SIGSCI_DEBUG_LOG_ALL_THE_THINGS SIGSCI_DEBUG_DISABLE_PROCESSING SIGSCI_LEGAL SIGSCI_VERSION SIGSCI_SITE_KEYS