Troubleshooting module-agent deployments

Agent or module is not detected

When the module and agent have been successfully installed you will be able to see them reporting within the Agents page of the control panel. In many cases, customers first realize there may be a problem with their configuration when they have started the agent and everything appears to be running normally but the agent or module are not listed correctly.

Agent is not detected

Although the agent appears to be running, it's possible for the agent to not be listed in the Agents page of the control panel. This is typically due to either the agent being misconfigured or a connection issue between the agent and our cloud-hosted backend. Run through the following troubleshooting steps:

1. Check if the agent is running:

   $ ps -aef | grep sigsci-agent

2. Try restarting the agent with:

   $ sudo restart sigsci-agent

3. If the agent is running, ensure communication between the agent and the cloud-hosted backend isn't blocked by your firewall. The Next-Gen WAF agent communicates with the following endpoints outbound via port 443/TCP:

   • c.signalsciences.net
   • wafconf.signalsciences.net
   • sigsci-agent-wafconf.s3.amazonaws.com
   • sigsci-agent-wafconf-us-west-2.s3.amazonaws.com

   Additional information about firewall restrictions can be found in our network requirements guide.

4. Review any log files for error messages:

   $ ls -l /var/log/sigsci-agent

   $ tail -n 20 /var/log/sigsci-agent

5. If the agent is not starting and nothing is written to the log files, run the agent manually and check what messages are displayed:

   $ stop sigsci-agent
   $ /usr/sbin/sigsci-agent

6. Run the debug tool and send the output, along with a detailed description of the issue and all log files, to our Support team.

   $ /usr/sbin/sigsci-agent-diag

Module is not detected

Alternatively, although the control panel may show that the agent is reporting, the module may be listed as "undetected". There are a few possible causes to this scenario and the following steps are intended to help troubleshoot this condition:

1. It is necessary to send a request through the system in order for the module to report to the agent. Generating a manual 404 to the server in question by requesting a page that doesn't exist is the easiest way to start seeing traffic validated on the control panel. Allow up to 30 seconds from the time of the request for the module to report and the control panel to display the anomaly.

2. Confirm the steps for module installation specific to your web server, and any optional configuration changes, have been made correctly.

3. Restart the web server after module installation.

4. If the module is still not reporting and no data is showing in the control panel, check for issues related to domain socket permissions. By default, the agent and module are configured to use /var/run/sigsci.sock as the local domain socket under Linux operating systems and will require sufficient privileges to run properly:

   • If using Red Hat/CentOS, check for SELinux:

     $ sestatus

     If SELinux is enabled, refer to the SELinux support guide.

   • If using Ubuntu, check for AppArmor and adjust security profiles if necessary:

     $ sudo apparmor_status

5. If the module is still not reporting, reach out to our Support team with a detailed description of the issue and the following logs:

   • NGINX or Apache error.log, IIS error logs (default %SystemDrive%\inetpub\logs\LogFiles)

   • If NGINX is your web server, capture the output of:

     $ /opt/sigsci/bin/check-nginx

   • Collect the configuration files /etc/sigsci/agent.conf and if running NGINX /etc/nginx/nginx.conf or if running Apache your httpd.conf normally located in /etc/httpd/conf/httpd.conf.

Data is not showing in the control panel but the agent and module are running

If both the agent and module are reporting as active within the control panel, but no data is displayed when requests are processed, then the system time on the agent is likely out of sync. This can cause events to be reported at times significantly in the past or future. This is especially likely in a dev environment using a VM or container that gets in a paused state and is not updated via cron.

To determine whether this condition is occurring:

1. Log in to the Next-Gen WAF control panel.
2. From the Sites menu, select a site if you have more than one site.
3. Click Agents in the navigation bar.
4. Click on the name of the agent.
5. Inspect the graph for Agent clock skew (seconds). The agent clock skew should not be more than a few seconds. If this is a large value updating the system time and maintaining ntpd should rectify the issue.

Requests in the control panel aren't reporting any signals

Confirm your operating system and web server are supported

Check out our supported versions to confirm which operating system and web server versions are supported.

Confirm your agent and module are running correctly

1. Log in to the Next-Gen WAF control panel.
2. From the Sites menu, select a site if you have more than one site.
3. Click Agents in the navigation bar.
4. In the Status column, confirm the agent is listed as online.
5. In the Module column, confirm the module is listed as detected.
6. Click on the name of the agent.
7. Review the listed agent metrics to confirm the control panel is receiving telemetry from the agent. If the control panel is not receiving telemetry from the agent, some metrics will be listed as Unknown or 0 ms.
8. Confirm agent clock skew.

Check NGINX

If NGINX is your web server, you can confirm that NGINX, the agent, and the module are configured correctly by running the following:

$ /opt/sigsci/bin/check-nginx

Contact Support

If you have confirmed any issues with the previous steps, gather any necessary data and reach out to our Support team for assistance.

1. Enable verbose debug logging by adding the following line to your agent configuration file (by default at /etc/sigsci/agent.conf):

   debug-log-all-the-things = true

2. Restart the agent and collect the verbose log entries.

3. Generate an agent diagnostic package by running the following command:

   $ sigsci-agent-diag

4. Collect the agent configuration file located by default at /etc/sigsci/agent.conf.

5. Collect server configuration files:

   • NGINX: /etc/nginx/nginx.conf
   • Apache: /etc/httpd/conf/httpd.conf
   • IIS: %SystemDrive%\System32\inetsrv\config\applicationHost.config

6. Collect server error log files (if applicable):

   • NGINX: /var/log/nginx/error
   • Apache: /var/log/apache2/error.log
   • IIS: %SystemDrive%\inetpub\logs\LogFiles

7. If NGINX is your web server, collect the output of:

   $ /opt/sigsci/bin/check-nginx

8. Reach out to our Support team with a detailed description of the issue and all collected logs and configuration files.