Performance
Here are some answers to general questions about Signal Sciences performance.
How does your architecture ensure high performance and reliability?
One of the key reasons for the architectural split between the module and the agent is to optimize for maximum performance and reliability. If the agent ever crashes, your application does not go down because the module fails open if it doesn’t hear back from the agent within a set time limit. This claim is simple to verify in a deployment, as the module can be enabled without running the agent, and the site will continue to load as normal. From the performance side, this set time limit is also the worst case latency that Signal Sciences could introduce to a request.
Can I see the actual performance impact of Signal Sciences on my systems?
Yes. We provide graphs and data on resources used by the agent in the agent details page in the console.
How much memory does Signal Sciences consume?
Most clients see median memory usage of 1024mb (1GB) in production deployments.
How much CPU does Signal Sciences consume?
CPU varies by machine size. By default the number of available cores determines the maximum cores the agent can use.
The agent scales using the following by default (you can override these - see below):
| Available Cores | Agent Core Limit |
|---|---|
| 1 | 1 |
| 2 - 3 | 2 |
| 4 or more | 50% of available |
Agent and Module
Here are some answers to general questions about Signal Sciences agents and modules.
How much time does the agent spend processing a request?
Most clients see a median time of 0.6ms to 2.0ms in production deployments.
How often does the agent poll for new decisions?
By default every 30 seconds the agent will poll for any new decisions made by the back end, and this value is configurable via the agent command line.
What measures are in place to ensure agent updates are from an authorized source?
Agent updates, such as new decisions made by the back end, are encrypted by the back end and then decrypted by the agent using the agent keys.
What impact does the agent to backend communication have on my egress bandwidth?
Impact to egress bandwidth is minimal. Every 30 seconds, we compress any data we have collected and send it to our backend. In other words, it’s a ratio of n inbound attacks to one outbound request to our backend.
Are my production systems impacted if the Signal Sciences backend goes down?
No. All agent communication with the backend is asynchronous. Should the agent lose the ability to communicate with the Signal Sciences cloud backend the agent will continue to function with the following caveats:
- The agent will continue to perform detections of attacks, anomalies and any custom rules/signals
- The agent will continue to enforce existing blocking decisions
- The agent will not queue request logs and there will be an outage of data shown in the console, ability to look at individual requests or aggregate data will be lost until the connection is reestablished.
- The agent will not receive updates for new detections or enforcement decisions
How can I disable the agent?
All agents on a site can be disabled through the console:
- In the top navigation of the console, click on the agent mode.
- Click the Manage link. The agent configurations menu page appears.
- Select Off.
- Click Update.
Individual agent processes can also be disabled manually on your site/application host.
How do I increase the number of CPUs available to the agent?
By default the agent is configured to scale proportionally based on the number of available CPUs on a system. Refer to the section on how much CPU Signal Sciences consumes for details). This is typically a reasonable number, but cases of extremely high throughput can lead to resource contention, which manifests as higher latency and increased memory utilization with a slightly elevated decision time.
You can configure the number of cores available to the agent by setting the max-procs agent configuration option. Add the following line to your agent configuration file (by default at /etc/sigsci/agent.conf) where n is the number of CPU cores to use:
max-procs = n
You must then restart the agent for this change to take effect.
What’s the difference between the “Host CPU” and “Agent CPU” metrics?
The “Host CPU” metric indicates the CPU percentage for the full host wherein 100% is all cores.
The “Agent CPU” metric on the other hand doesn’t use a scale of 100%. The Agent CPU metric is the CPU by core.
For example, take a machine with 8 cores: the maximum Agent CPU percentage would be 800%. However, if the agent has been configured to be limited to only 4 cores, the maximum Agent CPU percentage would instead be 400%. In this example, if the agent is shown to be taking about 50% CPU, it’s actually only using 6% CPU (50% Agent CPU / 800% total CPU).
Updates
Here are some answers to general questions about Signal Sciences updates.
How frequently do you release updates to the agent and module?
See:
How are updates to the agent and module tested?
Our testing process includes:
- Unit tests
- Integration tests
- Security tests
- Signal detection tests (quality test)
- Module correctness tests
- Packaging / install tests
- Performance tests (load tests)
Most of these are completely automated and run regularly, if not constantly.