Kubernetes startup probe

By default, the agent prioritizes quick start up and performance readiness for preliminary inspection. However, quick startup isn't always desirable if you only want the agent to inspect traffic after loading your rules and configuration data. Within Kubernetes environments, you can use startup probes to delay the agent container's startup so that the container is only marked ready after the agent has loaded rules and configuration data.

Limitations and considerations

• Startup probes are available in agent versions 4.51.0 and above.
• For detailed information about tuning startup probes, see the Kubernetes documentation for configuring probes.

Configuring a startup probe

You can configure a startup probe in one of two ways: using an httpGet check or using a file-based startup probe.

Configuring an HTTP startup probe

To configure the agent endpoint to respond to httpGet checks, set the SIGSCI_STARTUP_PROBE_LISTENER environment variable to the desired address. Then, on the same prompt, add a startupProbe httpGet definition configured for path /startup.

The agent endpoint will send an HTTP 503 (Service Unavailable) response until the agent has loaded rules. After loading rules, the agent endpoint will send an HTTP 200 response.

Example diff using port 2024 for the startup probe:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

containers:
  - name: sigsci-agent
    image: signalsciences/sigsci-agent:latest
    ports:
+     - name: "startup-port"
+       containerPort: 2024 
    env: 
+     - name: SIGSCI_STARTUP_PROBE_LISTENER
+       value: "0.0.0.0:2024"
+   startupProbe:
+     httpGet:
+       path: /startup
+       port: startup-port
+     failureThreshold: 90
+     periodSeconds: 2

Configuring a file-based startup probe

If you want to avoid using an HTTP-based probe, you can configure the agent to create a file within the container and use an exec check to test for the existence of the file to indicate a ready state.

To configure a file-based startup probe, set the SIGSCI_STARTUP_PROBE_FILEPATH environment variable to the file path where you want the agent to create a file once rules are loaded. Then, use an exec command to check the existence of the file.

The file will be created once the agent has completed loading it's configuration data. The startup probe may generate Kubernetes warning events with a message of Startup probe failed: cat: can't open '/sigsci/tmp/startup': No such file or directory until the agent has finished loading configuration data. This is expected behavior when using the file-based startup probe.

Example diff to use /sigsci/tmp/startup for the startup probe:

1
2
3
4
5
6
7
8
9
10
11
12
13

containers:
  - name: sigsci-agent
    image: signalsciences/sigsci-agent:latest
    env: 
+     - name: SIGSCI_STARTUP_PROBE_FILEPATH
+       value: "/sigsci/tmp/startup"
+   startupProbe:
+     exec:
+       command:
+         - cat
+         - /sigsci/tmp/startup
+     failureThreshold: 90
+     periodSeconds: 2