Kubernetes Envoy

Last updated 2023-05-04

In this example, the Signal Sciences agent runs in a Docker sidecar and communicates directly with an Envoy proxy deployed on the application.

Integrating the Signal Sciences agent

The Signal Sciences Agent can be installed as a sidecar into each pod or as a service for some specialized needs.

The recommended way of installing the Signal Sciences Agent in Kubernetes is by integrating the sigsci-agent into a pod as a sidecar. This means adding the sigsci-agent as an additional container to the Kubernetes pod. As a sidecar, the agent will scale with the app/service in the pod instead of having to do this separately. However, in some situations, it may make more sense to install the sigsci-agent container as a service and scale it separately from the application.

The sigsci-agent container can be configured in various ways depending on the installation type and module being used.

You can use the preStop container hook to slow the pod's shutdown and ensure drain timeouts are met.

1preStop:
2  exec:
3    command:
4      - sleep
5      - "30"

Getting and updating the Signal Sciences agent container image

An official signalsciences/sigsci-agent container image is available from the Signal Sciences account on Docker Hub.

Alternatively, if you want to build your own image or need to customize the image, then follow the sigsci-agent build instructions.

These instructions reference the latest version of the agent with imagePullPolicy: Always, which will pull the latest agent version even if one already exist locally. This is so the documentation does not fall out of date and anyone using this will not have an agent that stays stagnant. However, this may not be what if you need to keep installations consistent or on a specific version of the agent. In these cases, you should specify an agent version. Images on Docker Hub are tagged with their versions and a list of versions is available on Docker Hub.

Whether you choose to use the latest image or a specific version, there are a few items to consider to keep the agent up-to-date.

Using the `latest` Signal Sciences container image

If you do choose to use the latest image, then you will want to consider how you will keep the agent up to date.

If you have used the imagePullPolicy: Always option, then the latest image will be pulled on each startup and your agent will continue to get updates.
Alternatively, you may instead choose to manually update the local cache by periodically forcing a pull instead of always pulling on startup:
```
$ docker pull signalsciences/sigsci-agent:latest
```
Then, use latest with imagePullPolicy: Never set in the configuration so that pulls are never done on startup (only manually as above):
```
1- name: sigsci-agent
2    image: signalsciences/sigsci-agent:latest
3    imagePullPolicy: Never
4    ...
```

Using a versioned Signal Sciences container image

To use a specific version of the agent, replace latest with the agent version (represented here by x.xx.x). You may also want to change imagePullPolicy: IfNotPresent in this case as the image should not change.

1- name: sigsci-agent
2    image: signalsciences/sigsci-agent:x.xx.x
3    imagePullPolicy: IfNotPresent
4    ...

This will pull the specified agent version and cache it locally. If you use this method, then it is recommended that you parameterize the agent image, using Helm or similar, so that it is easier to update the agent images later on.

Using a custom tag for the Signal Sciences container image

It is also possible to apply a custom tag to a local agent image. To do this, pull the agent image (by version or use latest), apply a custom tag, then use that custom tag in the configuration. You will need to specify imagePullPolicy: Never so local images are only updated manually. After doing so, you will need to periodically update the local image to keep the agent up-to-date.

For example:

$ docker pull signalsciences/sigsci-agent:latest
$ docker tag signalsciences/sigsci-agent:latest signalsciences/sigsci-agent:testing

Then use this image tag in the configuration:

1- name: sigsci-agent
2    image: signalsciences/sigsci-agent:testing
3    imagePullPolicy: Never
4...

Configuring the Signal Sciences agent container

Agent configuration is normally done via the environment. Most configuration options are available as environment variables. Environment variables names have the configuration option name all capitalized, prefixed with SIGSCI_ and any dashes (-) changed to underscores (_). For example, the max-procs option would become the SIGSCI_MAX_PROCS environment variable. For more details on what options are available, see the Agent Configuration documentation.

The sigsci-agent container has a few required options that need to be configured:

Agent credentials (Agent Access Key and Agent Secret Key).
A volume to write temporary files.

Agent credentials

The sigsci-agent credentials are configured with two environment variables. These variables must be set or the agent will not start.

SIGSCI_ACCESSKEYID: The Agent Access Key identifies which site in the Signal Sciences console that the agent is configured for.
SIGSCI_SECRETACCESSKEY: The Agent Secret Key is the shared secret key to authenticate and authorize the agent.

The credentials can be found by following these steps:

Log in to the Signal Sciences console.
From the Sites menu, select a site if you have more than one site.
Click Agents in the navigation bar. The agents page appears.
Click View agent keys. The agent keys window appears.
Copy the Agent Access Key and Agent Secret Key.

Because of the sensitive nature of these values, we recommend you use the built in secrets functionality of Kubernetes. With this configuration, the agent will pull the values from the secrets data instead of reading hardcoded values into the deployment configuration. This also makes any desired agent credential rotation easier to manage by having to change them in only one place.

Use the valueFrom option instead of the value option to use the secrets functionality. For example:

1env:
2  - name: SIGSCI_ACCESSKEYID
3    valueFrom:
4      secretKeyRef:
5        # Update my-site-name-here to the correct site name or similar identifier
6        name: sigsci.my-site-name-here
7        key: accesskeyid
8  - name: SIGSCI_SECRETACCESSKEY
9    valueFrom:
10      secretKeyRef:
11        # Update my-site-name-here to the correct site name or similar identifier
12        name: sigsci.my-site-name-here
13        key: secretaccesskey

The secrets functionality keeps secrets in various stores in Kubernetes. This guide uses the generic secret store in its examples, however any equivalent store can be used. Agent secrets can be added to the generic secret store using YAML similar to the following example:

1apiVersion: v1
2kind: Secret
3metadata:
4  name: sigsci.my-site-name-here
5stringData:
6  accesskeyid: 12345678-abcd-1234-abcd-1234567890ab
7  secretaccesskey: abcdefg_hijklmn_opqrstuvwxy_z0123456789ABCD

This can also be created from the command line with kubectl such as with the following example:

1$ kubectl create secret generic sigsci.my-site-name-here \
2  --from-literal=accesskeyid=12345678-abcd-1234-abcd-1234567890ab \
3  --from-literal=secretaccesskey=abcdefg_hijklmn_opqrstuvwxy_z0123456789ABCD

Additional information about Kubernetes secrets functionality can be found here.

Agent temporary volume

For added security, we recommended the sigsci-agent container be executed with the root filesystem mounted as read only. However, the agent still needs to write some temporary files such as the socket file for RPC communication and some periodically updated files such as GeoIP data.

To accomplish this with a read only root filesystem, there needs to be a writeable volume mounted. This writeable volume can also be shared to expose the RPC socket file to other containers in the same pod.

The recommended way of creating a writeable volume is to use the builtin emptyDir volume type. This is typically configured in the volumes section of a deployment, as shown in the following example:

1volumes:
2  - name: sigsci-tmp
3    emptyDir: {}

Containers will then mount this volume at /sigsci/tmp:

1volumeMounts:
2  - name: sigsci-tmp
3    mountPath: /sigsci/tmp

The default in the official agent container image is to have the temporary volume mounted at /sigsci/tmp. If this needs to be moved for the agent container, then the following agent configuration options should also be changed from their defaults to match the new mount location:

rpc-address defaults to /sigsci/tmp/sigsci.sock
shared-cache-dir defaults to /sigsci/tmp/cache

Integrating the Signal Sciences agent into an Envoy Proxy

You can deploy the Signal Sciences Agent for integration with the Envoy Proxy via the External Authorization (ext_authz), HTTP filter. This filter communicates with the sigsci-agent via gRPC.

Generic Envoy Proxy

Configuration for Envoy and the Signal Sciences agent are documented with the other modules in the Envoy install guide. This guide is for deploying the Signal Sciences agent as a sidecar to your existing Envoy configuration. Deploying the sigsci-agent container as a sidecar to Envoy is similar to a typical module based deployment, but configuration is slightly different.

To deploy the Signal Sciences agent as a sidecar to Envoy, you must:

Modify your existing Envoy configuration as noted in the Envoy install guide.
Add the sigsci-agent container to the pod, configured in Envoy gRPC listener mode.
Add an emptyDir{} volume as a place for the sigsci-agent to write temporary data.

Modifying the Envoy Proxy configuration

Modify your existing Envoy configuration as detailed in the Envoy install guide.

Add the Signal Sciences Agent as an Envoy gRPC Service:

1...
2      containers:
3      # Example Envoy front proxy running on port 8000
4      - name: envoy-frontproxy
5        image: signalsciences/envoy-frontproxy:latest
6        imagePullPolicy: IfNotPresent
7        args:
8        - -c
9        - /etc/envoy/envoy.yaml
10        - --service-cluster
11        - front-proxy
12        - -l
13        - info
14        ports:
15        - containerPort: 8000
16      # Example helloworld app running on port 8080 without sigsci configured (accessed via Envoy proxy)
17      - name: helloworld
18        image: signalsciences/example-helloworld:latest
19        imagePullPolicy: IfNotPresent
20        args:
21        # Address for the app to listen on
22        - localhost:8080
23        ports:
24        - containerPort: 8080
25      # Signal Sciences Agent running in Envoy gRPC mode (SIGSCI_ENVOY_GRPC_ADDRESS configured)
26      - name: sigsci-agent
27        image: signalsciences/sigsci-agent:latest
28        imagePullPolicy: IfNotPresent
29        # Configure the agent to use Envoy gRPC on port 9999
30        env:
31        - name: SIGSCI_ACCESSKEYID
32          valueFrom:
33            secretKeyRef:
34              # This secret needs added (see docs on sigsci secrets)
35              name: sigsci.my-site-name-here
36              key: accesskeyid
37        - name: SIGSCI_SECRETACCESSKEY
38          valueFrom:
39            secretKeyRef:
40              # This secret needs added (see docs on sigsci secrets)
41              name: sigsci.my-site-name-here
42              key: secretaccesskey
43        # Configure the Envoy to expect response data (if using a gRPC access log config for Envoy)
44        - name: SIGSCI_ENVOY_EXPECT_RESPONSE_DATA
45          value: "1"
46        # Configure the Envoy gRPC listener address on any unused port
47        - name: SIGSCI_ENVOY_GRPC_ADDRESS
48          value: localhost:9999
49        ports:
50        - containerPort: 9999
51        securityContext:
52          # The sigsci-agent container should run with its root filesystem read only
53          readOnlyRootFilesystem: true

Adding the Signal Sciences agent temp volume definition to the deployment

The agent temp volume must be defined for use by the other containers in the pod. This example uses the builtin emptyDir: {} volume type:

1...
2      volumes:
3      # Define a volume where sigsci-agent will write temp data and share the socket file,
4      # which is required with the root filesystem is mounted read only
5      - name: sigsci-tmp
6        emptyDir: {}

Do not use this form to send sensitive information. If you need assistance, contact support. This form is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.