Client IP addresses

Often the server being protected is behind a load balancer or other proxy. In this case, the server will see this load balancer or proxy IP address as the remote (client) IP address. To get around this common issue, most load balancers or proxies offer the ability to record the real remote IP address in an HTTP header that will be added to the request for other devices to use. The most common HTTP headers used for this are the X-Forwarded-For and X-Real-Ip headers. By default, the agent will take the real remote address from the X-Forwarded-For HTTP header when it is present, but the agent may need to be configured to use a different header (or none at all) in your environment. This (or another) HTTP header must be added by configuring the load balancer or proxy with access to the real remote address. In most cases this has already been done as it is generally required by other services as well.

To be the most compatible out of the box, the default for the agent is to take the real remote address from the X-Forwarded-For HTTP header. Without any additional configuration, the agent will use the remote address specified by this HTTP header. While this normally gives correct results, this method may not work in some environments that use a different header or another means of obtaining the real remote address.

Setting alternative headers in the control panel

You can set alternative client IP headers for the agent to source the real remote IP address directly from the control panel:

1. Log in to the Next-Gen WAF control panel.
2. From the Sites menu, select a site if you have more than one site.
3. From the Manage menu, select Site Settings.
4. Click Agent Configurations.
5. Under Client IP Headers, click Add header.
6. In the Header text box that appears, enter the header name. Headers are not case sensitive.
7. (Optional) Click Add header again and enter another header name in the new Header text box.
8. Click Update.

You can specify up to 10 different headers. Headers will be used in order from top to bottom, meaning if the first header is not present in the request, the agent will proceed to check for the second header, and so on, until one of the listed headers is found. If none of the defined headers exist, or the value is not an IP address, then the agent will use the socket address.

Removing alternate headers in the control panel

1. Log in to the Next-Gen WAF control panel.
2. From the Sites menu, select a site if you have more than one site.
3. From the Manage menu, select Site Settings.
4. Click Agent Configurations.
5. Under Client IP Headers, click Delete header to the right of the header you want to delete.

Setting alternative headers directly in the agent

You can set alternative headers directly in the agent following the details below.

Alternative HTTP header

If your environment uses a different HTTP header to pass the real remote address, you will need to configure the agent to use that header. You can set an alternative header using the client-ip-header agent configuration option. For example, you can specify the agent use the X-Real-IP header by adding the following line to the /etc/sigsci/agent.conf file:

client-ip-header = "X-Real-Ip"

As this is such a common issue, most web servers offer an alternative module for interpreting the real remote address. If one of these is used, the remote address will be correctly passed to the agent and you will want to disable the agent from interpreting the default X-Forwarded-For header. If this is not done, then the agent may misinterpret the remote address. To do this, you will need to set the client-ip-header option to an empty value:

client-ip-header = " "

If the agent configuration is updated, the agent will then need to be restarted.

X-Forwarded-For header configuration

When a request is received, the agent will read the left-most IP address from the X-Forwarded-For (XFF) header.

For example, if a received request contains:

X_FORWARDED_FOR="127.0.0.1, 203.0.113.63"

The agent will report:

127.0.0.1

To ensure that the true IP address is being identified in the above case, the agent can be configured to read XFF IP addresses from right to left instead. You can set the agent to read XFF IP addresses from right to left by setting the local-networks agent configuration option to private. Add the following line to your agent configuration file (by default at /etc/sigsci/agent.conf):

local-networks = "private"

By setting the local-networks option to private, the agent will instead read the IP addresses in the XFF header from right to left and choose the first non-local IP address. In the example above, the agent would then report:

203.0.113.63

Alternatives with various web servers

There are a number of alternative modules for interpreting the real remote address. If one of these is used, be sure to disable the agent from interpreting the headers as outlined above.

NGINX - http_realip_module

The http_realip_module that is included with NGINX will allow you to extract the real IP from an HTTP header and use it internally. This performs some configurable validation and is far less prone to spoofing. In addition, the module seamlessly replaces the remote address so that NGINX will just do the right thing.

To use the http_realip_module in NGINX, you will need that module built into the binary. For binaries supplied by Fastly, the module is already included (as is true for most vendor supplied NGINX binaries). However, if you are building NGINX from source, then you will need to configure NGINX to enable this module.

The NGINX documentation on this module provides more details.

The recommended configuration for this module is to set the set_real_ip_from directive to all trusted (internal) addresses or networks and enable recursion via the real_ip_recursive directive. For example, if your load balancer IP is 192.0.2.54 and is adding the X-Forwarded-For header, then you might use the following configuration in NGINX in either the http or server blocks:

1
2
3

set_real_ip_from  192.0.2.54;
real_ip_header    X-Forwarded-For;
real_ip_recursive on;

NGINX http_realip_module - Proxy Protocol

If your NGINX deployment is configured behind a load balancer or similar that communicates to NGINX over the proxy protocol, then the real_ip_header needs to be sourced from the proxy_protocol parameter. This can be configured in either the http or server blocks.

set_real_ip_from  192.0.2.54;
real_ip_header    proxy_protocol;

For more configuration guidance around this type of deployment, check out the NGINX documentation.

Apache Web Server 2.4+ - mod_remoteip

The mod_remoteip module that is included with Apache Web Server 2.4+ will allow you to extract the real IP from an HTTP header and use it internally. This performs some configurable validation and is far less prone to spoofing. In addition, the module seamlessly replaces the remote address so that the web server will just do the right thing.

To use the mod_remoteip, you will need to load the module and configure it.

The Apache documentation on this module provides more details.

The recommended configuration for this module is to set the RemoteIPHeader directive with the appropriate header (e.g., X-Forwarded-For) containing the real client IP address. This can be used in conjunction with RemoteIPInternalProxy which adds one or more addresses (or address blocks) to trust for presenting a valid RemoteIPHeader value (e.g., a load balancer or reverse proxy).

For example, if your load balancer IP address is 192.0.2.54 and is adding the X-Forwarded-For header, then you might use the following in Apache HTTP Server configuration (typically apache2.conf or httpd.conf):

1
2
3
4
5
6

# Load the module (see also a2enmod command)
LoadModule remoteip_module mod_remoteip.so

# Configure
RemoteIPInternalProxy 192.0.2.54
RemoteIPHeader X-Forwarded-For

On Debian/Ubuntu, you will typically use the a2enmod command to dynamically enable a module instead of adding the LoadModule directive directly to your configuration. For example:

$ sudo a2enmod remoteip

Apache Web Server 2.2 or less - various solutions

The Apache Web Server prior to 2.4 does not supply a module to interpret an HTTP header to get the real remote address. However, there are a number of third party modules that can be used similar to Apache Web Server 2.4+ above.

Take a look at one of these popular third party modules:

• mod_realip2
• mod_rpaf

Known issues

When managing real client IP addresses, keep the following in mind.

Google Container Engine

If you have downgraded or not upgraded Kubernetes in Google Container Engine (GKE) to at least Kubernetes v1.1, then you may not be able to get the real client IP address. The solution is to upgrade Kubernetes.

Kubernetes prior to v1.1

If you are using Kubernetes prior to v1.1, then currently the only non-beta load balancer option is their network load balancer. The network load balancer does not add the extra X-Forwarded-For header as the HTTPS load balancer. Because of this, the real remote address cannot be obtained. The HTTPS load balancer that does add in this support is currently in beta and should be available with Kubernetes v1.1.

• Google Container Network Load Balancer: https://cloud.google.com/container-engine/docs/load-balancer
• Google Container HTTP Load Balancer (beta): https://cloud.google.com/container-engine/docs/tutorials/http-balancer
• Kubernetes Ingress Load Balancing: https://kubernetes.io/docs/concepts/services-networking/ingress/#load-balancing