search close

Real Remote (Client) IP Addresses

access_time Updated Sep 26, 2021

Often the server being protected is behind a load balancer or other proxy. In this case, the server will see this load balancer or proxy IP address as the remote (client) IP address. To get around this common issue, most load balancers or proxies offer the ability to record the real remote IP address in an HTTP header that will be added to the request for other devices to use. The most common HTTP headers used for this are the X-Forwarded-For and X-Real-Ip headers. By default, the agent will take the real remote address from the X-Forwarded-For HTTP header when it is present, but the agent may need to be configured to use a different header (or none at all) in your environment. This (or another) HTTP header must be added by configuring the load balancer or proxy with access to the real remote address. In most cases this has already been done as it is generally required by other services as well.

To be the most compatible out of the box, the default for the agent is to take the real remote address from the X-Forwarded-For HTTP header. Without any additional configuration, the agent will use the remote address specified by this HTTP header. While this normally gives correct results, this method may not work in some environments that use a different header or another means of obtaining the real remote address.

Setting Alternative Headers in the Console

Alternative client IP headers for the agent to source the real remote IP address from can be set directly in the console by going to Site Manage > Site Settings.

You can specify up to 10 different headers. Headers will be used in order from top to bottom, meaning if the first header is not present in the request, the agent will proceed to check for the second header, and so on, until one of the listed headers is found. Headers are not case sensitive. If none of the defined headers exist, or the value is not an IP address, then the agent will use the socket address.

Note: Alternative client IP headers set in the console take priority and will override any alternative client IP headers set directly in the agent.

Setting Alternative Headers Directly in the Agent

Alternative HTTP Header

If your environment uses a different HTTP header to pass the real remote address, such as using X-Real-Ip, you will need to configure the agent by adding a line to the /etc/sigsci/agent.conf file specifying the correct header name to use.

client-ip-header = "X-Real-Ip"

As this is such a common issue, most web servers offer an alternative module for interpreting the real remote address. If one of these is used, however, the remote address will be correctly passed to the agent and you will want to disable the agent from interpreting the default X-Forwarded-For header. To do this, you will need to configure the agent by adding a line to the /etc/sigsci/agent.conf to specify that no header should be used. If this is not done, then the agent may misinterpret the remote address.

client-ip-header = " "

If the agent configuration is updated, the agent will then need to be restarted.

X-Forwarded-For Header Configuration

When a request is received, the agent will read the left-most IP address from the X-Forwarded-For (XFF) header.

For example, if a received request contains:

X_FORWARDED_FOR="127.0.0.1, 203.0.113.63"

The agent will report:

127.0.0.1

To ensure that the true IP address is being identified in the above case, the agent can be configured to read XFF IP addresses from right to left instead. Use the local-networks directive by adding the following line to your agent configuration file (/etc/sigsci/agent.conf):

local-networks = "private"

By setting the local-networks directive to private, the agent will instead read the IP addresses in the XFF header from right to left and choose the first non-local IP address. In the example above, the agent would then report:

203.0.113.63

Additional information about agent configuration options can be found here.

Alternatives with Various Web Servers

There are a number of alternative modules for interpreting the real remote address. If one of these is used, be sure to disable the agent from interpreting the headers as outlined above.

Nginx - http_realip_module

The http_realip_module that is included with nginx will allow you to extract the real IP from an HTTP header and use it internally. This performs some configurable validation and is far less prone to spoofing. In addition, the module seamlessly replaces the remote address so that nginx will just do the right thing.

To use the http_realip_module in nginx, you will need that module built into the binary. For Signal Sciences supplied binaries, this is already included (as is most vendor supplied nginx binaries). However, if you are building nginx from source, then you will need to configure nginx to enable this module.

See the documentation on this module for more details: http://nginx.org/en/docs/http/ngx_http_realip_module.html

The recommended configuration for this module is to set the set_real_ip_from directive to all trusted (internal) addresses or networks and enable recursion via the real_ip_recursive directive. For example, if your load balancer IP is 192.0.2.54 and is adding the X-Forwarded-For header, then you might use the following configuration in nginx in either the http or server blocks:

set_real_ip_from  192.0.2.54;
real_ip_header    X-Forwarded-For;
real_ip_recursive on;

Apache Web Server 2.4+ - mod_remoteip

The mod_remoteip module that is included with Apache Web Server 2.4+ will allow you to extract the real IP from an HTTP header and use it internally. This performs some configurable validation and is far less prone to spoofing. In addition, the module seamlessly replaces the remote address so that the web server will just do the right thing.

To use the mod_remoteip, you will need to load the module and configure it.

See the documentation on this module for more details: https://httpd.apache.org/docs/2.4/mod/mod_remoteip.html

The recommended configuration for this module is to set the set_real_ip_from directive to all trusted (internal) addresses or networks and enable recursion via the real_ip_recursive directive. For example, if your load balancer IP is 192.0.2.54 and is adding the X-Forwarded-For header, then you might use the following config:

# Load the module (see also a2enmod command)
LoadModule remoteip_module mod_remoteip.so

# Configure
RemoteIPInternalProxy 192.0.2.54
RemoteIPHeader X-Forwarded-For

Note: On Debian/Ubuntu, you will typically use the a2enmod command to enable the module vs. adding the LoadModule directive directly. For example:

sudo a2enmod remoteip

Apache Web Server 2.2 or less - various solutions

The Apache Web Server prior to 2.4 does not supply a module to interpret an HTTP header to get the real remote address. However, there are a number of third party modules that can be used similar to Apache Web Server 2.4+ above.

Take a look at one of these popular third party modules:

Known Issues

Google Container Engine

If you have downgraded or not upgraded Kubernetes in Google Container Engine (GKE) to at least Kubernetes v1.1, then you may not be able to get the real client IP address. The solution is to upgrade Kubernetes. See further notes on this below.

Kubernetes Prior to v1.1

If you are using Kubernetes prior to v1.1, then currently the only non-beta load balancer option is their network load balancer. The network load balancer does not add the extra X-Forwarded-For header as the HTTP(S) load balancer. Because of this, the real remote address cannot be obtained. The HTTP(S) load balancer that does add in this support is currently in beta and should be available with Kubernetes v1.1.