Hi!

Please read this important information about creating issues.

If you are reporting a new issue, make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead.

If you suspect your issue is a bug, please edit your issue description to include the BUG REPORT INFORMATION shown below. If you fail to provide this information within 7 days, we cannot debug your issue and will close it. We will, however, reopen it if you later provide the information.

This is an automated, informational response.

Thank you.

For more information about reporting issues, see https://github.com/docker/docker/blob/master/CONTRIBUTING.md#reporting-other-issues

BUG REPORT INFORMATION

Use the commands below to provide key information from your environment:

docker version:
docker info:
uname -a:

Provide additional environment details (AWS, VirtualBox, physical, etc.):

List the steps to reproduce the issue:
1.
2.
3.

Describe the results you received:

Describe the results you expected:

Provide additional info you think is important:

----------END REPORT ---------

#ENEEDMOREINFO

Description of problem:

I have an hard time figuring out how to get the real remote ip of a client connecting to a (web) service inside a container which port is exposed to a port on the host.

So far the solution seems to be either one of:

• use --net=host
• disable the userland proxy
• configure nginx on the host to forward the real ip address?

Here are related issues:

#7540
nginx-proxy/nginx-proxy#130
nginx-proxy/nginx-proxy#133

None of the current solutions seems to be "right" and they all seem temporary. Can you clarify/explain how things should be, even if that means that the current docker 1.7.1 is bugged in that regard and that we have to use hack X or Y until things are fixed in 1.8.0.

docker version:

Client version: 1.7.1
Client API version: 1.19
Go version (client): go1.4.2
Git commit (client): 786b29d
OS/Arch (client): linux/amd64
Server version: 1.7.1
Server API version: 1.19
Go version (server): go1.4.2
Git commit (server): 786b29d
OS/Arch (server): linux/amd64

docker info:

Containers: 1
Images: 508
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 510
 Dirperm1 Supported: false
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.13.0-58-generic
Operating System: Ubuntu 14.04.2 LTS
CPUs: 4
Total Memory: 15.64 GiB
Name: philippe-desktop
ID: QLBB:VYAR:TKYL:NZQN:MGON:SNU4:7BLV:XFEJ:AIGR:LZY6:LCIF:GHBU

uname -a:

Linux philippe-desktop 3.13.0-58-generic #97-Ubuntu SMP Wed Jul 8 02:56:15 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

Environment details (AWS, VirtualBox, physical, etc.):

Physical

How reproducible:

Easily.

Steps to Reproduce:

1. FIXME

Actual Results:

We get docker0 bridge ip.

Expected Results:

We get the remote client ip.

Additional info:

Creating issues in this format is a PITA and goes against github flow.

disable the userland proxy

This is the correct answer, and is soon to become the default action (see #14856)

Edit: If this solution seems unsatisfactory, can you explain your thoughts on the matter?

Alright, thanks!

@phemmer

Edit: If this solution seems unsatisfactory, can you explain your thoughts on the matter?

the problem is that if you use --iptables=false you can not disable userland proxy. Please take a look here
Will appreciate any suggestions. Thanks.

Ok, what's the latest recommended solution if --userland-proxy=false is unworkable (as described in #14856)?

• use --net=host
• disable the userland proxy
• configure nginx on the host to forward the real ip address?

Also, can we re-open this issue if #14856 was reverted? Or has it been documented somewhere?

+1 for reopen, as "no userland proxy" is not viable at the moment.

+1 to reopen from me. Looking for the real IP address to set SSL via a reverse proxy

+1 this issue is really scary.

+1 we need more documentation on this subject and workable solution.

+1
receiving the wrong source IP inside the container results in problems with SPF validatio, spam scoring, access statistics etc.

+1
Need a way to handle geolocated requests.

+1 need to know which IP source connect to my app, and fail2ban if needed

Since we already have so many +1s, let's have another. And a minimal setup I used to test this & exhibit the problem: http://serverfault.com/questions/813298/

Another +1 here. Need to have true client IP address for logging and security purposes. Re-open with some documentation please :)

+1 here. Really need to get true client IP address.

+1 here.

$ docker version
Client:
 Version:      1.12.3
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   6b644ec
 Built:        Wed Oct 26 21:44:32 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.3
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   6b644ec
 Built:        Wed Oct 26 21:44:32 2016
 OS/Arch:      linux/amd64

$ uname -r
3.13.0-100-generic

Failed using the geoip module....

+1 The NAT should only be used on outbound traffic and not on inbound traffic

+1 here

+1 here. Need the real IP address for access statistics, logging and security purposes.

+1 This is a real problem. We need the real IP for our firewall

+1. Please reopen the issue.

+1 IMO this should be flagged as critical and fixed ASAP

You may read through all the 100+ comments in here to find out why this happens, but...
If in your deployment you:

• Require a network for your containers to communicate between themselves, and the docker daemon as built you a bridged network, to your instruction or not, instead of having built a network with the host driver (bridged docker internal networks have rules that allow the docker daemon to NAT requests coming into the host with the default gateway of docker's virtual interface, usually docker0 or eth0 with a default gateway ipv4 values of: 127.X.0.1 || 198.X.0.1 ||10.X.0.1
• Plan to have one or several of these containers run an HTTP server like NGINX and act as a reverse proxy for other container services inside your deployment, meaning this container is exposed to the public network and will act as a load balancer HTTP requests meant to be proxied to other containers.
• Clients will be directly accessing your host where the exposed containers running NGINX are, meaning client requests do not contain an X-Forwarded-For header present when they arrive at your reverse-proxy, this will be the case if your Client has simply oppened up his browser and made a GET to your domain, if you've written code on your static files to attach XFF headers on fetch requests this will obviously not be the case, but consider that for any initial request a regular client could type from their browser, this isn't an option.
• As reported by some of the people here, if you already use a PaaS provider with http load balancing capabilities, and this is where the client's request is coming from to your exposed containers running an HTTP server, an XFF with the clients reported $remote_addr value attached, while searching I saw people reporting such behaviour from Cloudflare and DigitalOcean specifically.

If you do manage to establish a correctly working HTTP reverse-proxy behaviour from your exposed container's HTTP server to the upstream containers running your other services, you'll notice the value of $remote_addr on every request logged on your http server logs will be the default gateway of the virtual interface (so 127.X.0.1... etc) for all incoming requests. , as opposed to the "correct" (quotations used for anyone unfamiliar with IP spoofing and proper XFF chain validations) value of the client's IP, and depending on the request, you'll see no value in the X-Forwarded-For header.

You want to keep both the bridged network container capabilities while being able to report the "real" $remote_addr of the request, the only gimmik that does this on a deployment on windows hosts is to use another non containarized reverse proxy to front for your exposed container and attach an XFF header with the reported remote_addr value of the client at the time the request was made, this is easy to do and I recommend it, you get to keep your compose / swarm deployment because even if the iptables docker daemon sets up spoof the value of the $remote_addr you'll still have this value on the XFF header chain, you'll simply be losing the value of your new reverse-proxy's $remote_addr, which you know anyway and can pass along the upstream with simple configuration. The other option involves dabbling with the host's running exposed containers iptables, I wouldn't attempt this on windows if my life depended on it, the user Struanb posted a shellscript for it and there's more comments that might help you should you want to enjoy doing that yourself.

This issue being up for 7 years and still causing problems tells me that anyone who came across this should start looking into Kubernettes as a container orcheastration tool instead.

I found the solution on the traefik page and used with jwilder/nginx-proxy

The relevant part is:

    # Listen on port 80, default for HTTP, necessary to redirect to HTTPS
      - target: 80
        published: 80
        mode: host
      # Listen on port 443, default for HTTPS
      - target: 443
        published: 443
        mode: host

Main point is you don't have to put the "whole container" to "host networking mode" just those ports. Meaning it remains part of the default docker-compose network too, so available for all the "microservice" containers.

For some reason this also doesn't work for me. Rebooting the host doesn't change that. (Running ubuntu 20.04)
I don't really understand why this (apparently huge) problem won't get fixed...

I bet this won't be fixed by the end of this decade!

It will be a decade soon :P

I'm still having this same issue...

Turns out the problem was I was using docker rootless mode. You can either use normal docker or you can use rootless and install the slirpn4netns package and change the port forwarder, following these steps (the section about changing the port forwarder): https://rootlesscontaine.rs/getting-started/docker/#changing-the-port-forwarder

Insights

The below is focused on Linux host, so may not be as helpful to Windows / macOS users (although I've read the desktop apps are meant to handle networking better these days and userland-proxy is not relevant there anymore).

Local connections:

• localhost / 127.0.0.1 would be ambiguous between host and container, the docker gateway IP helps disambiguate that?
• userland-proxy: true => May be useful if you need your own LAN IP to be visible to a container as a client.
• userland-proxy: false => Can retain container IP when connecting to another container indirectly through the host IP (I don't know why you'd want to do this locally though, not compatible across separate container networks either).

Remote connections:

• Shouldn't be an issue except IPv6 reachable host with userland-proxy: true (may need ip6tables: true to avoid remote IP as IPv4/IPv6 gateway address causing various monitoring and security issues).

Is there anything else missing? What kind of remote IP sources are problematic to everyone here?

I have documented config + test detailing a common remote client IP container scenario with Caddy (reverse-proxy / web server like nginx). If you experience this issue, perhaps you can try follow that as a reproduction?

Gateway IP via local client connection

Here is a reference for userland-proxy enabled / disabled, showing the remote client IP a container gets (as RemoteAddr) for various connections:

• Host to container via localhost / 127.0.0.1 and host IP. Both via containers published port.
• Host to container IP directly.
• 2nd container routed through host IP published port.
• 2nd container direct to container IP.

You will find the only difference for the remote IP presently (Docker v23, linux) via this setting depends on if you're accessing a container from the host indirectly, or via another container through the host IP:

• Host to Container (eg: web browser to nginx container):
  • userland-proxy: true will preserve remote IP (if you connect to the containers published port through that same IP).
  • Regardless of userland-proxy setting, host to container connection via localhost / 127.0.0.1 (or the container IP directly) will use the docker gateway IP.
• Container to Container (via host IP):
  • userland-proxy: false will properly keep the client container IP if you need to access the container indirectly (this will not work for containers on separate networks, they are isolated preventing connection).
  • Regardless of userland-proxy, container to container connection within the same network will retain the correct remote IP of the client container.

Gateway IP from an external client connection

When the host receives an actual remote client from another system via an external interface IP (that containers have published ports to be reachable). This should retain the remote client IP across that bridged connection using NAT:

• Provided iptables / ip6tables are enabled in /etc/docker/daemon.json.
• Without ip6tables: true, an IPv6 client connection results in either:
  • userland-proxy: true docker gateway IP becomes the remote IP: IPv4 (via NAT64), or if the containers network has IPv6 enabled, IPv6 (via NAT66).
  • userland-proxy: false prevents container from being reachable over IPv6 (same effect for IPv4 when iptables: false). Not the case if using separate IPv6 GUA (publicly routable) addresses to containers where NAT is not required.

Thread summary / highlights

• The author of this issue notes that the remote IP is reliable if it is not via the loopback address for localhost:

  We see that for localhost requests, the IP is the docker ip. For requests done on the remote interface, the IP is the correct one.

  So the only bug we are talking about is that 127.0.0.1 ends up as the docker IP, which does no seem outrageous to me.

• Another comment here compares a service listening on 0.0.0.0:80 on the host, preventing a container publishing to port 80 for a specific interface. They "fix" it by having iptables rule to intercept port 80 for the specific IP they wanted to forward to the container. That works because iptables rule has priority (the behaviour they're trying to workaround with -p is intentional, not a bug).

• Then there was this comment about userland-proxy: false not working for them with an IPv6 issue (which has improved since 2018), and they also cite that it's specific to localhost usage.

• This UDP comment was a less common issue to see reported, and AFAIK resolved from a PR in Jan 2023, so should be resolved with v24 of Docker Engine and any earlier releases it is back ported to.

• This comment details disabling iptables in Docker and delegating to UFW. Presumably not that relevant to most chiming in here, but would be good to know how many of the +1 comments are on a linux system with UFW enabled.

Many other comments only contribute additional information regarding --network host and it's lack of support in Windows / macOS (both which run in VMs, complicating it further).

Provide more info with your +1 / "me too" comments

Those that are affected by this. Could you share some more real-world usage issues related to it?:

• Windows / macOS affected users, presumably this is in a development / learning environment, not serving real users on external devices? (and more akin to localhost / 127.0.0.1 being involved?)
• Other users may have some usage for hobby / personal use? Serving some devices on the local network, or some external ones via a VPN or other means to be reachable via the internet?

Does it affect anyone deploying for production, or to remote systems like a small VPS?:

• The most common issue observed here is userland-proxy: true will route an IPv6 connection at the host to a container in a IPv4 only docker network, replacing the remote IP with the gateway. Avoidable via correctly configured IPv6 GUA address assignment, or with IPv6 ULA with ip6tables: true (similar to IPv4 NAT with docker, simpler and more portable).

It would great to get a better understanding of the issue, as right now it seems to be several related issues are all lumped into a single issue, piling up comments that aren't helpful (burying the more useful ones that I linked).

Separate issues could be opened to be tracked instead (especially for Windows/macOS concern, and probably localhost too).

I found the solution on the traefik page and used with jwilder/nginx-proxy

The relevant part is:

    # Listen on port 80, default for HTTP, necessary to redirect to HTTPS
      - target: 80
        published: 80
        mode: host
      # Listen on port 443, default for HTTPS
      - target: 443
        published: 443
        mode: host

Main point is you don't have to put the "whole container" to "host networking mode" just those ports. Meaning it remains part of the default docker-compose network too, so available for all the "microservice" containers.

Great. You save my life.

This thread once again just point out THE STILL EXISITING NEED for a good working and EASY to implemet solution. I guess it's hard for the NGINX devs to do it.. otherwise I won't understand, why a search shows so many threads accross several boards withing 7+ (?) years.

Personally I prefer to have ONE instance of NGINX to mange all incoming http/s directly OR to proxy them to my dockerized services. My services are spread across different docker-networks. No chance to set my dockerized NGINX to host-mode.

But without real-IP I face serveral security issues. fail2ban does not work and, and, and ......
It is annoying!

(sorry for contributing nothing else than my frustration.)

(sorry for contributing nothing else than my frustration.)

At least contribute a minimal example that can help express the issue you're facing, along with version of Docker.

From Docker v27 onwards IPv6 is no longer experimental opt-in, it should be available by default IIRC, thus the concern of inbound clients to the host via IPv6 shouldn't be an issue anymore?

What other situation are you still having trouble with?

Sharing this information when you add a comment would be helpful:

• compose.yaml config or docker run command
• How are you producing the failure? Is it all on the same system, or an actual remote client?
• Is the remote connection made over IPv4 or IPv6?
• How are you identifying the remote IP not being preserved is a fault of Docker?
  • If your proxied service is not getting the IP, you must ensure the reverse proxy itself is configured to actually proxy it over (I say this since some users don't realize this).
  • You can confirm it works without Docker? Or is that all you've tried? If you've not tried without Docker are you just assuming it's the fault of Docker? (to be clear, yes you may know it is possible, I am asking if you've actually tried as a verification step not just an assumption)

But without real-IP I face serveral security issues. fail2ban does not work and, and, and ......
It is annoying!

It works in Docker, I've explained how. I've demonstrated it plenty of times on issues. But users don't seem to want to read?

Provide the information requested above, I can probably explain why it's not working for you.

I bet this won't be fixed by the end of this decade!

It will be a decade soon :P

Happy new year 2025 everybody!

It was decade now, with the support of Claude Sonnet and GPT-4o I still unable to do this on WSL2 on Windows, does anyone have success solution on WSL2? I'm using double Caddy server to maintain IP outside docker but still failed. Does kubernetes or anything else can preserve IP on WSL2? I'm willing to learn it this new year

@hiepxanh you don't say which networking mode you are using for wsl2, and depending on that, and whether you are using a browser running in wsl2 or windows, it may or may not be possible to do this. Generally, it is better to develop software that doesn't rely on specific ip addresses these days, especially for development.

@hiepxanh you don't say which networking mode you are using for wsl2, and depending on that, and whether you are using a browser running in wsl2 or windows, it may or may not be possible to do this. Generally, it is better to develop software that doesn't rely on specific ip addresses these days, especially for development.

You are correct. double proxy server can help workaround missing source IP.

I used to using Asus router hosting a website and I need client source IP for analytic tracking and prevent spam. So it have no option to preserve source IP.
Switch to OpenWRT with external address option help me now to track client IP. On Windows 11 the network_mode: host not working, so using double proxy with X-real-clientIP on Caddy server at host machine, then forward all to current Caddy on docker machine work as expect!

Three months to debugging and find out this solution

After that setup the host caddy proxy at port 80 then proxy X-real-clientIP to your docker caddy at port 8080 for example and the inside caddy can use it

WORKING SOLUTION: using DOUBLE Caddy at host machine to preserve X-forward-for

to maintain X-Forward-For having Source IP

0) ROUTER CONFIG: openWRT proxy 80 and proxy 443 to 192.168.1.95 at port 80 with external address

1) HOST SERVER CONFIG: download Caddy then add caddy at statup program:

• open RUN window
• shell:startup
• create new shortcut
• using proxy-server.bat to run without windows: C:\Users\hiepx\small-cosmos\caddy_2.9.0_windows_amd64\caddy.exe reverse-proxy --from :80 --to 127.0.0.1:8080
• or running as service: https://caddyserver.com/docs/running#windows-service

2) CADDY DOCKER CONFIG: Caddy at docker require trusted_proxies label

services:
  caddy:
    image: lucaslorentz/caddy-docker-proxy:ci-alpine
    restart: always
    labels:
      caddy.servers.trusted_proxies: static private_ranges
    ports:
      - "8080:80"
      - "8019:2019"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /var/lib/hiepxanh/data/caddy:/data
  whoami:
    image: traefik/whoami
    restart: always
    ports:
      - "8888:80"
    labels:
      caddy_1: http://whoami.docker.localhost
      caddy_1.reverse_proxy: "{{ upstreams 80}}"

I faced the same issue while configuring IP filtering for some APIs protected by nginx. In my case everything was fine till the moment Docker Desktop started to facing an issue, reported on their website, that some users may encounter a problem with a popup saying there is a malware and etc.

I reinstalled Docker Desktop and after that $remote_addr stopped working as I expected and locally a started to receive an IP of the gateway, for a network type = "bridge".

I decided to test my application on Linux, so I deployed it on Digital Ocean, as I was doing before for testing, under Ubuntu 22.04. And for my surprise, everything worked properly as previously: $remote_addr was returning a IP of the host which was making a request to my backend.

Summary:

• $remote_addr works properly on Ubuntu 22.04
• $remote_addr stopped working for mac OS Sonoma 14.6.1 after updating Docker Desktop from version 4.32 to 4.37

@Stashevich when you run in Mac or Windows you're actually running in a VM. The best is still to use Linux with mode: host here's my most current caddy snippet

services:
  edge:
    environment:
      OTEL_SERVICE_NAME: edge
    image: mirror.gcr.io/library/caddy
    configs:
      - source: Caddyfile
        target: /etc/caddy/Caddyfile
    volumes:
      - caddy_data:/data
      - caddy_config:/config
    ports:
      - target: 80
        published: 80
        protocol: tcp
        # mode: host do not enable for port 80, because only one port can be made mode: host
      - target: 443
        published: 443
        protocol: tcp
        mode: host
    cap_add:
      - NET_BIND_SERVICE
    healthcheck:
      test: nc -zv 127.0.0.1 443
    # networks:
    #   default:
    #     ipv4_address: 172.21.0.8
    deploy:
      mode: global
      resources:
        reservations:
          cpus: "1.0"
          memory: 256M
        limits:
          cpus: "1.0"
          memory: 256M

@trajano Thanks for the advise.

The best is still to use Linux with mode: host

Why would you do that with the given limitation vs network: host? It's not a solution if only one port were to preserve the IP, but you require it for all other used ports too.

Disable the userland-proxy (on windows/mac or Docker Desktop, this isn't likely to help though), and if you have any other indirection such as a reverse-proxy container connecting to another container, for HTTP traffic it can preserve that information, while other TCP traffic needs to be configured with PROXY protocol.

You'll have a similar issue with external proxies, like Cloudflare sitting between the connecting client and your service. Again a reverse proxy like Caddy can be configured to handle that and preserve the correct information that Cloudflare forwards with the connection.

Depends on what the person needs, in my case port 80 simply redirects to 443 for my application purposes so there's no point for me to track those hits. It's also the simplest to work with on Docker and works with swarm.