Description
Output of docker version:
Client:
Version: 1.12.0
API version: 1.24
Go version: go1.6.3
Git commit: 8eab29e
Built: Thu Jul 28 22:00:36 2016
OS/Arch: linux/amd64
Server:
Version: 1.12.0
API version: 1.24
Go version: go1.6.3
Git commit: 8eab29e
Built: Thu Jul 28 22:00:36 2016
OS/Arch: linux/amd64
Output of docker info:
Containers: 155
Running: 65
Paused: 0
Stopped: 90
Images: 57
Server Version: 1.12.0
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 868
Dirperm1 Supported: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: host overlay null bridge
Swarm: active
NodeID: 0ddz27v59pwh2g5rr1k32d9bv
Is Manager: true
ClusterID: 32c5sn0lgxoq9gsl1er0aucsr
Managers: 1
Nodes: 1
Orchestration:
Task History Retention Limit: 5
Raft:
Snapshot interval: 10000
Heartbeat tick: 1
Election tick: 3
Dispatcher:
Heartbeat period: 5 seconds
CA configuration:
Expiry duration: 3 months
Node Address: 172.31.24.209
Runtimes: runc
Default Runtime: runc
Security Options: apparmor
Kernel Version: 3.13.0-92-generic
Operating System: Ubuntu 14.04.4 LTS
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 31.42 GiB
Name: ip-172-31-24-209
ID: 4LDN:RTAI:5KG5:KHR2:RD4D:MV5P:DEXQ:G5RE:AZBQ:OPQJ:N4DK:WCQQ
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Username: panj
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Insecure Registries:
127.0.0.0/8
Additional environment details (AWS, VirtualBox, physical, etc.):
Steps to reproduce the issue:
- run following service which publishes port 80
docker service create \
--name debugging-simple-server \
--publish 80:3000 \
panj/debugging-simple-server
- Try connecting with
http://<public-ip>/.
Describe the results you received:
Neither ip nor header.x-forwarded-for is the correct user's IP address.
Describe the results you expected:
ip or header.x-forwarded-for should be user's IP address. The expected result can be archieved using standalone docker container docker run -d -p 80:3000 panj/debugging-simple-server. You can see both of the results via following links,
http://swarm.issue-25526.docker.takemetour.com:81/
http://container.issue-25526.docker.takemetour.com:82/
Additional information you deem important (e.g. issue happens only occasionally):
This happens on both global mode and replicated mode.
I am not sure if I missed anything that should solve this issue easily.
In the meantime, I think I have to do a workaround which is running a proxy container outside of swarm mode and let it forward to published port in swarm mode (SSL termination should be done on this container too), which breaks the purpose of swarm mode for self-healing and orchestration.
Metadata
Metadata
Assignees
Labels
Type
Projects
Milestone
Relationships
Development
Participants
Activity
thaJeztah commentedon Aug 9, 2016
/cc @aluzzardi @mrjana ptal
mavenugo commentedon Aug 9, 2016
@PanJ can you please share some details on how debugging-simple-server determines the
ip? Also what is the expectation if a service is scaled to more than 1 replica across multiple hosts (or global mode) ?PanJ commentedon Aug 9, 2016
@mavenugo it's koa's request object which uses node's
remoteAddressfromnetmodule. The result should be the same for any other libraries that can retrieve remote address.The expectation is that
ipfield should always be remote address regardless of any configuration.marech commentedon Sep 19, 2016
@PanJ you still use your workaround or found some better solution?
sanimej commentedon Sep 19, 2016
@PanJ When I run your app as a standalone container..
docker run -it --rm -p 80:3000 --name test panj/debugging-simple-serverand access the published port from another host I get this
192.168.33.11 is the IP of the host in which I am running curl. Is this the expected behavior ?
PanJ commentedon Sep 19, 2016
@sanimej Yes, it is the expected behavior that should be on swarm mode as well.
PanJ commentedon Sep 19, 2016
@marech I am still using the standalone container as a workaround, which works fine.
In my case, there are 2 nginx intances, standalone and swarm instances. SSL termination and reverse proxy is done on standalone nginx. Swarm instance is used to route to other services based on request host.
sanimej commentedon Sep 19, 2016
@PanJ The way the published port of a container is accessed is different in swarm mode. In the swarm mode a service can be reached from any node in the cluster. To facilitate this we route through an
ingressnetwork.10.255.0.xis the address of theingressnetwork interface on the host in the cluster from which you try to reach the published port.PanJ commentedon Sep 19, 2016
@sanimej I kinda saw how it works when I dug into the issue. But the use case (ability to retrieve user's IP) is quite common.
I have limited knowledge on how the fix should be implemented. Maybe a special type of network that does not alter source IP address?
Rancher is similar to Docker swarm mode and it seems to have expected behavior. Maybe it is a good place to start.
marech commentedon Sep 20, 2016
@sanimej good idea could be add all IPs to X-Forwarded-For header if its possible then we can see all chain.
@PanJ hmm, and how your nignx standalone container communicate to swarm instance, via service name or ip? Maybe can share nginx config part where you pass it to swarm instance.
426 remaining items