Open
Description
Description of problem:
When issuing UDP connections from docker container to docker host, no incoming UDP packets are received in container.
UDP works for accessing external resources.
TCP works for accessing docker host.
docker version
:
Mostly tested on
Client:
Version: 1.8.0-dev
API version: 1.20
Go version: go1.4.2
Git commit: b900aaa
Built: Fri Jul 17 15:15:47 UTC 2015
Still actual in:
Go version: go1.4.2
Git commit: 9d3ad6d
Built: Wed Jul 29 16:26:04 UTC 2015
docker info
:
Containers: 283
Images: 687
Storage Driver: overlay
Backing Filesystem: extfs
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 4.1.2███████
Operating System: Gentoo/Linux
CPUs: 8
Total Memory: 7.746 GiB
Name: ███████
ID: 7WYK:23Z2:J7KR:JFF2:QMLD:E5Z6:FF5O:LAAA:XOW4:56ZR:BOSD:WWAI
uname -a
: Linux ███████ 4.1.2-███████ #2 SMP Mon Jul 20 14:13:03 CEST 2015 x86_64 Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz GenuineIntel GNU/Linux
Environment details (AWS, VirtualBox, physical, etc.): physical
How reproducible:
Steps to Reproduce:
- you need to start any UDP echo server (nmap's ncat works just fine:
ncat -e /bin/cat -k -u -l 5151
, also I rechecked withudpqotd
, yeah that one from old Perl Cookbook) on Docker host outside of Docker (not tested starting server in docker container) - you need to test it with "netcat -u ip port" to see if you receive incoming traffic
- you need to start "netcat -u ip port" in your favorite minimal-shell container and try accessing host, then verify your results by testing various TCP services and UDP echo services located outside of your docker host.
Actual Results:
- No incoming traffic received from UDP server when running in on docker host
- Incoming traffic successfully received from UDP server when running in on outside host
Expected Results:
- Incoming traffic received into docker from UDP server running in on docker host; it doesn't looks like a security problem because host is still accessible via TCP.
Additional info:
- test container baseimage ubuntu:14.04
- firewalls are off, accessing TCP ports at docker host works
- no --net or flags given . Routing/traces seem to be ok. No --userland-proxy option is set in daemon config.
- no sysctl changes done; reboot was performed to verify results.
- UPD: confirms on docker 1.5, docker 1.6.
- UPD: port is accessible, when accessing host by docker0 ip, but I need to access it by host host IP.
Metadata
Metadata
Assignees
Type
Projects
Milestone
Relationships
Development
No branches or pull requests
Activity
chpio commentedon Jan 16, 2016
same here with docker v1.9.1 ("a34a1d5-dirty") @ Arch
👍 please fix this bug
chpio commentedon Jan 17, 2016
is there a workaround for this?
maybe it's an iptables bug. i have no other iptables rules, besides the docker rules, 2 networks (docker0 + 1 other). My host has the ip 172.16.0.1/16.
LK4D4 commentedon Oct 18, 2016
ping @mrjana @sanimej @aboch
Could you pls try and close or fix issue?
timothyklim commentedon May 13, 2017
Unbelievable, bug still in current docker version and no one cares from docker dev team (@thaJeztah).
mavenugo commentedon May 13, 2017
@timothyklim sorry about that. this fell out of radar. We will take a look at it soon.
timothyklim commentedon Jun 21, 2017
1 month later and still no progress
chrisxaustin commentedon Jun 28, 2017
I'm having a [possibly] related issue.
My app receives UDP syslog messages from most sources with the original source address preserved, but one device has the source rewritten to the docker0 interface's IP (seen by running tcpdump on docker0).
I tried disabling the userland proxy as an experiment. The ones that were working before still are, but that one source never makes it to docker0 now, which is obviously concerning.
Ubuntu 16.04.2, docker 1.12.6, docker-compose 1.8.0, launched w/ docker-compose up
EDIT: Running
conntrack -D -p udp
seems to have fixed it for me locally, so potentially unrelated.cizixs commentedon Aug 21, 2017
The problem happens because host has multiple network interfaces(
eth0
anddocker0
in this case), and when replying UDP, server choose the wrong IP as the source address.Running
tcpdump
to capture packets on the wire gives:172.17.0.1
(docker0 ip address), instead of172.16.13.13
as we expectedAs we can see the real cause is: the second packet chooses the wrong source address! And somehow, client does not recognize it, thinks it is INVALID, does not pass it to the application, but instead replies with a ICMP packet.
Choosing packet source address is kernel's work, for UDP, by default it will use the IP address of the interface the packet will be sent to. And as to why kernel drops the second packet, is because
netfilter
records valid UDP connection(since UDP is connectionless, it only meansnetfilter
stores <sourcess IP, source Port, destination IP, destination Port>), and drops those that are invalid(not in connection table).As for the first problem, socket can actually control which source address should be used with a combination of
sendmsg
andIP_PKTINFO
socket option. See how powerDNS does it: https://blog.powerdns.com/2012/10/08/on-binding-datagram-udp-sockets-to-the-any-addresses/In conclusion, this is not a problem caused by docker(although it does create a
docker0
which is the beginning of it) . And the solution comes in many ways:lagorsse commentedon Jul 24, 2018
I don't know if my issue is link or not but I can't send or receiver udp message in the docker container, works fine with tcp.
nc -l -u -p 8888
nc -u 127.0.0.1 8888
I am using
EXPOSE 8888/udp
or -p 8888:8888/udp
Netcat works fine with tcp so I don't get why it doesn't work over udp.
I am using osx sierra.
REPTILEHAUS commentedon Aug 31, 2018
Also hitting this issue.. any fix for this ?
running osx Sierra too
donovankim commentedon Nov 28, 2018
35 remaining items