Open
Description
I'm running Drone within Kubernetes on Azure, and by default it seems keen to use Google DNS for the docker build container:
time="2018-09-11T14:15:45.926616198Z" level=info msg="No non-localhost DNS nameservers are left in resolv.conf. Using default external servers: [nameserver 8.8.8.8 nameserver 8.8.4.4]"
time="2018-09-11T14:15:45.926647397Z" level=info msg="IPv6 enabled; Adding default IPv6 external servers: [nameserver 2001:4860:4860::8888 nameserver 2001:4860:4860::8844]"
This then fails at:
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gz
WARNING: Ignoring http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gz: temporary error (try again later)
WARNING: Ignoring http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gz: temporary error (try again later)
If I set custom_dns
to my kube-dns
(10.0.0.10
) service, everything works fine. Docker itself defaults to the host DNS settings:
https://docs.docker.com/config/containers/container-networking/#dns-services
I think this plugin should follow the default Docker behavior instead of defaulting to Google.
Activity
jpds commentedon Sep 11, 2018
The DinD container on the agent itself has:
jpds commentedon Sep 11, 2018
The above is with
17.12.0-ce-dind
from the Helm chart here: https://github.com/helm/charts/tree/master/stable/drone/Testing with
18.06.1-ce-dind
now.jpds commentedon Sep 11, 2018
This needs #194 to update the Docker version used by the plugin.
bradrydzewski commentedon Sep 11, 2018
This plugin does follow the default docker behavior.
The reason that the agent and Docker container have different dns configurations are because they are attached to different networks. Your agent is using the default network driver (bridge or whatever), however, all drone pipeline steps are attached to a user-defined network:
The user-defined networks in Docker using an embedded DNS server [1] and therefore have different rules for DNS configuration than the default bridge network. So there really isn't anything we can do in this plugin, but you may be able to alter the host machine configuration in some way to ensure the correct DNS server is used inside the plugin (I'm not sure, though, this is outside my area of expertise).
Alternatively you can use the global environment variables Enterprise feature [2] and set
PLUGIN_CUSTOM_DNS
as a global environment variable, so that you do not need to set the value in your yaml file.[1] https://docs.docker.com/v17.09/engine/userguide/networking/configure-dns/
[2] http://docs.drone.io/configure-global-environment/
jpds commentedon Sep 11, 2018
Sure, this is what I agree should be happening, but none of my other Kubernetes containers are doing this and suddenly defaulting to going to Google DNS.
The person at #176 says that they had (undefined) issues with
yum
which echos the issues I'm seeing withapk
in the DinD container, so perhaps the upgrade would fix this (I'm currently deploying a image of this plugin in my private repo to test this theory).c2h5oh commentedon Feb 28, 2019
Btw I'm seeing the same without kubernetes. Docker container started in dind (I know it's not the best idea) also default to goodle dns and is unable to resolve pipeline services. - this is on 1.0 RC4
sneak commentedon Feb 12, 2020
I'm running into this issue today. It seems to me a major security issue to leak DNS hostname requests unencrypted to a third party server that the user never selected or configured! My LAN DNS server encrypts all DNS traffic to the WAN; using unencrypted DNS to unauthorized servers is incorrect behavior.
ashwilliams1 commentedon Feb 12, 2020
This plugin uses the official docker-in-docker images as its base image, which starts docker-in-docker with default settings. The docker-in-docker deaemon is responsible for default DNS settings. If the docker-in-docker daemon is not using the correct settings or is not inheriting the settings from the parent container, and you think it should do so automatically, you should open an issue with the moby / docker team since they maintain the code responsible for DNS.
Alternatively you can configure custom DNS for the docker-in-docker plugin by globally setting the PLUGIN_CUSTOM_DNS variable. You can globally set environment variables for plugins using DRONE_RUNNER_ENVIRON.
tboerger commentedon Feb 12, 2020
IMHO this is a wontfix for this plugin, we are using the defaults of docker.
If this is an issue for you report it upstream.
mildebrandt commentedon Feb 12, 2020
While passing the issue upstream is the easy thing to do, I don't think it's the right thing in this case. People writing build scripts and using this plugin shouldn't need to be concerned about which DNS is being used. They expect each plugin to behave the same....if a network call works in one plugin, it should work in another.
The fact that this is performing a docker-in-docker is an implementation detail that the user should be isolated from knowing. The build engineer just wants to build a container.
If the solution is to set the PLUGIN_CUSTOM_DNS environment variable, that should be front and center in big bold letters as a prerequisite to using this plugin. If not, every new user that runs into this issue has to hop on Google and figure out why their container fails to build due to a server not being found.
Ideally, the plugin would pass its DNS settings into the child container every time.
ashwilliams1 commentedon Feb 12, 2020
how? If it were straight forward then I wonder why docker is not already doing this?
bradrydzewski commentedon Feb 12, 2020
This assumes there is something we can do in this codebase to solve this problem. It is not yet clear to me that we can solve this issue without docker's help but I am certainly open to suggestions.
For reference this is the existing issue for dns issues in docker-in-docker:
moby/moby#20037
mildebrandt commentedon Feb 12, 2020
The issue stems from the fact that drone uses a custom network which kicks-in docker's DNS service which isn't available to dind containers.
I also see that Drone treats certain containers as special already, including this one:
https://github.com/drone/drone/blob/d84b79f027247284d26066041b1521414f68c631/operator/runner/runner.go#L255
So, how about creating another special category which passes the host's DNS settings into a plugin via
--dns
and related switches? That way the child containers will get the proper settings since the nameserver wouldn't be a local address.blopker commentedon Feb 12, 2020
I'd also be interested in a Docker plugin that uses the host's daemon instead of docker-in-docker. For self hosted deployments this would have a few benefits, like having a persistent layer cache and it would be easier to configure /etc/docker/daemon.json.
16 remaining items