New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Proposal: Docker should assist in bandwidth limiting containers #26767
Comments
@vincentwoo i know libnetwork will handle overall network implement, so how about move the proposal to libnetwork issues? |
I'm not against the idea, but I have no idea if that's the right place or not. Can anyone else chime in? For instance, since this proposal would (should?) require changing the API surface of container creation, I'm not entirely sure it 100% belongs in libnetwork. More broadly, I think people need to decide if this is even something they want Docker to do. |
Also the question is whether you want to limit bandwidth or prioritise. The It would definitely help for people to detail their real world use cases for this. |
Great point. My real world use case is that I run many ephemeral containers, ranging in lifespan from a second to an hour, on behalf of many users, some of whom are potentially malicious. Many of the common ways to abuse the system have been addressed, but being able to strictly limit bandwidth (not prioritize) per-container using individual quotas is one of the last remaining issues. |
@vincentwoo are those per container limits or per some grouping of containers? (eg all the containers belonging to user X can have 1Mb/s)? |
Per-container |
See also #17064, "[RFE] Allow docker to report the veth interface used by a container" - stated use case on that proposal is ALSO to achieve use of |
From implementation point of view I think this can be done by libnetwork itself when it programs the container's interface, so without drivers involvement. But agree the API should be sorted out first. For example, maybe it makes also sense a bandwidth policy per network, where one may want to create one unlimited and one limited network and place containers on each network accordingly ? |
I think largely people will want to configure:
against various throttling strategies:
|
@vincentwoo To be clear, this is not on the roadmap yet. As @justincormack suggested, it is important to hear use-cases where this will be used (thanks for providing yours), ideally from more than one party so that we can can come out with an hopefully good abstraction to accomodate the most common cases. Besides that, community is at any time free/welcome to push a PR with a proposed implemenation to address this. That can also speed up the process. |
+1, we would use this in a second at wercker and I agree exactly with vincent's 3 reasons for it not being viable to manage this outside of docker |
+1, I agree as well. It would be great especially if a container would to be compromised and a hacker launched an DOS attack from the container to an outside party. |
+1 for this as well. We are planning to use Filebeat as a Docker Container. We use NIC bonding to provide a single network interface for application and logs. Isolating one NIC from the bond is something that we cannot control, as we use on-demand VMs (standardized). So, isolating the Filebeat traffic from App Services would be a great use of this feature. |
+1 - this should be a docker run option at least for bridged networking. Initially, something like: underneath, it can configure tc and tear it down. |
Hi, |
I'm uh, a bit confused in following your explanations, but it seems to me that this idea will only work for network bandwidth into a container on a very predefined port. |
prototype done for predefined port and also for a full container. I mean if we start a container by exposing a public host port then this new feature start limiting egress and ingress n/w io based on the port and if we just start a container without specifiying/exposing a port then it will limit based on the container net_cls cgroup class id |
@shivacherukuri how did you get around the issue in #19802? I'm running Ubuntu 16.04 with 4.4.0-47-generic and net_cls is not being marked on packets coming from the container, although it works fine on any process outside the container. |
This ticket has been quiet for awhile, even after at least some work on a potential implementation approach. Is there any discussion happening behind the scenes regarding bandwidth controls on containers and how this might actually be implemented. In my case, I'm mostly looking for a way to throttle individual containers, with the realization that outbound is likely easier than inbound, to control in a meaning full way. |
@aboch I have no idea how you just got unassigned, but apparently I can't add you back. |
Any updates? |
/cc @fcrisciani @abhinandanpb any thoughts? |
OP here, would love to hear what next steps for this request might be. Does the tram need to see more user need? |
My use-case: I regularly run backups using docker containers. I'd like to limit the (outbound) bandwidth of the container to avoid saturating my network pipe. As many tools are using http or worse, https for communication, putting tc rules at the router level based on protocol is not functional. |
Hi, just checking back in. I still would love to know what next steps are relevant for this request. |
Also have a use-case to filter traffic (all traffic) originating in a container into a class on the host corresponding to that container; this for managing some rate-limiting but with borrowing between specific classes. Getting the veth would probably be enough (or having docker set up mark iptables entries for me) since I can hook into the lifecycle at the point of docker creation and take care of the rest. |
I would like to see something like Docker's CPU throttling for bandwidth. If I understand correctly, CPU limits allows for a relative amount to be allocated and Docker will only enforce the limit when there is contention for the resource. That would be ideal because it would allow me to guarantee a percentage of the bandwidth in peak times while allowing the network to fully leveraged during non-peak times. |
This would be an incredibly useful feature, especially for multi-tenant supported environments. The main need is bytes/time window. Where are we at on this? Any update? |
@jmwilkinson No one has taken this work on AFAIK. |
any insight comment on it? |
This comment was marked as spam.
This comment was marked as spam.
I'm interested in funding this work if anyone's looking to take this on. Feel free to email me (address in my profile). |
This comment was marked as spam.
This comment was marked as spam.
Till Oct 15, 2018, I am damn committed. I will try talking to you after that.. seems like many are still interested in this feature. we may need to brainstorm on this topic again if we need to work on the implementation. |
Any updates? |
1 similar comment
Any updates? |
No one's mailed me but I'm still interested in funding this work. |
@vincentwoo can we implement it with plugin? |
Any updates on this? |
Any updates on this ? |
Any updates??? This is one of the long-requested things that are possibly not even considered yet |
+1 I would use this for testing and prototyping scenarios where I need to simulate a slow network connection. |
Any updates??? |
+1 |
In one of the very earliest Docker issues, #37: Add ability to throttle bandwidth by container, Ken proposes that Docker should allow per-container bandwidth throttling support. I think this idea should be reconsidered.
The current state of affairs is that people who want to do this are told to use
tc
on theveth
for a given container. I believe this to not great place to be for a few reasons:tc
is notoriously difficult to learn, so much so that various different suites of software have tried to paper over its rough edges. All of them are unsuited for use with docker and the large number ofveth
s it creates.tc
has to be configured at container creation time. If you are deploying at scale, containers will be brought up or down constantly, which will force you, a developer or operations person, to constantly bookkeep to keeptc
up to date. In practice this means you will give up. This problem is especially bad if you create many short lived containers per minute (ahem, me).I propose that Docker manage underlying network rules (however they are represented on the underlying system), to accept (at minimum) maximum bandwidth limits for inbound and outbound traffic, per-container. This should probably be implemented as an attribute on container creation.
THOUGHTS?
The text was updated successfully, but these errors were encountered: