Open
Description
In one of the very earliest Docker issues, #37: Add ability to throttle bandwidth by container, Ken proposes that Docker should allow per-container bandwidth throttling support. I think this idea should be reconsidered.
The current state of affairs is that people who want to do this are told to use tc on the veth for a given container. I believe this to not great place to be for a few reasons:
tcis notoriously difficult to learn, so much so that various different suites of software have tried to paper over its rough edges. All of them are unsuited for use with docker and the large number ofveths it creates.tchas to be configured at container creation time. If you are deploying at scale, containers will be brought up or down constantly, which will force you, a developer or operations person, to constantly bookkeep to keeptcup to date. In practice this means you will give up. This problem is especially bad if you create many short lived containers per minute (ahem, me).- As the Docker-centric ecosystem expands, more and more critical ops behavior will need to apply to specific containers. Right now, cluster management is focused on the allocation and distribution of containers across physical nodes, resource allotment, and load balancing / discovery. Rate limiting and more fine-grained resource allotment (ahem, disk quota) are logical next steps, and the network throttling side of all this has been neglected.
I propose that Docker manage underlying network rules (however they are represented on the underlying system), to accept (at minimum) maximum bandwidth limits for inbound and outbound traffic, per-container. This should probably be implemented as an attribute on container creation.
THOUGHTS?
Metadata
Metadata
Assignees
Labels
Type
Projects
Milestone
Relationships
Development
No branches or pull requests
Participants
Activity
xiaods commentedon Sep 21, 2016
@vincentwoo i know libnetwork will handle overall network implement, so how about move the proposal to libnetwork issues?
vincentwoo commentedon Sep 21, 2016
I'm not against the idea, but I have no idea if that's the right place or not. Can anyone else chime in? For instance, since this proposal would (should?) require changing the API surface of container creation, I'm not entirely sure it 100% belongs in libnetwork. More broadly, I think people need to decide if this is even something they want Docker to do.
justincormack commentedon Sep 21, 2016
Also the question is whether you want to limit bandwidth or prioritise. The
net_clscgroup can classify traffic fortc, but not sure where policy should be determined. It will depend on your network driver to some extent.It would definitely help for people to detail their real world use cases for this.
vincentwoo commentedon Sep 21, 2016
Great point. My real world use case is that I run many ephemeral containers, ranging in lifespan from a second to an hour, on behalf of many users, some of whom are potentially malicious.
Many of the common ways to abuse the system have been addressed, but being able to strictly limit bandwidth (not prioritize) per-container using individual quotas is one of the last remaining issues.
thaJeztah commentedon Sep 21, 2016
Also related; #4763, #9607, #16477
justincormack commentedon Sep 21, 2016
@vincentwoo are those per container limits or per some grouping of containers? (eg all the containers belonging to user X can have 1Mb/s)?
vincentwoo commentedon Sep 21, 2016
Per-container
vincentwoo commentedon Sep 21, 2016
See also #17064, "[RFE] Allow docker to report the veth interface used by a container" - stated use case on that proposal is ALSO to achieve use of
tc.aboch commentedon Sep 21, 2016
From implementation point of view I think this can be done by libnetwork itself when it programs the container's interface, so without drivers involvement.
But agree the API should be sorted out first.
For example, maybe it makes also sense a bandwidth policy per network, where one may want to create one unlimited and one limited network and place containers on each network accordingly ?
vincentwoo commentedon Sep 22, 2016
I think largely people will want to configure:
against various throttling strategies:
vincentwoo commentedon Oct 4, 2016
Hi @aboch, I saw that you self-assigned this and #21990. Can you give us a sense for where your thinking's at on this topic? Do we need to build additional consensus, etc? I'm unsure how the proposal process works in practice.
45 remaining items