Add runtime option to throttle I/O/network bandwidth? #4763

victorhooi · 2014-03-19T03:55:04Z

I know in Docker you can throttle memory using the -m option, and you can throttle CPU shares with -c:

http://docs.docker.io/en/latest/reference/run/#runtime-constraints-on-cpu-and-memory

Would it be possible to add an option in Docker to throttle I/O?

I believe the underlying cgroups lets you set limits on blkio devices:

https://www.kernel.org/doc/Documentation/cgroups/blkio-controller.txt

It would be neat if this could be integrated into Docker, as a runtime option?

The text was updated successfully, but these errors were encountered:

crosbymichael · 2014-04-02T23:47:12Z

I like this. Do you know of a small subset that we can implement easily and expose to a user?

vmarmol · 2014-04-29T22:23:54Z

In general, it supports giving each user a "weight" on disk time (similar to CPU shares) or giving an absolute max on IO operations. I think the first one makes more sense and is closer to what Docker does today for other resources

unclejack · 2014-05-12T14:46:25Z

ping @crosbymichael

dbason · 2014-09-30T22:47:23Z

Hi just wondering if there is any further work here? It looks like there has been some blkio stuff added to libcontainer, and this is something we're very interested in.

OmgImAlexis · 2016-09-14T02:56:56Z

Any updates on this?

cpuguy83 · 2016-09-14T16:11:14Z

@OmgImAlexis blkio is supported.
Network is a different beast.

      --device-read-bps value       Limit read rate (bytes per second) from a device (default [])
      --device-read-iops value      Limit read rate (IO per second) from a device (default [])
      --device-write-bps value      Limit write rate (bytes per second) to a device (default [])
      --device-write-iops value     Limit write rate (IO per second) to a device (default [])

drzraf · 2017-06-08T21:48:58Z

Does this issue apply to limiting bandwidth usage of "docker pull" (download are issued by the dockerd process)?
If yes, then : docker/for-mac#357

thaJeztah · 2017-06-08T22:56:32Z

@drzraf you can configure the number of concurrent downloads and uploads in the daemon configuration, that option has been around since docker 1.12 (see #22445); to disable concurrent downloads (i.e. only one layer downloaded at a time), set the following in your daemon.json configuration file (you can set this in the advanced settings in the Docker for Mac configuration panel);

{
    "max-concurrent-downloads": 1,
    "max-concurrent-uploads": 1
}

See the daemon docs here; https://github.com/moby/moby/blob/67fdf574d5acd6ddccb6ece0ffe0ace1c1608712/docs/reference/commandline/dockerd.md

drzraf · 2017-06-09T00:23:20Z

Yeah, that did the trick! Thank you for the pointer.

Let me write down here the two other suggestions about this (docker/for-mac#357):

resume downloads where they stop (wget/curl can do that for decades)
give hints in the documentation about how to throttle download bandwidth
My intents of running dockerd LD_PRELOAD wrapped by trickle didn't work, but they should have been.

japrogramer · 2017-12-31T05:31:44Z

@cpuguy83 I have tried to use blkio but I get blkio_config Additional property blkio_config is not allowed when I try stack deploy

 56   node:                                                                                                                                                                                                                                                                                                                                                                                                                               
 57     build:                                                                                                                                                                                                                                                                                                                                                                                                                            
 58       context: /home/archangel/git/my_projects/apple                                                                                                                                                                                                                                                                                                                                                                                  
 59       dockerfile: ./compose/node/Dockerfile-dev                                                                                                                                                                                                                                                                                                                                                                                       
 60     blkio_config:                                                                                                                                                                                                                                                                                                                                                                                                                     
 61       weight: 100                                                                                                                                                                                                                                                                                                                                                                                                                     
 62       weight_device:                                                                                                                                                                                                                                                                                                                                                                                                                  
 63         - path: /dev/sdb                                                                                                                                                                                                                                                                                                                                                                                                              
 64           weight: 100                                                                                                                                                                                                                                                                                                                                                                                                                 
 65       device_read_bps:                                                                                                                                                                                                                                                                                                                                                                                                                
 66         - path: /dev/sdb                                                                                                                                                                                                                                                                                                                                                                                                              
 67           rate: '12mb'                                                                                                                                                                                                                                                                                                                                                                                                                
 68       device_read_iops:                                                                                                                                                                                                                                                                                                                                                                                                               
 69         - path: /dev/sdb                                                                                                                                                                                                                                                                                                                                                                                                              
 70           rate: 120                                                                                                                                                                                                                                                                                                                                                                                                                   
 71       device_write_bps:                                                                                                                                                                                                                                                                                                                                                                                                               
 72         - path: /dev/sdb                                                                                                                                                                                                                                                                                                                                                                                                              
 73           rate: '1024k'                                                                                                                                                                                                                                                                                                                                                                                                               
 74       device_write_iops:                                                                                                                                                                                                                                                                                                                                                                                                              
 75         - path: /dev/sdb                                                                                                                                                                                                                                                                                                                                                                                                              
 76           rate: 30                                                                                                                                                                                                                                                                                                                                                                                                                    
 77     deploy:                                                                                                                                                                                                                                                                                                                                                                                                                           
 78       restart_policy:                                                                                                                                                                                                                                                                                                                                                                                                                 
 79         condition: any                                                                                                                                                                                                                                                                                                                                                                                                                
 80     command: ash -c "PATH=$$(npm bin):$$PATH && cd /app/frontclient && yarn watch-css & yarn start"                                                                                                                                                                                                                                                                                                                                   
 81     image: apple_node                                                                                                                                                                                                                                                                                                                                                                                                                 
 82     ports:                                                                                                                                                                                                                                                                                                                                                                                                                            
 83     - published: 3000                                                                                                                                                                                                                                                                                                                                                                                                                 
 84       target: 3000                                                                                                                                                                                                                                                                                                                                                                                                                    
 85     - published: 3001                                                                                                                                                                                                                                                                                                                                                                                                                 
 86       target: 3001                                                                                                                                                                                                                                                                                                                                                                                                                    
 87     volumes:                                                                                                                                                                                                                                                                                                                                                                                                                          
 88     - /home/archangel/git/my_projects/apple/frontclient:/app/frontclient:rw                                                                                                                                                                                                                                                                                                                                                           
 89     - /app/frontclient/node_modules

thaJeztah · 2018-01-02T11:07:21Z

@japrogramer the blkio and device-xx options are not yet supported for services/stacks; see #25303

mellertson · 2021-04-16T03:33:46Z

Any idea on if and when this might be supported?

ZWindL · 2021-04-20T08:36:36Z

Maybe that project helps
https://github.com/Silcet/docker-tc

But I'm encountering a memory leaking problem with it, still struggling.

Adrian-at-CrimsonAzure · 2021-07-16T15:04:56Z

I run a few various downloaders for various things and I'd like to cap each one to X mbps so that one doesn't choke out the rest. This issue has been around for 7 years and with swarm becoming so popular I can imaging that it's a feature that would make sense in the data center as well. Some containers don't need priority/high throughput and I'd like to be able to limit them.

luckman212 · 2023-10-22T22:03:17Z

Coming up on 10 years after this issue was raised, is there anything that can help rate limit a container nowadays?

JoaoOtavioS · 2023-10-22T22:38:52Z

Yes, try:

container_name
container_pid = State.Pid // docker inspect .......

rm -rf /var/run/netns/container_name && ln -sfT /proc/container_pid/ns/net /var/run/netns/container_name

// delete if exists.

tc -n container_name qdisc del dev eth0 root

// example (limit 10mbit)

tc -n container_name qdisc add dev eth0 handle 10: root tbf limit 10mbit burst 10mbit rate 10mbit

luckman212 · 2023-10-23T00:43:47Z

Thanks @JoaoOtavioS

I tried some variant of those commands, without luck.

# container_id=$(docker inspect speedtest_cfg -f '{{.State.Pid}}')
# mkdir -p /var/run/netns
# ln -sfT /proc/$container_id/ns/net /var/run/netns/speedtest_cfg
# ip netns exec speedtest_cfg ip -br -c link
lo               UNKNOWN        00:00:00:00:00:00 <LOOPBACK,UP,LOWER_UP>
sit0@NONE        DOWN           0.0.0.0 <NOARP>
eth0@if32        UP             02:42:ac:12:00:05 <BROADCAST,MULTICAST,UP,LOWER_UP>
# tc -n speedtest_cfg qdisc del dev eth0 root
RTNETLINK answers: No such file or directory
# tc -n speedtest_cfg qdisc del dev eth0@if32 root
Cannot find device "eth0@if32"
# tc -n speedtest_cfg qdisc add dev eth0 handle 10: root tbf limit 10mbit burst 10mbit rate 10mbit
RTNETLINK answers: No such file or directory
# tc -n speedtest_cfg qdisc add dev eth0@if32 handle 10: root tbf limit 10mbit burst 10mbit rate 10mbit
Cannot find device "eth0@if32"

JoaoOtavioS · 2023-10-24T14:54:02Z

Thanks @JoaoOtavioS

I tried some variant of those commands, without luck.

# container_id=$(docker inspect speedtest_cfg -f '{{.State.Pid}}')
# mkdir -p /var/run/netns
# ln -sfT /proc/$container_id/ns/net /var/run/netns/speedtest_cfg
# ip netns exec speedtest_cfg ip -br -c link
lo               UNKNOWN        00:00:00:00:00:00 <LOOPBACK,UP,LOWER_UP>
sit0@NONE        DOWN           0.0.0.0 <NOARP>
eth0@if32        UP             02:42:ac:12:00:05 <BROADCAST,MULTICAST,UP,LOWER_UP>
# tc -n speedtest_cfg qdisc del dev eth0 root
RTNETLINK answers: No such file or directory
# tc -n speedtest_cfg qdisc del dev eth0@if32 root
Cannot find device "eth0@if32"
# tc -n speedtest_cfg qdisc add dev eth0 handle 10: root tbf limit 10mbit burst 10mbit rate 10mbit
RTNETLINK answers: No such file or directory
# tc -n speedtest_cfg qdisc add dev eth0@if32 handle 10: root tbf limit 10mbit burst 10mbit rate 10mbit
Cannot find device "eth0@if32"

do you run this command inside the container? or outside?
*I ask this because I read another post of yours citing that you tried to execute it, it seems

jessfraz mentioned this issue Feb 25, 2015

Abilitiy to limit network bandwidth #7182

Closed

jessfraz changed the title ~~Add runtime option to throttle I/O?~~ Add runtime option to throttle I/O/network bandwidth? Feb 25, 2015

jessfraz added kind/feature Functionality or other elements that the project doesn't currently have. Features are new and shiny /system/networking labels Feb 25, 2015

jessfraz added area/networking and removed system/networking labels Sep 8, 2015

thaJeztah mentioned this issue Oct 18, 2015

Network traffic control of docker #16477

Closed

beenanner mentioned this issue Jun 29, 2016

Throttle the connection sitespeedio/sitespeed.io#895

Closed

thaJeztah mentioned this issue Sep 21, 2016

Proposal: Docker should assist in bandwidth limiting containers #26767

Open

drzraf mentioned this issue Jun 8, 2017

Pulling repositories fail with DNS resolve error docker/for-mac#357

Closed

This comment has been minimized.

Sign in to view

slimsag mentioned this issue Jul 21, 2020

Services: Add a generic form of network throttling sourcegraph/sourcegraph#5736

Open

3 tasks

This comment has been minimized.

Sign in to view

luckman212 mentioned this issue Oct 23, 2023

Ability to limit bandwidth from the Docker container? alexjustesen/speedtest-tracker#872

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add runtime option to throttle I/O/network bandwidth? #4763

Add runtime option to throttle I/O/network bandwidth? #4763

victorhooi commented Mar 19, 2014

crosbymichael commented Apr 2, 2014

vmarmol commented Apr 29, 2014

unclejack commented May 12, 2014

dbason commented Sep 30, 2014

OmgImAlexis commented Sep 14, 2016

cpuguy83 commented Sep 14, 2016

drzraf commented Jun 8, 2017

thaJeztah commented Jun 8, 2017

drzraf commented Jun 9, 2017

japrogramer commented Dec 31, 2017 •

edited

thaJeztah commented Jan 2, 2018

This comment has been minimized.

This comment has been minimized.

mellertson commented Apr 16, 2021

ZWindL commented Apr 20, 2021

Adrian-at-CrimsonAzure commented Jul 16, 2021

luckman212 commented Oct 22, 2023

JoaoOtavioS commented Oct 22, 2023

luckman212 commented Oct 23, 2023

JoaoOtavioS commented Oct 24, 2023 •

edited

Add runtime option to throttle I/O/network bandwidth? #4763

Add runtime option to throttle I/O/network bandwidth? #4763

Comments

victorhooi commented Mar 19, 2014

crosbymichael commented Apr 2, 2014

vmarmol commented Apr 29, 2014

unclejack commented May 12, 2014

dbason commented Sep 30, 2014

OmgImAlexis commented Sep 14, 2016

cpuguy83 commented Sep 14, 2016

drzraf commented Jun 8, 2017

thaJeztah commented Jun 8, 2017

drzraf commented Jun 9, 2017

japrogramer commented Dec 31, 2017 • edited

thaJeztah commented Jan 2, 2018

This comment has been minimized.

This comment has been minimized.

mellertson commented Apr 16, 2021

ZWindL commented Apr 20, 2021

Adrian-at-CrimsonAzure commented Jul 16, 2021

luckman212 commented Oct 22, 2023

JoaoOtavioS commented Oct 22, 2023

luckman212 commented Oct 23, 2023

JoaoOtavioS commented Oct 24, 2023 • edited

japrogramer commented Dec 31, 2017 •

edited

JoaoOtavioS commented Oct 24, 2023 •

edited