Please make sure your IO scheduler is cfq
cat /sys/block/sdx/queue/scheduler replace sdx with your block device.

@coolljt0725
I followed you suggestion to change IO scheduler to cfq, which seems take effect. However, it is not my expected result.
I started two containers as follows

$ docker run -ti --name c1 --blkio-weight 10 ubuntu:14.04 /bin/bash
$ docker run -ti --name c2 --blkio-weight 1000 ubuntu:14.04 /bin/bash

but the output is as follows
C1:

root@75dcf0be5526:/# time dd if=/dev/zero of=test.out bs=1M count=1024 oflag=direct      
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 2.49889 s, 430 MB/s

real    0m2.501s
user    0m0.000s
sys 0m0.404s

C2:

root@41435bed761f:/# time dd if=/dev/zero of=test.out bs=1M count=1024 oflag=direct      
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 1.28073 s, 838 MB/s

real    0m1.286s
user    0m0.000s
sys 0m0.396s

Is there any more potential problems? Or the proportion of the weight is just like this? Thx

@SunWeicheng0001 run cli in the container
while true;do time dd if=/dev/zero of=test.out bs=1M count=100 oflag=direct;done
I can see these on my testing

104857600 bytes (105 MB) copied, 2.79353 s, 37.5 MB/s
real    0m2.795s
user    0m0.000s
sys     0m0.053s
100+0 records in
100+0 records out

100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 206.645 s, 507 kB/s
real    3m26.648s
user    0m0.002s
sys     0m0.060s

@coolljt0725 WOW! How can you do this?
Just start two containers with 10 and 1000 blkio.weight and run the command while true;do time dd if=/dev/zero of=test.out bs=1M count=100 oflag=direct;done in both containers in the same time?
However, I found that it is the same to the before. Please tell me how to reproduce your test,thanks a lot!

@SunWeicheng0001

Just start two containers with 10 and 1000 blkio.weight and run the command while true;do time dd if=/dev/zero of=test.out bs=1M count=100 oflag=direct;done in both containers in the same time?

Yes, I just do this

Does it work for you too @SunWeicheng0001 ? It never work for me also it gets devided 50-50?

I try on AWS ubuntu VM and as below

$ docker run -ti --name c1 --blkio-weight 300 ubuntu:14.04 /bin/bash
$ docker run -ti --name c2 --blkio-weight 600 ubuntu:14.04 /bin/bash
and
while true;do time dd if=/dev/urandom of=test1.out bs=1M count=50 oflag=direct; rm test1.out;done

see also my comment here: #14466 (comment)

this is still not working on

 Version:      17.10.0-ce
 API version:  1.33 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   f4ffd25
 Built:        Tue Oct 17 19:05:23 2017
 OS/Arch:      linux/amd64
 Experimental: true

I'm also seeing the same result as @odedpriva sees on a DigitalOcean droplet; looking at @coolljt0725's suggestion, there looks to be no scheduler set;

cat /sys/block/vda/queue/scheduler 
none

the cgroup looks to be set correctly for the container;

cat /sys/fs/cgroup/blkio/docker/a144caf4fa2b6dd1878685028f4c035af21919f903ef9d14aefe17c3f3275949/blkio.weight
10

This droplet is using overlay2 as storage driver (in case it's relevant here);

ping @coolljt0725 any suggestions?

@odedpriva Have you set your IO scheduler to cfq?

this is my current configuration ? is it o.k ?

/ # cat /sys/block/sda/queue/scheduler
noop [deadline] cfq

I also find the same issue. No matter how to set the --blkio-weight-device "/dev/xvda:100" blkio-weight 100 , the write speed of the two containers are the same.

root@SZX1000353068:/mnt/go/src/k8s.io/kubernetes# docker version
Client:
 Version:      17.06.1-ce
 API version:  1.30
 Go version:   go1.8.3
 Git commit:   874a737
 Built:        Thu Aug 17 22:53:09 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.06.1-ce
 API version:  1.30 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   874a737
 Built:        Thu Aug 17 22:51:03 2017
 OS/Arch:      linux/amd64
 Experimental: false

Is there any update on this issue? I am facing the same problem :(

Hi,

i think the two containers should write to different file, but not both output to test.out

just try, works

time dd if=/dev/zero of=test.out bs=1M count=1024 oflag=direct
time dd if=/dev/zero of=test1.out bs=1M count=1024 oflag=direct 

l have same problems,my docker version is 19.03,How can I solve this problem

I am seeing the same issue with Docker version 20.10.7.

By looking at the file like cat /sys/fs/cgroup/blkio/docker/<container_ID>/blkio.weight it seems as if the weight has been applied. But the read speed of two test containers is identical.

I'm seeing the same issue with 20.10.20.

--blkio-weight has no effect on default AWS/Ubuntu with Docker version 20.10.20, build 9fdeb9c3de2f2d9f5799be373f27b2f9df44609d.

The example from docker's webpage results in identical read/write times (whereas docker's webpage claims they should be different if different blkio-weight vallues are used.):
https://docs.docker.com/engine/reference/run/#block-io-bandwidth-blkio-constraint

The reason is that most kernels dont have CONFIG_BLK_CGROUP=y enabled by default or CONFIG_CFQ_GROU_IOSCHED= is not enabled.

I found this information helpful:
https://android.googlesource.com/kernel/common/+/bcmdhd-3.10/Documentation/cgroups/blkio-controller.txt

I think it would be helpful for docker to throw an warning if --blkio-weight is specified but the underlaying scheduler is not available. The documentation on the webpage should be updated.

Also, CFQ was removed from Linux 5.x. BFQ works instead.

When looking at the cgroup slice in /sys/fs/cgroup/blkio/ then there was no blkio.weight file either - so docker failed to create it (for above mentioned reasons).

on cgroup v2 there was the io.weight file (with correct content: 1) but it had no effect. I guess because the /sys/class/block scheduler was not set to BFQ.

I got it working on default AWS/ubuntu 5.15.0-1022-aws / docker 20.10.21:

Make sure it's cgroup2 (NOT cgroup; see below)

[[ -d /sys/fs/cgroup/blkio ]] && echo "ERROR: This looks like cgroup v1" || echo OK

Kernel Config:

grep BFQ /boot/config*
CONFIG_IOSCHED_BFQ=m
CONFIG_BFQ_GROUP_IOSCHED=y

Enable BFQ on all block devices:

modprobe bfq
find -L /sys/class/block -maxdepth 3 -name scheduler 2>/dev/null | while read x; do echo bfq >"$x"; done

Now the example works:

docker run --rm --blkio-weight=1000 ubuntu dd bs=1M count=10024 if=/dev/zero of=/dump.dat oflag=direct status=progress
4221566976 bytes (4.2 GB, 3.9 GiB) copied, 31 s, 136 MB/s

docker run --rm --blkio-weight=10 ubuntu dd bs=1M count=10024 if=/dev/zero of=/dump.dat oflag=direct status=progress
36700160 bytes (37 MB, 35 MiB) copied, 12 s, 3.0 MB/s

Weight ratio is 10 vs 1000 (or 1 vs 100 using cgroup's v2 notation). Expecting the 2nd contianer to be 100x slower at 1 MB/s but i'm ok with 3.0MB/s vs 130 MB/s. Beggar's cant be choosers.

I could not get it working with cgroup v1 and --blkio-weight=. Docker keeps complaining and bleeding all over the place.

WARNING: Your kernel does not support Block I/O weight or the cgroup is not mounted. Weight discarded.

It's obviously a lie. It works if i create the cgroup manually and attach it with --cgroup-parent=... (docker still complains about that it's not supported but it works well...3MB/s vs 150 MB/s).

mkdir -p /sys/fs/cgroup/blkio/guest
echo 1 >/sys/fs/cgroup/blkio/guest/blkio.bfq.weight
docker run --rm --blkio-weight=10 --cgroup-parent=/guest ubuntu  dd bs=1M count=10024 if=/dev/zero of=/dump.dat oflag=direct status=progress