Skip to content

Small Object Storage Very Slow,does there has any way to improve the performance? #12762

Closed
@gaoyangcaiji

Description

@gaoyangcaiji

My minio version is:Version: 2021-06-17T00:10:46Z
My minio cluster have 4 drivers
My driver used zfs, 30 device makes one dirver.such as:

here are my zfs speed:
image

here are my uploading a large object to minio server's speed:
223M/s
here are my uploading many small objects to minio server's speed:
4M/s
here are my "rclone copy minio:/testspeed/data/ /tmp -P" to minio server's speed:
50M/s

I have read the minio blog:
https://blog.min.io/minio-optimizes-small-objects/
https://blog.min.io/small_objects/

I used "mc cp .tar play/mybucket --disable-multipart --attr "X-Amz-Meta-Snowball-Auto-Extract=true"",the speed is:
189M/s

but i have to upload small files without ".tar". " tar" the small files spend a lot of time.So, is there any way to impove the performance to upload a lot of small files?Dose there have any "attr" to do this?Such as "X-Amz-Meta-Snowball-Auto-Extract=true".
I know you have improve the performance of upload small files in thre relese 2021.04.22,but it seems not efficient,my speed still "4M/s".so hope your helps!

Activity

harshavardhana

harshavardhana commented on Jul 21, 2021

@harshavardhana
Member

Do not use ZFS, use XFS and provide single drives directly to MinIO instead of providing single drive.

locked as resolved and limited conversation to collaborators on Jul 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @harshavardhana@gaoyangcaiji

        Issue actions

          Small Object Storage Very Slow,does there has any way to improve the performance? · Issue #12762 · minio/minio