[Gluster-devel] [Gluster-users] glusterfs replica volume self heal lots of small file very very slow?how to improve? why slow?

[Jocelyn Hotte]

Hi Just Gluster,

Sorry for the bad formatting of the reply, I just copy/pasted the values from the daily digest

As for using GlusterFS with small files, it is also the use case in which we're trying to use it.
Unfortunately, we weren't successful yet in having a smooth ride with a Gluster cluster with a lot of small files.

Our use case is storing several million files, averaging under 1mb in size (most of them around 10kb). We still have some files at about 1gb, but very few.

When a self-heal hits in our use-case, it is a direct impact in performance for the users. The CPU of the Gluster nodes hits 100%, and maintains this for usually 1 hour, but sometimes goes up to 4-5 hours.
This usually renders the Gluster cluster unusable, with a high impact for us.

We're hoping the refactoring of the AFR in version 3.6 will help. We're also looking in testing some values in here: http://www.gluster.org/community/documentation/index.php/Documenting_the_undocumented

But as of yet, we haven't found anything that makes our use case work.
If you do find something though, please share it with the community, I would be thrilled to make it work :)

Jocelyn