[Gluster-users] Very slow rsync to gluster volume UNLESS `ls` or `find` scan dir on gluster volume first

[Artem Russakovskii]

An update, and a very interesting one!

After I started stracing rsync, all I could see was lstat calls, quite slow
ones, over and over, which is expected.

For example: lstat("uploads/2016/10/nexus2cee_DSC05339_thumb-161x107.jpg",
{st_mode=S_IFREG|0664, st_size=4043, ...}) = 0

I googled around and found
https://gist.github.com/nh2/1836415489e2132cf85ed3832105fcc1, which is seeing
this exact issue with gluster, rsync and xfs.

Here's the craziest finding so far. If while rsync is running (or right
before), I run /bin/ls or find on the same gluster dirs, it immediately
speeds up rsync by a factor of 100 or maybe even 1000. It's absolutely
insane.

I'm stracing the rsync run, and the slow lstat calls flood in at an
incredible speed as soon as ls or find run. Several hundred of files per
minute (excruciatingly slow) becomes thousands or even tens of thousands of
files a second.

What do you make of this?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180204/8929a2a9/attachment.html>