[Gluster-users] Extremely slow du
mohammad kashif
kashif.alig at gmail.com
Fri Jun 9 12:05:10 UTC 2017
Hi
I have just moved our 400 TB HPC storage from lustre to gluster. It is part
of a research institute and users have very small files to big files ( few
KB to 20GB) . Our setup consists of 5 servers, each with 96TB RAID 6 disks.
All servers are connected through 10G ethernet but not all clients.
Gluster volumes are distributed without any replication. There are
approximately 80 million files in file system.
I am mounting using glusterfs on clients.
I have copied everything from lustre to gluster but old file system exist
so I can compare.
The problem, I am facing is extremely slow du on even a small directory.
Also the time taken is substantially different each time.
I tried du from same client on a particular directory twice and got these
results.
time du -sh /data/aa/bb/cc
3.7G /data/aa/bb/cc
real 7m29.243s
user 0m1.448s
sys 0m7.067s
time du -sh /data/aa/bb/cc
3.7G /data/aa/bb/cc
real 16m43.735s
user 0m1.097s
sys 0m5.802s
16m and 7m is too long for a 3.7 G directory. I must mention that the
directory contains huge number of files (208736)
but running du on same directory on old data gives this result
time du -sh /olddata/aa/bb/cc
4.0G /olddata/aa/bb/cc
real 3m1.255s
user 0m0.755s
sys 0m38.099s
much better if I run same command again
time du -sh /olddata/aa/bb/cc
4.0G /olddata/aa/bb/cc
real 0m8.309s
user 0m0.313s
sys 0m7.755s
Is there anything I can do to improve this performance? I would also like
hear from some one who is running same kind of setup.
Thanks
Kashif
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170609/0f49b5c2/attachment.html>
More information about the Gluster-users
mailing list