[Gluster-users] GlusterFS saturates server disk IO due to write brick temporary file to ".glusterfs" directory

[Qing Wang]

Hi,

I have GridFTP + a network speedup solution in network + GlusterFS as a
file system component in a disk-to-disk data transferring scenario. For
glusterfs, I start with creating bricks inside /dev/sda1 filesystem. During
the file transfer (12 GB), it seems that glusterfs tries to write a
temporary file into the .glusterfs directory at a very high rate and
essentially the disk IO rate of the server host is ~100%. This leads the
file transfer performance unpredictable as file transfer(i.e., GridFTP)
would sometimes wait and send nothing due to the disk busy of glusterfs
server. As as test, I change the brick location to tmpfs (in memory) then
the disk I/O won't be saturated, this makes sense b/c the file writing
would happen in memory in this case, but this is not disk-to-disk transfer
anymore so this can't be an effective workaround in my case.

I wonder if there is any way I can turn off glusterfs to write temporary
files into the .gluster directory during the file transfer? Based on my
reading so far, I understand that this is a glusterfs design decision for
some recovering purpose, just in my case the writing rate is too fast and
saturates the disk IO and makes my disk-to-disk performance unstable.

My glusterfs version is: glusterfs 3.7.6 built on Dec 25 2015 20:50:46. I
am glad to post more volume or other setup details if necessary.

Thanks,
Qing
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20200529/b892a17a/attachment.html>