[Gluster-users] GlusterFS saturates server disk IO due to write brick temporary file to ".glusterfs" directory

[Strahil Nikolov]

Hi Qing,

I think that it is not  possible, yet  someone more experience can offer a  solution.

You can play with gluster  settings to reduce I/O load  by reducing the thread  count  tunables:

server.event-threads
client.event-threads
performance.io-thread-count

For details  check this  article (it's  for small files tuning,  but  describes  the options above): https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.1/html/administration_guide/small_file_performance_enhancements

Best  Regards,
Strahil Nikolov

На 29 май 2020 г. 22:25:29 GMT+03:00, Qing Wang <qw at g.clemson.edu> написа:
>Hi,
>
>I have GridFTP + a network speedup solution in network + GlusterFS as a
>file system component in a disk-to-disk data transferring scenario. For
>glusterfs, I start with creating bricks inside /dev/sda1 filesystem.
>During
>the file transfer (12 GB), it seems that glusterfs tries to write a
>temporary file into the .glusterfs directory at a very high rate and
>essentially the disk IO rate of the server host is ~100%. This leads
>the
>file transfer performance unpredictable as file transfer(i.e., GridFTP)
>would sometimes wait and send nothing due to the disk busy of glusterfs
>server. As as test, I change the brick location to tmpfs (in memory)
>then
>the disk I/O won't be saturated, this makes sense b/c the file writing
>would happen in memory in this case, but this is not disk-to-disk
>transfer
>anymore so this can't be an effective workaround in my case.
>
>I wonder if there is any way I can turn off glusterfs to write
>temporary
>files into the .gluster directory during the file transfer? Based on my
>reading so far, I understand that this is a glusterfs design decision
>for
>some recovering purpose, just in my case the writing rate is too fast
>and
>saturates the disk IO and makes my disk-to-disk performance unstable.
>
>My glusterfs version is: glusterfs 3.7.6 built on Dec 25 2015 20:50:46.
>I
>am glad to post more volume or other setup details if necessary.
>
>Thanks,
>Qing