[Gluster-users] Gluster-users Digest, Vol 51, Issue 46

[Ben England]

4. Re: kernel parameters for improving gluster writes on millions of small writes (long) (Harry Mangalam)

Harry, You are correct, Glusterfs throughput with small write transfer sizes is a client-side problem, here are workarounds that at least some applications could use. 

1) NFS client is one workaround, since it buffers writes using the kernel buffer cache.

2) If your app does not have a configurable I/O size, but it lets you write to stdout, you can try piping your output to stdout and letting dd aggregate your I/O to the filesystem for you.  In this example we triple single-thread write throughput for 4-KB I/O requests in this example.

[root at perf56 ~]# mount -t glusterfs perf66-10ge:/repl2 /mnt/repl2

[root at perf56 ~]# dd if=/dev/zero of=/mnt/repl2/a.dd bs=4k count=1024k
1048576+0 records in
1048576+0 records out
4294967296 bytes (4.3 GB) copied, 55.8191 s, 76.9 MB/s

[root at perf56 ~]# dd if=/dev/zero bs=4k count=1024k | dd of=/mnt/repl2/a.dd bs=1024k
1048576+0 records in
1048576+0 records out
4294967296 bytes (4.3 GB) copied, 20.6882 s, 208 MB/s
0+58116 records in
0+58116 records out
4294967296 bytes (4.3 GB) copied, 19.9023 s, 216 MB/s

3) If your program is written in "C" and it uses stdio.h, you can probably do setvbuf() "C" RTL call to increase buffer size to something greater than 8 KB, which is the default in gcc-4.4.

http://en.cppreference.com/w/c/io/setvbuf