[Gluster-users] Understanding gluster performance

Gionatan Danti g.danti at assyoma.it
Tue Jan 21 17:16:56 UTC 2020

Il 21-01-2020 11:40 Yaniv Kaul ha scritto:
> How did you fix this?
> How did you spot this?

I used iperf3 between the two hosts. It shows that, albeit bandwidth was 
near the 1 Gbps limit, there were frequent retransmissions. "netstat -s 
| grep retran" confirmed that retransmissions happened during my gluster 

> fsync requires each write to land on disk and get an ack on it - it's
> probably the slowest kind of write you can imagine, and you seem be
> doing it for every small (4K?) block.
> This is not very realistic. But in your case you say you are writing
> to /dev/shm, so that is strange. Can you try with a different fio
> engine and see what you get?

True, but putting my briks on /dev/shm means they are actually are 
in-memory, with no disks/seeks slowing down the syncs. I also tried with 
a very simple "dd if=/dev/urandom of=test.img bs=4k count=1024" and the 
results were idential (about 250 IOPs).

To exclude network latency, I did a local, two-brick /dev/shm backed 
volume (both bricks were on the same machine). Locally mounting the 
gluser volume via fuse, I only get about 500-600 IOPs.

> What are you trying to test here?

Database and virtual machine performance (which are both fsync and 4k 

> Good question. Perhaps profiling would help here. Perhaps too many
> threads are contending for CPU? Some lock contention?

What kind or profiling should I do? strace on glusterd process? perf 


Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti at assyoma.it - info at assyoma.it
GPG public key ID: FF5F32A8

More information about the Gluster-users mailing list