[Gluster-users] Poor Gluster performance

[Lars Hanke]

Am 18.02.2015 um 22:05 schrieb Ben Turner:
> ----- Original Message -----
>> From: "Lars Hanke" <debian at lhanke.de>
>> To: gluster-users at gluster.org
>> Sent: Wednesday, February 18, 2015 3:01:54 PM
>> Subject: [Gluster-users] Poor Gluster performance
>>
>> I set up a distributed, replicated volume consisting of just 2 bricks on
>> two physical nodes. The nodes are peered using a dedicated GB ethernet
>> and can be accessed from the clients using a separate GB ethernet NIC.
>>
>> Doing a simple dd performance test I see about 11 MB/s for read and
>> write. Running a local setup, i.e. both bricks on the same machine and
>> local mount, I saw even 500 MB/s. So network sould be the limiting
>> factor. But using NFS or CIFS on the same network I see 110 MB/s.
>>
>> Is gluster 10 times slower than NFS?
>
> Something is going on there.  On my gigabit setups I see 100-120 MB / sec writes for pure distribute and about 45-55 MB / sec with replica 2.  What block size are you using?  I could see that if you were writing something like 4k or under but 64k and up you should be getting about what I said.  Can you tell me more about your test?

Block size is 50M:

root at gladsheim:/# mount -t glusterfs node2:/test ~/mnt
root at gladsheim:/# dd if=/dev/zero of=~/mnt/testfile.null bs=50M count=10
10+0 records in
10+0 records out
524288000 bytes (524 MB) copied, 46.6079 s, 11.2 MB/s
root at gladsheim:/# dd if=~/mnt/testfile.null of=/dev/null bs=50M count=10
10+0 records in
10+0 records out
524288000 bytes (524 MB) copied, 45.7487 s, 11.5 MB/s

It doesn't depend on whether I use node1 or node2 for the mount.

BTW: does the cut of the bandwidth to half in replicated mode mean that 
the client writes to both nodes, i.e. doubles the network load on the 
client side network? I hoped that replication would be run on the server 
side network.

Regards,
  - lars.