[Gluster-users] performance stops at 1Gb

[Gotwalt, P.]

Hi All,

I am doing some tests with gluster (3.1) and have a problem of not
getting higher throughput than 1 Gb (yes bit!) with 4 storage bricks.

My setup:

4 storage bricks (dualcore, 4GB mem) each with 3 sata 1Tb disks,
connected to a switch with 1 Gb nics.  In my tests I only use 1 SATA
disk as a volume, per brick.
1 client (2xquad core, 16 GB mem) with a 10Gb nic to the same switch as
the bricks.

When using striped of distributed configurations, with all 4 bricks
configured to act as a server, the performance will never be higher than
just below 1 Gb! I tested with 4, 8 and 16 parallel bonnie++ runs. 

The idea is that parallel bonnie's create enough files to get
distributed over the storage bricks. And all this bonnie's wil deliver
enough throughput to fill up this 10Gb line. I expect the throughput to
be maximum 4Gb because that's the maximum the 4 storage bricks together
can produce.

I also tested the throughput of the network with iperf3 and got:
- 5Gb to a second temporary client on another switch 200 Km from my
site, connected with a 5Gb fiber
- 908-920 Mb to the interfaces of the bricks. 
So the network seems ok.

Can someone advise me on why I don't get 4Gb? Or can someone advise me
on a better setup with the equipment I have? 


Peter Gotwalt