[Gluster-users] Performance

[Mohit Anchlia]

I am getting miserable performance in LAN setup with 6 servers with
distributed and 2 replicas with 1GigE. I am using native glsuter
clients (mount -t glusterfs server1:/vol /mnt) I am only able to get
20Mb/s. This is some of the output from sar:

Each server has 4 10K SAS drives RAID0. This is a new setup and I
expected to get much much higher performance. Can someone please help
with recommendations?

!sar
sar -B 1 100


05:44:38 PM  pgpgin/s pgpgout/s   fault/s  majflt/s
05:44:39 PM      0.00      0.00   1413.00      0.00
05:44:40 PM      0.00  29896.00     29.00      0.00
05:44:41 PM      0.00   4510.89   1523.76      0.00
05:44:42 PM      0.00     16.16     20.20      0.00
05:44:43 PM      0.00     12.00     16.00      0.00
05:44:44 PM      0.00    102.97     15.84      0.00
05:44:45 PM      0.00  21100.00     14.00      0.00
05:44:46 PM      0.00   8092.00     19.00      0.00

sar -ru 1 100

05:46:48 PM kbmemfree kbmemused  %memused kbbuffers  kbcached
kbswpfree kbswpused  %swpused  kbswpcad
05:46:49 PM    250696  98742512     99.75   6327752  87155568
101056220       284      0.00         0

05:46:49 PM       CPU     %user     %nice   %system   %iowait
%steal     %idle
05:46:50 PM       all      1.75      0.00     16.62      1.87
0.00     79.76

05:46:49 PM kbmemfree kbmemused  %memused kbbuffers  kbcached
kbswpfree kbswpused  %swpused  kbswpcad
05:46:50 PM    257364  98735844     99.74   6328180  87149728
101056220       284      0.00         0

05:46:50 PM       CPU     %user     %nice   %system   %iowait
%steal     %idle
05:46:51 PM       all      0.00      0.00      0.54      4.12
0.00     95.34

05:46:50 PM kbmemfree kbmemused  %memused kbbuffers  kbcached
kbswpfree kbswpused  %swpused  kbswpcad
05:46:51 PM    257388  98735820     99.74   6328184  87150080
101056220       284      0.00         0

05:46:51 PM       CPU     %user     %nice   %system   %iowait
%steal     %idle
05:46:52 PM       all      0.12      0.00      3.08      0.08
0.00     96.72

05:46:51 PM kbmemfree kbmemused  %memused kbbuffers  kbcached
kbswpfree kbswpused  %swpused  kbswpcad
05:46:52 PM    257436  98735772     99.74   6328284  87151456
101056220       284      0.00         0

05:46:52 PM       CPU     %user     %nice   %system   %iowait
%steal     %idle
05:46:53 PM       all      0.17      0.00      1.62      0.04
0.00     98.17

05:46:52 PM kbmemfree kbmemused  %memused kbbuffers  kbcached
kbswpfree kbswpused  %swpused  kbswpcad
05:46:53 PM    253468  98739740     99.74   6328368  87155004
101056220       284      0.00         0

05:46:53 PM       CPU     %user     %nice   %system   %iowait
%steal     %idle
05:46:54 PM       all      0.17      0.00      3.08      0.00
0.00     96.76

05:46:53 PM kbmemfree kbmemused  %memused kbbuffers  kbcached
kbswpfree kbswpused  %swpused  kbswpcad
05:46:54 PM    252320  98740888     99.75   6328444  87156888
101056220       284      0.00         0

05:46:54 PM       CPU     %user     %nice   %system   %iowait
%steal     %idle
05:46:55 PM       all      0.00      0.00      0.00      0.79
0.00     99.21

05:46:54 PM kbmemfree kbmemused  %memused kbbuffers  kbcached
kbswpfree kbswpused  %swpused  kbswpcad
05:46:55 PM    252240  98740968     99.75   6328448  87156884
101056220       284      0.00         0

05:46:55 PM       CPU     %user     %nice   %system   %iowait
%steal     %idle
05:46:56 PM       all      0.83      0.00     23.89      0.00
0.00     75.28

05:46:55 PM kbmemfree kbmemused  %memused kbbuffers  kbcached
kbswpfree kbswpused  %swpused  kbswpcad
05:46:56 PM    247652  98745556     99.75   6328712  87159900
101056220       284      0.00         0


---