[Gluster-users] Performance

Wed Apr 20 02:43:32 UTC 2011

It's HW RAID we are using. These are Dell C6100 server with RAID
controller. We expect around 100MB/sec. When I run "dd" I get 20MB/sec
and since I have 6 servers I at least expected 3 X 20 MB/sec since it
replica of 2. We are in a subnet inside a LAB with only us doing the
testing so network is not a issue for sure.

File distribution are as follows:

bytes %
130000 19.762%
70000 30.101%
100000 20.165%
230000 20.016%
1100000 0.447%
430000 5%
2000000 039%
630000 4.47%

On Tue, Apr 19, 2011 at 7:07 PM, Joe Landman
<landman at scalableinformatics.com> wrote:
> On 04/19/2011 08:52 PM, Mohit Anchlia wrote:
>>
>> I am getting miserable performance in LAN setup with 6 servers with
>> distributed and 2 replicas with 1GigE. I am using native glsuter
>> clients (mount -t glusterfs server1:/vol /mnt) I am only able to get
>> 20Mb/s. This is some of the output from sar:
>>
>> Each server has 4 10K SAS drives RAID0. This is a new setup and I
>> expected to get much much higher performance. Can someone please help
>> with recommendations?
>
> It seems like I just handled a case like this a few months ago ...
>
> What does your IO workload actually look like?  Much more interested in
> iostat like output (though dstat also works very well for bandwidth heavy
> loads).
>
> Gluster isn't going to do well with small IO operations without serious
> caching (NFS client).  Despite the fact that these are RAID0 across 4x
> 10kRPM SAS drives, this is *not* a high performance IO system in most senses
> of the definition.  The design of the storage should be driven by the
> application and anticipated workloads.
>
> More to the point, what specifically are your goals in terms of
> throughput/bandwidth ... what will your storage loads look like?
>
> What RAID cards are you using (if any)?  If software raid, could you report
> output of
>
>        mdadm --detail /dev/MD
>
> where /dev/MD is your MD raid device.  Which SAS 10k disks are you using?
>  How are they connected to the machine if not through a RAID card?  Is the
> RAID0 a hardware or software RAID?
>
>> !sar
>> sar -B 1 100
>>
>>
>> 05:44:38 PM  pgpgin/s pgpgout/s   fault/s  majflt/s
>> 05:44:39 PM      0.00      0.00   1413.00      0.00
>> 05:44:40 PM      0.00  29896.00     29.00      0.00
>> 05:44:41 PM      0.00   4510.89   1523.76      0.00
>> 05:44:42 PM      0.00     16.16     20.20      0.00
>> 05:44:43 PM      0.00     12.00     16.00      0.00
>> 05:44:44 PM      0.00    102.97     15.84      0.00
>> 05:44:45 PM      0.00  21100.00     14.00      0.00
>> 05:44:46 PM      0.00   8092.00     19.00      0.00
>
> sar isn't too useful for figuring out whats going on in the io channel.
> iostat is much better.  dstat, atop, vmstat are all specifically good tools.
>  If you want too much data (e.g. it gathers everything of value), use
> collectl, with a 1 second interval, and the right options.
>
>
> --
> Joseph Landman, Ph.D
> Founder and CEO
> Scalable Informatics, Inc.
> email: landman at scalableinformatics.com
> web  : http://scalableinformatics.com
>       http://scalableinformatics.com/sicluster
> phone: +1 734 786 8423 x121
> fax  : +1 866 888 3112
> cell : +1 734 612 4615
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>