[Gluster-users] GlusterFS performance
Michael Brown
michael at netdirect.ca
Thu May 23 13:54:24 UTC 2013
That's only a single data point (one LOOKUP call) - this tells you nothing.
You need to check after it's been running (and processing traffic) for a
while.
For example, here's the stats off one of my bricks (an SSD):
Brick: fearless2:/export/bricks/500117310007a84c/glusterdata
------------------------------------------------------------
Cumulative Stats:
Block Size: 32b+
64b+ 128b+
No. of Reads: 0
0 1
No. of Writes: 1
5634 4252
Block Size: 256b+ 512b+
2048b+
No. of Reads: 0
1 0
No. of Writes: 343
24 1
Block Size: 4096b+ 8192b+
16384b+
No. of Reads: 8
7 10
No. of Writes: 4
0 0
Block Size: 32768b+ 65536b+
131072b+
No. of Reads: 25
165 436
No. of Writes: 2
7 36
%-latency Avg-latency Min-Latency Max-Latency No. of
calls Fop
--------- ----------- ----------- -----------
------------ ----
0.00 0.00 us 0.00 us 0.00 us 120
FORGET
0.00 0.00 us 0.00 us 0.00 us 2940
RELEASE
0.00 0.00 us 0.00 us 0.00 us 4554
RELEASEDIR
0.00 104.00 us 104.00 us 104.00 us 1
TRUNCATE
0.00 125.00 us 110.00 us 140.00 us 2
XATTROP
0.01 12.62 us 8.00 us 25.00 us 146
ACCESS
0.04 81.77 us 65.00 us 112.00 us 60
SETXATTR
0.05 30.99 us 25.00 us 57.00 us 212
SETATTR
0.06 12.73 us 8.00 us 59.00 us 574
INODELK
0.09 188.43 us 140.00 us 244.00 us 60
CREATE
0.11 25.24 us 15.00 us 149.00 us 533
STATFS
0.12 260.72 us 206.00 us 430.00 us
60 MKDIR
0.26 11.34 us 5.00 us 127.00 us
2925 FLUSH
0.27 15.14 us 7.00 us 90.00 us 2274
ENTRYLK
0.36 102.52 us 81.00 us 161.00 us
442 RMDIR
0.63 27.61 us 17.00 us 606.00 us
2880 OPEN
0.76 171.58 us 91.00 us 5691.00 us 555
UNLINK
0.87 22.66 us 8.00 us 469.00 us 4812
READDIR
0.87 24.37 us 10.00 us 1302.00 us
4506 STAT
0.94 61.67 us 16.00 us 194.00 us 1917
GETXATTR
1.06 51.20 us 10.00 us 224.00 us
2600 FSTAT
1.14 31.46 us 18.00 us 1016.00 us 4554
OPENDIR
2.56 31.19 us 18.00 us 4373.00 us
10304 WRITE
2.58 417.28 us 15.00 us 1860.00 us
776 READ
3.64 17.26 us 6.00 us 4824.00 us 26507
FINODELK
24.03 146.42 us 49.00 us 9854.00 us 20622
FXATTROP
26.67 652.70 us 42.00 us 89705.00 us 5134
READDIRP
32.86 128.19 us 9.00 us 4617.00 us 32204
LOOKUP
On 13-05-23 09:03 AM, Yavor Marinov wrote:
> I've just enabled profiling of the volume and this is the information
> from the profile info printed:
>
> [root at gfs1 ~]# gluster volume profile test info
> Brick: 93.123.32.41:/data
> -------------------------
> Cumulative Stats:
> %-latency Avg-latency Min-Latency Max-Latency No. of
> calls Fop
> --------- ----------- ----------- -----------
> ------------ ----
> 100.00 148.00 us 148.00 us 148.00 us
> 1 LOOKUP
>
> Duration: 13950 seconds
> Data Read: 0 bytes
> Data Written: 0 bytes
>
> Interval 4 Stats:
>
> Duration: 7910 seconds
> Data Read: 0 bytes
> Data Written: 0 bytes
>
> [root at gfs1 ~]#
>
> Anything here that might be useful ?
>
>
> ---
> Find out about our new Cloud service - Cloudware.bg
> <http://cloudware.bg/?utm_source=email&utm_medium=signature&utm_content=link&utm_campaign=newwebsite>
> Access anywhere. Manage it yourself. Pay as you go.
> ------------------------------------------------------------------------
> *Yavor Marinov*
> System Administrator
>
> Neterra Ltd.
> Telephone: +359 2 975 16 16
> Fax: +359 2 975 34 36
> Mobile: +359 888 610 048
> www.neterra.net <http://www.neterra.net>
>
>
> On 05/23/2013 01:10 PM, ???? ??????? wrote:
>> I've made a mistake we are using 30Mbit connectivity on all of the
>> nodes. Below is a iperf test between the node and the client
>>
>> [root at gfs4 ~]# iperf -c 93.123.32.41
>> ------------------------------------------------------------
>> Client connecting to 93.123.32.41, TCP port 5001
>> TCP window size: 23.2 KByte (default)
>> ------------------------------------------------------------
>> [ 3] local 93.123.32.44 port 49838 connected with 93.123.32.41 port 5001
>> [ ID] Interval Transfer Bandwidth
>> [ 3] 0.0-10.1 sec 49.9 MBytes 41.5 Mbits/sec
>> [root at gfs4 ~]#
>>
>> But when trying to copy a 1Gb file on the client's mounted volume the
>> speed between the client and the node is ~500kb/s
>>
>>
>> ---
>> Find out about our new Cloud service - Cloudware.bg
>> <http://cloudware.bg/?utm_source=email&utm_medium=signature&utm_content=link&utm_campaign=newwebsite>
>> Access anywhere. Manage it yourself. Pay as you go.
>> ------------------------------------------------------------------------
>> *Yavor Marinov*
>> System Administrator
>>
>> Neterra Ltd.
>> Telephone: +359 2 975 16 16
>> Fax: +359 2 975 34 36
>> Mobile: +359 888 610 048
>> www.neterra.net <http://www.neterra.net>
>>
>>
>> On 05/23/2013 12:16 PM, Nux! wrote:
>>> On 23.05.2013 09:41, ???? ??????? wrote:
>>>> Thanks for your reply.
>>>>
>>>> No matter how many nodes (currently the volume is only with its own
>>>> node) the speed is really slow. For testing purposes, i made a volume
>>>> with only one node, without any replication - however the speed is
>>>> still ~500kb/s. The cloud servers are limited to 30Gbit/s but still
>>>> the traffic when writing to the node is ~500kb/s
>>>>
>>>> i'm using 3.3.1 glusterfsd with kernel 2.6.18-348.el5xen and i need
>>>> to know if the the problem is within the kernel.
>>>
>>> I don't think it is a problem with gluster; I never used el5 for
>>> this, but I doubt there's an inherent problem with it either. That
>>> speed limit looks odd to me and I think it's somewhere in your setup.
>>> Have you done any actual speed tests in the VMs?
>>>
>>
>>
>>
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> http://supercolony.gluster.org/mailman/listinfo/gluster-users
>
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-users
--
Michael Brown | `One of the main causes of the fall of
Systems Consultant | the Roman Empire was that, lacking zero,
Net Direct Inc. | they had no way to indicate successful
?: +1 519 883 1172 x5106 | termination of their C programs.' - Firth
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20130523/f456b355/attachment.html>
More information about the Gluster-users
mailing list