[Gluster-users] gluster local vs local = gluster x4 slower
Jeremy Enos
jenos at ncsa.uiuc.edu
Wed Mar 24 08:44:39 UTC 2010
I haven't tried all performance options disabled yet- I can try that
tomorrow when the resource frees up. I was actually asking first before
blindly trying different configuration matrices in case there's a clear
direction I should take with it. I'll let you know.
Jeremy
On 3/24/2010 2:54 AM, Stephan von Krawczynski wrote:
> Hi Jeremy,
>
> have you tried to reproduce with all performance options disabled? They are
> possibly no good idea on a local system.
> What local fs do you use?
>
>
> --
> Regards,
> Stephan
>
>
> On Tue, 23 Mar 2010 19:11:28 -0500
> Jeremy Enos<jenos at ncsa.uiuc.edu> wrote:
>
>
>> Stephan is correct- I primarily did this test to show a demonstrable
>> overhead example that I'm trying to eliminate. It's pronounced enough
>> that it can be seen on a single disk / single node configuration, which
>> is good in a way (so anyone can easily repro).
>>
>> My distributed/clustered solution would be ideal if it were fast enough
>> for small block i/o as well as large block- I was hoping that single
>> node systems would achieve that, hence the single node test. Because
>> the single node test performed poorly, I eventually reduced down to
>> single disk to see if it could still be seen, and it clearly can be.
>> Perhaps it's something in my configuration? I've pasted my config files
>> below.
>> thx-
>>
>> Jeremy
>>
>> ######################glusterfsd.vol######################
>> volume posix
>> type storage/posix
>> option directory /export
>> end-volume
>>
>> volume locks
>> type features/locks
>> subvolumes posix
>> end-volume
>>
>> volume disk
>> type performance/io-threads
>> option thread-count 4
>> subvolumes locks
>> end-volume
>>
>> volume server-ib
>> type protocol/server
>> option transport-type ib-verbs/server
>> option auth.addr.disk.allow *
>> subvolumes disk
>> end-volume
>>
>> volume server-tcp
>> type protocol/server
>> option transport-type tcp/server
>> option auth.addr.disk.allow *
>> subvolumes disk
>> end-volume
>>
>> ######################ghome.vol######################
>>
>> #-----------IB remotes------------------
>> volume ghome
>> type protocol/client
>> option transport-type ib-verbs/client
>> # option transport-type tcp/client
>> option remote-host acfs
>> option remote-subvolume raid
>> end-volume
>>
>> #------------Performance Options-------------------
>>
>> volume readahead
>> type performance/read-ahead
>> option page-count 4 # 2 is default option
>> option force-atime-update off # default is off
>> subvolumes ghome
>> end-volume
>>
>> volume writebehind
>> type performance/write-behind
>> option cache-size 1MB
>> subvolumes readahead
>> end-volume
>>
>> volume cache
>> type performance/io-cache
>> option cache-size 1GB
>> subvolumes writebehind
>> end-volume
>>
>> ######################END######################
>>
>>
>>
>> On 3/23/2010 6:02 AM, Stephan von Krawczynski wrote:
>>
>>> On Tue, 23 Mar 2010 02:59:35 -0600 (CST)
>>> "Tejas N. Bhise"<tejas at gluster.com> wrote:
>>>
>>>
>>>
>>>> Out of curiosity, if you want to do stuff only on one machine,
>>>> why do you want to use a distributed, multi node, clustered,
>>>> file system ?
>>>>
>>>>
>>> Because what he does is a very good way to show the overhead produced only by
>>> glusterfs and nothing else (i.e. no network involved).
>>> A pretty relevant test scenario I would say.
>>>
>>> --
>>> Regards,
>>> Stephan
>>>
>>>
>>>
>>>
>>>> Am I missing something here ?
>>>>
>>>> Regards,
>>>> Tejas.
>>>>
>>>> ----- Original Message -----
>>>> From: "Jeremy Enos"<jenos at ncsa.uiuc.edu>
>>>> To: gluster-users at gluster.org
>>>> Sent: Tuesday, March 23, 2010 2:07:06 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi
>>>> Subject: [Gluster-users] gluster local vs local = gluster x4 slower
>>>>
>>>> This test is pretty easy to replicate anywhere- only takes 1 disk, one
>>>> machine, one tarball. Untarring to local disk directly vs thru gluster
>>>> is about 4.5x faster. At first I thought this may be due to a slow host
>>>> (Opteron 2.4ghz). But it's not- same configuration, on a much faster
>>>> machine (dual 3.33ghz Xeon) yields the performance below.
>>>>
>>>> ####THIS TEST WAS TO A LOCAL DISK THRU GLUSTER####
>>>> [root at ac33 jenos]# time tar xzf
>>>> /scratch/jenos/intel/l_cproc_p_11.1.064_intel64.tgz
>>>>
>>>> real 0m41.290s
>>>> user 0m14.246s
>>>> sys 0m2.957s
>>>>
>>>> ####THIS TEST WAS TO A LOCAL DISK (BYPASS GLUSTER)####
>>>> [root at ac33 jenos]# cd /export/jenos/
>>>> [root at ac33 jenos]# time tar xzf
>>>> /scratch/jenos/intel/l_cproc_p_11.1.064_intel64.tgz
>>>>
>>>> real 0m8.983s
>>>> user 0m6.857s
>>>> sys 0m1.844s
>>>>
>>>> ####THESE ARE TEST FILE DETAILS####
>>>> [root at ac33 jenos]# tar tzvf
>>>> /scratch/jenos/intel/l_cproc_p_11.1.064_intel64.tgz |wc -l
>>>> 109
>>>> [root at ac33 jenos]# ls -l
>>>> /scratch/jenos/intel/l_cproc_p_11.1.064_intel64.tgz
>>>> -rw-r--r-- 1 jenos ac 804385203 2010-02-07 06:32
>>>> /scratch/jenos/intel/l_cproc_p_11.1.064_intel64.tgz
>>>> [root at ac33 jenos]#
>>>>
>>>> These are the relevant performance options I'm using in my .vol file:
>>>>
>>>> #------------Performance Options-------------------
>>>>
>>>> volume readahead
>>>> type performance/read-ahead
>>>> option page-count 4 # 2 is default option
>>>> option force-atime-update off # default is off
>>>> subvolumes ghome
>>>> end-volume
>>>>
>>>> volume writebehind
>>>> type performance/write-behind
>>>> option cache-size 1MB
>>>> subvolumes readahead
>>>> end-volume
>>>>
>>>> volume cache
>>>> type performance/io-cache
>>>> option cache-size 1GB
>>>> subvolumes writebehind
>>>> end-volume
>>>>
>>>> What can I do to improve gluster's performance?
>>>>
>>>> Jeremy
>>>>
>>>> _______________________________________________
>>>> Gluster-users mailing list
>>>> Gluster-users at gluster.org
>>>> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>>>> _______________________________________________
>>>> Gluster-users mailing list
>>>> Gluster-users at gluster.org
>>>> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>>>>
>>>>
>>>>
>>>
>>>
>>
>
>
More information about the Gluster-users
mailing list