[Gluster-devel] AFR between two bricks over 3000 miles

[nathan at robotics.net]

Any update on this? Would love to be able to use gluster between east and 
west coast sites.


><>
Nathan Stratton
nathan at robotics.net
http://www.robotics.net

On Tue, 4 Mar 2008, Anand Avati wrote:

> 2008/3/4, nathan at robotics.net <nathan at robotics.net>:
>>
>>
>> I tried moving this to server side and that did not help. The 74 ms delay
>> between the sites appears to make gluster unusable, even tho there is 100
>> meg of free capacity. Any plans to background some of the back and forth
>> communication so that if you use write-behind you can get close to local
>> speed on writes?
>
>
> currently write-behind works for only write() calls. I too am seeing the
> issue of the highest latency subvolume becoming the bottleneck for
> operations while it is not unfair to expect application performance at the
> fastest subvolume speed. Will discuss with Krishna about the impact and side
> effects and let you know what we come up with.
>
> avati
>
>
>
>> <>
>> Nathan Stratton
>> nathan at robotics.net
>> http://www.robotics.net
>>
>>
>> On Mon, 3 Mar 2008, nathan at robotics.net wrote:
>>
>>> On Mon, 3 Mar 2008, Anand Avati wrote:
>>>
>>>> the bottleneck seems to be create/open which is synchronous over all
>>>> subvolumes. do you have numbers without involving the remote site/afr ?
>>>
>>> Just mounted nyc rather then mirror and that cut the time to .076 sec vs
>>> local .01 sec.
>>>
>>> This is a 100 meg link between NYC and SJC and only transferring 748K.
>> My
>>> guess is even if we had a full gig link between the two we would have
>> the
>>> same problem. A message sent from NYC to SJC will still take 60ms to fly
>>> across the country. So if you have a lot of messages flying back and
>> forth
>>> waiting for data to be written you will have this problem.
>>>
>>> Another clue is that it takes 27 sec just to rm -fr the files! If I
>> moved afr
>>> from the client to the server would that help?
>>>
>>> [root at xen0 glusterfs]# time cp -r /etc/sysconfig/ /share/mirror
>>>
>>> real    0m0.076s
>>> user    0m0.000s
>>> sys     0m0.007s
>>> [root at xen0 glusterfs]# time cp -r /etc/sysconfig/ /share/
>>>
>>> real    0m0.010s
>>> user    0m0.001s
>>> sys     0m0.006s
>>> [root at xen0 glusterfs]#
>>>
>>> With mirror mounted (sjc and nyc up):
>>>
>>> [root at xen0 glusterfs]# time cp -r /etc/sysconfig/ /share/mirror
>>>
>>> real    0m52.195s
>>> user    0m0.000s
>>> sys     0m0.001s
>>>
>>> [root at xen0 glusterfs]# time rm -fr /share/mirror/*
>>>
>>> real    0m27.030s
>>> user    0m0.000s
>>> sys     0m0.002s
>>>
>>>> <>
>>> Nathan Stratton
>>> nathan at robotics.net
>>> http://www.robotics.net
>>>
>>
>
>
>
> -- 
> If I traveled to the end of the rainbow
> As Dame Fortune did intend,
> Murphy would be there to tell me
> The pot's at the other end.
>