[Gluster-devel] AFR between two bricks over 3000 miles

[nathan at robotics.net]

I tried moving this to server side and that did not help. The 74 ms delay 
between the sites appears to make gluster unusable, even tho there is 100 
meg of free capacity. Any plans to background some of the back and forth 
communication so that if you use write-behind you can get close to local 
speed on writes?

><>
Nathan Stratton
nathan at robotics.net
http://www.robotics.net

On Mon, 3 Mar 2008, nathan at robotics.net wrote:

> On Mon, 3 Mar 2008, Anand Avati wrote:
>
>> the bottleneck seems to be create/open which is synchronous over all
>> subvolumes. do you have numbers without involving the remote site/afr ?
>
> Just mounted nyc rather then mirror and that cut the time to .076 sec vs 
> local .01 sec.
>
> This is a 100 meg link between NYC and SJC and only transferring 748K. My 
> guess is even if we had a full gig link between the two we would have the 
> same problem. A message sent from NYC to SJC will still take 60ms to fly 
> across the country. So if you have a lot of messages flying back and forth 
> waiting for data to be written you will have this problem.
>
> Another clue is that it takes 27 sec just to rm -fr the files! If I moved afr 
> from the client to the server would that help?
>
> [root at xen0 glusterfs]# time cp -r /etc/sysconfig/ /share/mirror
>
> real    0m0.076s
> user    0m0.000s
> sys     0m0.007s
> [root at xen0 glusterfs]# time cp -r /etc/sysconfig/ /share/
>
> real    0m0.010s
> user    0m0.001s
> sys     0m0.006s
> [root at xen0 glusterfs]#
>
> With mirror mounted (sjc and nyc up):
>
> [root at xen0 glusterfs]# time cp -r /etc/sysconfig/ /share/mirror
>
> real    0m52.195s
> user    0m0.000s
> sys     0m0.001s
>
> [root at xen0 glusterfs]# time rm -fr /share/mirror/*
>
> real    0m27.030s
> user    0m0.000s
> sys     0m0.002s
>
>> <>
> Nathan Stratton
> nathan at robotics.net
> http://www.robotics.net
>