[Gluster-devel] AFR between two bricks over 3000 miles

[nathan at robotics.net]

On Mon, 3 Mar 2008, Anand Avati wrote:

> the bottleneck seems to be create/open which is synchronous over all
> subvolumes. do you have numbers without involving the remote site/afr ?

Just mounted nyc rather then mirror and that cut the time to .076 sec vs 
local .01 sec.

This is a 100 meg link between NYC and SJC and only transferring 748K. My 
guess is even if we had a full gig link between the two we would have the 
same problem. A message sent from NYC to SJC will still take 60ms to fly 
across the country. So if you have a lot of messages flying back and forth 
waiting for data to be written you will have this problem.

Another clue is that it takes 27 sec just to rm -fr the files! If I moved 
afr from the client to the server would that help?

[root at xen0 glusterfs]# time cp -r /etc/sysconfig/ /share/mirror

real    0m0.076s
user    0m0.000s
sys     0m0.007s
[root at xen0 glusterfs]# time cp -r /etc/sysconfig/ /share/

real    0m0.010s
user    0m0.001s
sys     0m0.006s
[root at xen0 glusterfs]#

With mirror mounted (sjc and nyc up):

[root at xen0 glusterfs]# time cp -r /etc/sysconfig/ /share/mirror

real    0m52.195s
user    0m0.000s
sys     0m0.001s

[root at xen0 glusterfs]# time rm -fr /share/mirror/*

real    0m27.030s
user    0m0.000s
sys     0m0.002s

><>
Nathan Stratton
nathan at robotics.net
http://www.robotics.net