[Gluster-users] A faster way to to replicate?

[Liam Slusser]

All -

Recently one of bricks lost a hard drive, during the rebuild (3ware
9690 controller) we lost another drive and then had a few ECC errors
on a third.  This was on a ~30tb 24 drive RAID6 array.  I was able to
force the controller to rebuild with a ignore_ECC flag which has
completed successfully and the XFS partition appears to be fine.  In
the 3ware device logs I see a dozen alerts about bad sectors/ecc
errors.  The partition is at 97% full so its a pretty good chance we
have some data corruption.

But not to worry, Gluster to the rescue right??  We currently have two
copies of our data and gluster handles the replication between them -
let's call them "A" and "B" clusters. Our "A" cluster is the cluster
having issues.  We've been planning on adding a third "C" cluster for
extra reliability and mostly for the added performance.  So since I
have a working good copy of the brick that is having issues on our "B"
cluster I started a gluster sync of our "B" cluster to the new "C"
cluster.

And OMG its so slow.  I've been running a "ls -alR" for the last week
and its only done 3.8% (replicated ~9.9 million files) of our total
space with an estimate finish date of another 223 days - thats the end
of August!  So my question is how can I get this done quicker?  Can I
rsync one brick to another brick directly - I know that will not copy
the extended attributes correctly and I believe will mess up gluster
right?

Anybody have some great ideas?

I'm running gluster 2.0.9 with 64bit Centos 5.7/6.2.  Each A/B/C
cluster is 4 x 30tb xfs raid6 bricks for a total of ~120tb (84tb in
use).

liam