[Gluster-devel] Re: Uh, another gotcha with AFR, pre-existing data specific

[Brandon Lamb]

On Mon, May 5, 2008 at 5:58 PM, Brandon Lamb <brandonlamb at gmail.com> wrote:
> I just did some testing, and came to the conclusion that trying to
> setup afr using one server with pre-existing data and a blank server,
> and copying your data and removing xattr's on the copied data then
> initiating afr DOES NO GOOD.
>
> server1 - 400 megs of data in 10 tarballs, removed all xattr
> server2 - copied the files from server1
> server1 - started glusterfsd, then ran setfattr
> trusted.glusterfs.version to 1, files on server2 have no xattr.
>
> At this point i should have identical copies of data *assuming* i had
> no writes in between.
>
> SO, now in a client i do head -c 1 file0.tar.bz2 and it seems that
> since files on server2 have NO xattr, it copies them all over again!!!
>
> So, is there no viable way to PRECOPY a copy of pre-existing data?
> Looks like what we will have to do is a directory by directory
> migration or stop all services that rely on the data store, copy the
> data to both machines while there are no writes (no changes) going on,
> then start everhting back up.
>
> For those of us that need this in a mail storage scenario, this is not
> good. I cant stop my entire mail system for 4 hours while I copy over
> 170 gigs of 4 million files.
>
> Now I will have to think of something a little more tricky like moving
> a single maildir subdirectory letter at a time.
>
> Thoughts, comments, suggestions?

I am stepping way over my head now, but here goes...

Is there any way either with a translator or I dont know what, but to
implement some kind of algorithm (md5 or whatnot) against the file in
this situation? Something that could/would only be used when initially
setting up a cluster? I dont knwo if that would require just a
seperate script written in whatever language or if it would be
something that could belong to glusterfs or what.

In the case i just described, it would be nice to have something to go
through all files such as the find trick, and have both servers do an
md5 check or whatever and if they arethe same update the version on
the COPY to the same version?

Is this TOTALLY broken/whackass to do? I know pretty much nothing
about file system schematics and such so please dont beat me up too
badly.

=P