[Gluster-devel] Re: Uh, another gotcha with AFR, pre-existing data specific

[Joey Novak]

I think Gluster already does what you want.  The procedure you are
following, is if you want Gluster to copy the data to the other server, but
preserve the data on Server1.  If you don't want it to copy to the other
server, then you will need to atleast make sure BOTH servers data have the
xattr attributes set to the same thing.  Otherwise, gluster will think one
copy is newer then the other, and copy the data with the xattr's to the
other server.

  JOey

On Tue, May 6, 2008 at 8:40 AM, Christopher Hawkins <
chawkins at veracitynetworks.com> wrote:

> I have a need for something quite similar to that. Seems reasonable to
> me...
>
>
> My guess is that what you want to do is:
>
> Set xattr version = 2 for all gluster files on server 1. Start gluster
> with
> AFR. Do a find on all files on server 1, which should copy them to server
> 2,
> whether they already exist there or not (but really server2 should pretty
> much be empty).
>
> Now all files should be in sync. I will be doing some testing on this soon
> and intend to write a script to handle it... The Md5 check is a good idea
> and I'll try to find a way to build that in. Let me know how this goes for
> you... Your feedback will be very helpful!
>
> PS - You say "it did no good" but I disagree. In your setup below, even
> though it copied the data all over again to server 2, it did so while
> server
> 1 was online because you manually set the xattrs on the existing data on
> server 1. You had much less downtime than if you had re-copied all the
> data
> from server 1 TO server 1, from non-gluster storage into gluster storage.
>
> Chris
>
> > -----Original Message-----
> > From:
> > gluster-devel-bounces+chawkins=veracitynetworks.com at nongnu.org
> >
> > [mailto:gluster-devel-bounces+chawkins<gluster-devel-bounces%2Bchawkins>
> =veracitynetworks.com at no
> ngnu.org] On Behalf Of Brandon Lamb
> > Sent: Monday, May 05, 2008 9:07 PM
> > To: Gluster Devel
> > Subject: [Gluster-devel] Re: Uh, another gotcha with
> > AFR,pre-existing data specific
> >
> > On Mon, May 5, 2008 at 5:58 PM, Brandon Lamb
> > <brandonlamb at gmail.com> wrote:
> > > I just did some testing, and came to the conclusion that trying to
> > > setup afr using one server with pre-existing data and a
> > blank server,
> > > and copying your data and removing xattr's on the copied data then
> > > initiating afr DOES NO GOOD.
> > >
> > > server1 - 400 megs of data in 10 tarballs, removed all xattr
> > > server2 - copied the files from server1
> > > server1 - started glusterfsd, then ran setfattr
> > > trusted.glusterfs.version to 1, files on server2 have no xattr.
> > >
> > > At this point i should have identical copies of data
> > *assuming* i had
> > > no writes in between.
> > >
> > > SO, now in a client i do head -c 1 file0.tar.bz2 and it seems that
> > > since files on server2 have NO xattr, it copies them all
> > over again!!!
> > >
> > > So, is there no viable way to PRECOPY a copy of pre-existing data?
> > > Looks like what we will have to do is a directory by directory
> > > migration or stop all services that rely on the data store,
> > copy the
> > > data to both machines while there are no writes (no
> > changes) going on,
> > > then start everhting back up.
> > >
> > > For those of us that need this in a mail storage scenario,
> > this is not
> > > good. I cant stop my entire mail system for 4 hours while I
> > copy over
> > > 170 gigs of 4 million files.
> > >
> > > Now I will have to think of something a little more tricky
> > like moving
> > > a single maildir subdirectory letter at a time.
> > >
> > > Thoughts, comments, suggestions?
> >
> > I am stepping way over my head now, but here goes...
> >
> > Is there any way either with a translator or I dont know
> > what, but to implement some kind of algorithm (md5 or
> > whatnot) against the file in this situation? Something that
> > could/would only be used when initially setting up a cluster?
> > I dont knwo if that would require just a seperate script
> > written in whatever language or if it would be something that
> > could belong to glusterfs or what.
> >
> > In the case i just described, it would be nice to have
> > something to go through all files such as the find trick, and
> > have both servers do an
> > md5 check or whatever and if they arethe same update the
> > version on the COPY to the same version?
> >
> > Is this TOTALLY broken/whackass to do? I know pretty much
> > nothing about file system schematics and such so please dont
> > beat me up too badly.
> >
> > =P
> >
> >
> > _______________________________________________
> > Gluster-devel mailing list
> > Gluster-devel at nongnu.org
> > http://lists.nongnu.org/mailman/listinfo/gluster-devel
> >
>
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> http://lists.nongnu.org/mailman/listinfo/gluster-devel
>



-- 
C) 540-460-9848
W) 757-233-0834