[Gluster-devel] Replication Initialization from Existing Directory

[gordan at bobich.net]

On Thu, 24 Apr 2008, Krishna Srinivas wrote:

>>  I'm trying to move a large volume of data from local disk to GlusterFS. I
>> could just copy it, but copying ~ 1TB of data is slow. So, what I've tried
>> to do (with some randomly generated data for a test case) is to specify the
>> directory already containing the data as the data source for the underlying
>> storage brick.
>>
>>  I then fire up glusterfsd and glusterfs on the same machine, and I can see
>> all the data via the mountpoint.
>>
>>  On another node, I start glusterfsd and glusterfs, and I can see and read
>> the data. But, the data doesn't appear on the underlying data brick on the
>> 2nd node after I have done cat * > /dev/null in the mounted directory.
>>
>>  So it looks like GluserFS isn't causing the data to get copied on reads in
>> this scenario.
>>
>>  Can anyone hazard a guess as to why this might be? I am guessing that it's
>> to do with the fact that the xattrs/metaddata have not been initialized by
>> glusterfs because the files were added "underneath" rather than via the
>> mountpoint. Is there a workaround for this, e.g. by manually setting some
>> xattrs on the files (in a hope that this might be faster than copying the
>> whole volume)?
>
> Your guess is right, just set xattr "trusted.glusterfs.version" to 3 to the
> entire tree structure files/dirs (including the exported directory) and try
> find + cat, it should work

Thanks for that, much appreciated. Would I be correct in assuming that 
setting this attribute to "3" is actually meant to mean "set it to a 
higher value on the server than it might be set on the client"?

Another question -  if the value of the trusted.glusterfs.version is 
higher on node2 (for whatever reason, e.g. reusing an old store 
dump directory) than on node1, where a fresh storage brick is initialized, 
would that lead to the empty directory on node2 clobbering all the data on 
node1, because the empty directory has a higher version number? I can see 
how that would make sense if it is the case, but that means that there are 
major dangers involved in manually initializing the store. The problem is 
similar to what happens in a split-brain situation (half of the files 
simply get silently dropped/replaced) if the same file was modified on 
both sides.

Thanks.

Gordan