[Gluster-devel] trusted.glusterfs.version xattr

[Gordan Bobic]

Martin Fick wrote:
> --- Gordan Bobic <gordan at bobich.net> wrote:
>> Martin Fick wrote:
>>>  /dir1/dir2/file
>>>
>>>   file and dir2 are deleted
>>>   dir2 is re-added
>>>   file is re-added and dir2 version is now the
>>>        same as it was before it was deleted.
>>>
>>> Same problem as before but one level higher. 
>>> You would need to version all the way to the 
>>> root, "/", for this to work, wouldn't you?
>> No. You just need to treat directories the same as
>> any other file. When a create/delete operation 
>> happens, the directory version gets bumped up.
>> So removing and creating a subdirectory (or a file)
>> would result in the created file/directory having a 
>> major version 2 versions higher than the
>> previous instance.
> 
> Think parent directory, not subdirectory.  If the 
> parent directory of the file and the file itself 
> are deleted and recreated you may end up with 
> the same versions of both the parent directory and
> file!

No - because the parent directory will have the incremented major 
version due to it's parent directory's minor version having been upped 
when the previous version of the subdirectory is deleted.

>   Original creation process and versioning:
> 
>   /
>  v1
>   /dir1/
>  v2   v1
>   /dir1/dir2/
>  v2   v2   v1
>   /dir1/dir2/file
>  v2   v2   v2  v1
> 
> Mirror goes off-line with version #s of dir2 and file
> as: v2/v1.
> 
> -> file deleted
> 
>   /dir1/dir2/
>  v2   v2   v3
> 
> -> dir2 deleted
> 
>   /dir1/
>  v2   v3
> 
> -> dir2 recreated
> 
>   /dir1/dir2/
>  v2   v4   v1
> 
> -> file recreated
> 
>   /dir1/dir2/file
>  v2   v4   v2   v1
> 
> Uh oh, now if I look just at the version #s of dir2 
> and file I get: v2/v1  these are the same as they 
> were above when our mirror went off-line, the file 
> looks like it is the same when in fact both it and 
> its parent directory have been recreated!  

You're missing the point. The proposal was for having TWO version 
numbers: major and minor.

Major is the parent directory's minor version number. This changes on 
the parent directory whenever a file/directory in it is created or 
deleted. This paren't minor version gets assigned as the file's major 
version at creation time only. Further updates to the file cause it's 
minor version to increase. When the file gets deleted, this falls under 
the creation/deletion, and causes the paren't minor version to increase. 
That means that creating a subdirectory or a file will get it's major 
version increased, so even if the minor version ends up the same, it'll 
still be a different total version.

Thinking about it, I don't think it's necessary to increase the parent's 
minor version on file deletes, only on file creates, provided this is 
done before the new file gets tagged with the major version (which is 
the same as parent's minor version at creation time.

Hmm, OK, I think I just reached the same conclusion as you - parent's 
minor version isn't sufficient - it'd have to be the complete version 
number of the parent, at which point there's a version chaining problem. 
Well spotted, Martin. :)

> However, if we were looking at the versions all the 
> way to the root, when the mirror went off-line we 
> would have had: /v2/v2/v2/v1 and now we have: 
> /v2/v4/v2/v1.  There is a chance that we are 
> talking about different files now.  Of course, the
> problem I see now is that the files could in fact 
> have been the same even though the version number is 
> different with this scheme!  Since the only version
> # that is different is that of dir1 (v4), this could 
> have been caused by simply adding two new files to 
> that directory!  

Indeed, then we have to resync the files that may well be the same. I 
was saying in jest before that we might as well be using a DHT if 
reliability and consistency aren't required, but now I'm thinking that 
perhaps tagging the file with it's SHA hash might be a possible way. 
Then we don't check if the version is newer, merely if it's different. 
Of course, the problem there is that calculating the hash of a big file 
is _expensive_ if we have to do it on each write.

> We now have the reverse problem, possible resyncing 
> when not needed.  This means that possibly every 
> single subdirectory/file of a directory needs to be 
> resynced.  Yikes, this problem would also be 
> prevalent (although less intense) even when just 
> using the parent's version # wouldn't it?  Every time
> a directory is reversioned, all the files in it are
> now reversioned?

Yes, I think so. Not really workable.

>>  > > Directory moves could create a similar problem:
>>  > >
>>  > >   /dir1/dir2/file
>>  > >   /dir1/dir3/file
>>  > >
>>  > >   /file and dir2 deleted.
>>  > >   dir3 moved to dir2 and happened to match file
>>  > >     and dir2 version #s.
>>  > >
>>  > > but I think that versioning to the root would
>>>> again solve this?
>> You don't need versioning up to the root, you just
>> have to treat moves the same way as copy+delete from
> 
>> the versioning point of view. This doesn't mean you 
>> actually have to copy+delete - you just have to 
>> update the metadata as if you did.
> 
> I wasn't implying that a move was a special case,
> rather that by moving the parent directory to a file
> you could end up with the same problem that I describe
> above where a file is a completely different file than
> on the mirror, but both the file and parent
> directory's version #s are the same.

Indeed, you've convinced me. I agree now. I don't think this is the 
solution.

Gordan