[Gluster-devel] file version on glusterfs using libgit

[Kaleb KEITHLEY]

On 03/08/2013 04:50 PM, Niels de Vos wrote:
> On Fri, Mar 08, 2013 at 06:00:24AM -0500, Shishir Gowda wrote:
>> Hi Niels,
>>
>> Thinking out aloud, I think the snaps(in file version context) can be
>> displayed as branches (list).
>
> Well, I am not sure if branches are really needed. Isn't linear history
> sufficient? Every change should be committed to the master branch
> anyway. Branches may be useful for switching between versions, but
> nothing prevents you from checking out (or "git ls-files") with a commit
> or a date.

I think snapshots can be implemented with tags, and had also envisioned 
a virtual directory scheme like Niels has described. There could/should 
be a cron job that makes yearly, monthly, daily, and/or even hourly 
snapshots.

I could imagine using branches for a kind of auto-dedupe copy mechanism. 
If I copy a file to another location, a branch is made, and the new copy 
is a magic symlink that points to the branch.

>
> I'm thinking of a virtual .snaps directory:
>
> $ cd $VOLUME/.snaps
> $ ls
> 2013-03-07/
> 2013-03-06/
> .....
> current/
> changelog
> yesterday -> 2013-03-07/
>
> This makes it possible to do something like:
> $ cat changelog
>     - virtual file, showing the contents of 'git log'
>     - find a commit you're interested in
> $ mkdir $GIT_COMMIT_ID
> $ ls $GIT_COMMIT_ID/
>     - get the state just like 'git checkout $GIT_COMMIT_ID'
>
> Maybe it would be helpful to be able to create tags inside this .snaps
> directory. But I would refrain from branches for now (unless there is
> a clear use-case).
>
> Cheers,
> Niels
>
>> Once the user cd's into any one of them, we could do a git checkout of the branch.
>>
>> That should mimic the behaviour.
>>
>> With regards,
>> Shishir
>>
>> ----- Original Message -----
>> From: "Shishir Gowda" <sgowda at redhat.com>
>> To: "Niels de Vos" <ndevos at redhat.com>
>> Cc: gluster-devel at nongnu.org
>> Sent: Friday, March 8, 2013 4:04:42 PM
>> Subject: Re: [Gluster-devel] file version on glusterfs using libgit
>>
>> Hi Niels,
>>
>> My inclination too is to load git ontop of posix xlator.
>>
>> I was thinking of making previous versions (based on some policy) to be treated a new branch.
>>
>> We could see how to export these branches as user visible dirs.
>>
>> With regards,
>> Shishir
>>
>> ----- Original Message -----
>> From: "Niels de Vos" <ndevos at redhat.com>
>> To: "Shishir Gowda" <sgowda at redhat.com>
>> Cc: gluster-devel at nongnu.org
>> Sent: Friday, March 8, 2013 3:25:57 PM
>> Subject: Re: [Gluster-devel] file version on glusterfs using libgit
>>
>> On Thu, Mar 07, 2013 at 12:54:41AM -0500, Shishir Gowda wrote:
>>> Hi All,
>>>
>>> Was playing around with git on glusterfs volume, to provide was of file version support.
>>>
>>> And initial run is encouraging.
>>>
>>> A brief overview what was tried:
>>>
>>> Approach 1: Glusterfs volume as a git repo
>>>
>>> 1. created a 2 brick distribute volume
>>> 2. inited a git repo on fuse volume
>>> 3. created files, committed them in git.
>>> 4. Modified files, and committed them again
>>> 5. Did branch check-outs, to simulate versions @ point in time
>>> 6. reset branch heads, and was able access older version of files (after a stash).
>>> 7. Was able to create files/dirs/symlinks/hardlinks
>>> 8. Both NFS/FUSE clients were used.
>>>
>>> Approach 2: Glusterfs bricks as git repo's
>>>
>>> 1. created a 2 brick distribute volume
>>> 2. inited git repo on brick1
>>> 3. inited git repo on brick2
>>> 4. created files, committed the relevant brick's git.
>>> 5. Modified files, and committed them again on brick's git
>>> 6. Did branch check-outs, to simulate versions @ point in time on individual bricks
>>> 7. reset branch heads, and was able access older version of files (after a stash).
>>> 8. Was able to create files/dirs/symlinks/hardlinks
>>> 9. Both NFS/FUSE clients were used.
>>>
>>> Buoyed by this, will start prototyping integration of libgit2 as xlator for file version support.
>>>
>>> There are 3 approaches to consider:
>>>
>>> 1. Load git xlator on clients volfiles
>>> 2. Load git xlator on server volfiles
>>> 3. Replace posix interface with git interface.
>>>
>>> Please provide feedback, on what would be more desirable.
>>
>> Very interesting! Option 2 makes most sense to me, the posix xlator
>> contains some access checks and such, which you probably should not need
>> to duplicate.
>>
>> Have you thought about making the previous version accessible through
>> the glusterfs/nfs mount? Other vendors seem to have a .snapshot
>> directory with previous versions, would something like that be possible?
>> Users would be able to recover deleted files themselves that way.
>>
>> Also, I do not know if git stores xattrs and their changes...
>>
>> Cheers,
>> Niels
>>
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at nongnu.org
>> https://lists.nongnu.org/mailman/listinfo/gluster-devel
>