[Gluster-devel] DHT: Making Dir rename op's crash consistent

[Anand Avati]

On Sun, Oct 28, 2012 at 11:30 PM, Shishir Gowda <sgowda at redhat.com> wrote:

> Hi All,
>
> This is a proposed enhancement to DHT directory rename operations to make
> it recoverable in-case of crashes.
>
> Please feel free to review/comment on the design. There also 2 open issues
> which need to tackled (see below in recovery logic)
>
> We propose to add 2 new on-disk xattrs SRC(<key to be decided>:destination
> path) and DST(<key to be decided>:src path/gfid).
>
> Consider these scenarios
>
> case1. Only source directory exists
> case2. Both source, and destination directories exist.
>
> The tasks for rename would be as follows:
>
> 1. Set SRC key on all source directories
> 2. If step 1 fails, remove xattrs, and fail rename
> 3. If case2, set xattrs on destination directories
> 4. If failure in case2, ignore
> 5. Rename directories (opendir on dst, readdir(ENOEMPTY error), rename
> dst_hashed subvol first, and then rest)
>

We probably need to guarantee empty dst dirs on _all_ dst servers before
progressing to step 6?(else, fail ENOTEMPTY)?


> 6. If step 5 fails with any error other than ENOTCONN, fail rename, and
> remove xattrs
> 7. If failure is because of ENOTCONN, proceed with rename and return a
> success.
>
> Recovery steps (once the brick comes up):
>
> 1. On lookup/readdir (NFS requirement?) query for these SRC and DST key.
> 2. If SRC key is found , validate:
>    a. If mtime is less than 5 seconds of the lookup request, then do not
> heal, as rename might be in progress (Can we make this more fool proof?)
>    b. If dst does, not exist, proceed
>

This is a bit confusing. Wasn't DST xattr set only on the destination
directory in step 3?


>    c. If dst exists, check its key and see if they match. If mismatch, do
> not rename, as it might lead to gfid mis-match.


> 3. Proceed with checks rename of directories (similar to step 5 of above
> (rename).
> 4. If successful, remove xattrs, return success.
> 5. If failure what needs to be done? (other rename's might have succeeded,
> this might fail due to ENOTEMPTY(even due to race)
>
>
> As for subvol down, we can't guarantee in the scenarios of brick going
> down after stage 1(setxattr).
>
> Brick going down before start of subvolume: We do not allow rename to
> progress anywhere.
>
> If a brick goes down after setxattr, if it has files, or files are created
> after its up (possible race), then we cant recover.
>
>
> With regards,
> Shishir
>
>
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> https://lists.nongnu.org/mailman/listinfo/gluster-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-devel/attachments/20130123/b8353b22/attachment-0001.html>