[Gluster-devel] Implementing Flat Hierarchy for trashed files

[Jiffin Tony Thottan]

Comments inline.

On 18/08/15 09:54, Niels de Vos wrote:
> On Mon, Aug 17, 2015 at 06:20:50PM +0530, Anoop C S wrote:
>> Hi all,
>>
>> As we move forward, in order to fix the limitations with current trash
>> translator we are planning to replace the existing criteria for trashed
>> files inside trash directory with a general flat hierarchy as described
>> in the following sections. Please have your thoughts on following
>> design considerations.
>>
>> Current implementation
>> ======================
>> * Trash translator resides on glusterfs server stack just above posix.
>> * Trash directory (.trashcan) is created during volume start and is
>>    visible under root of the volume.
>> * Each trashed file is moved (renamed) to trash directory with an
>>    appended time stamp in the file name.
>> * Exact directory hierarchy (w.r.t the root of volume) is maintained
>>    inside trash directory whenever a file is deleted/truncated from a
>>    directory
>>
>> Outstanding issues
>> ==================
>> * Since renaming occurs at the server side, client-side is unaware of
>>    trash doing rename or create operations.
>> * As a result files/directories may not be visible from mount point.
> This might be something upcall could help with. If the trash xlator is
> placed above upcall, any clients interested in the .trashcan directory
> (or subdirs) could get an in/revalidation request.
>
>> * Files/Directories created from from trash translator will not have
>>    gfid associated with it until lookup is performed.
> When a client receives an invalidation of the parent directory (from
> upcall), a LOOKUP will follow on the next request.

If I understand it correctly , solution become more complex if integrate 
both translator and upcall together.
1.) Upcall notification can be send to a client only if it has accessed 
.trashcan
2.) There should be translator at client side to initiate lookup after 
receiving upcall notification
3.) Performance hit. Say file `foo`is present in a/b/c/. We need to 
create path a/b/c/ inside trash directory.
So ideally trash xlator will first create directory 'a' , then send 
upcall notification to all of the client and then clients will initiate 
lookup on 'a',
perform gfid healing on that directory. After that it will create `b` 
and repeat the same procedure.
>> Proposed Flat hierarchy
>> =======================
> I'm missing a bit of info here, what limitations need to be addressed?

all above mentioned outstanding issues can be addressed by the flat 
hierarchy.
>> * Instead of creating the whole directory under trash, we will rename
>>    the file and place it directly under trash directory (of course with
>>    appended time stamp).
>> * Directory hierarchy can be stored via either of the following two
>>    approaches:
>> 	(a) File name will contain the whole path with time stamp
>> 	    appended
>> 	(b) Store whole hierarchy as an xattr
> If this is needed, definitely go with (b). Filenames have a limit, and
> the full path (directories + filename + timestamp) could surely hit
> that.

Thanks for the suggestion.

>> Other enhancements
>> ==================
> Have these been filed as bugs/RFEs? If not, please do so and include a
> good description of the work that is needed. Maybe others in the Gluster
> community are interested in providing patches, and details on what to do
> is very helpful.

Sure. We will file different RFE's as soon as possible and sent it in 
different mail.

> Thanks,
> Niels
>
>> * Create the trash directory only
>> when trash xlator is enabled.
>> * Operations such as unlink, rename etc
>> will be prevented on trash
>>    directory only when trash xlator is
>> enabled.
>> * A new trash helper translator on client side(loaded only when
>> trash
>>    is enabled) to resolve split brain issues with truncation of
>> files.
>> * Restore files from trash with the help of an explicit setfattr
>> call.
>>
>> Thanks & Regards,
>> -Anoop C S
>> -Jiffin Tony Thottan
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>

--
Jiffin
>>
>>
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20150818/46db547a/attachment-0001.html>