[Gluster-devel] Implementing Flat Hierarchy for trashed files

[Niels de Vos]

On Tue, Aug 18, 2015 at 04:51:58PM +0530, Jiffin Tony Thottan wrote:
> Comments inline.
> 
> On 18/08/15 09:54, Niels de Vos wrote:
> >On Mon, Aug 17, 2015 at 06:20:50PM +0530, Anoop C S wrote:
> >>Hi all,
> >>
> >>As we move forward, in order to fix the limitations with current trash
> >>translator we are planning to replace the existing criteria for trashed
> >>files inside trash directory with a general flat hierarchy as described
> >>in the following sections. Please have your thoughts on following
> >>design considerations.
> >>
> >>Current implementation
> >>======================
> >>* Trash translator resides on glusterfs server stack just above posix.
> >>* Trash directory (.trashcan) is created during volume start and is
> >>   visible under root of the volume.
> >>* Each trashed file is moved (renamed) to trash directory with an
> >>   appended time stamp in the file name.
> >>* Exact directory hierarchy (w.r.t the root of volume) is maintained
> >>   inside trash directory whenever a file is deleted/truncated from a
> >>   directory
> >>
> >>Outstanding issues
> >>==================
> >>* Since renaming occurs at the server side, client-side is unaware of
> >>   trash doing rename or create operations.
> >>* As a result files/directories may not be visible from mount point.
> >This might be something upcall could help with. If the trash xlator is
> >placed above upcall, any clients interested in the .trashcan directory
> >(or subdirs) could get an in/revalidation request.
> >
> >>* Files/Directories created from from trash translator will not have
> >>   gfid associated with it until lookup is performed.
> >When a client receives an invalidation of the parent directory (from
> >upcall), a LOOKUP will follow on the next request.
> 
> If I understand it correctly , solution become more complex if integrate
> both translator and upcall together.
> 1.) Upcall notification can be send to a client only if it has accessed
> .trashcan

Correct, and those are the only clients we care about. Clients that do
not have the .trashcan directory entries cached do not need to
invalidate that cache.

> 2.) There should be translator at client side to initiate lookup after
> receiving upcall notification

No, that is not needed. A LOOKUP will happen when the directory/inode
needs a revalidate. After an invalidate from upcall, the next revalidate
will cause a LOOKUP.

> 3.) Performance hit. Say file `foo`is present in a/b/c/. We need to create
> path a/b/c/ inside trash directory.
> So ideally trash xlator will first create directory 'a' , then send upcall
> notification to all of the client and then clients will initiate lookup on
> 'a',
> perform gfid healing on that directory. After that it will create `b` and
> repeat the same procedure.

Yes, directories are more tricky. But I think this is still a better
approach then renaming a file to include the full path.

> >>Proposed Flat hierarchy
> >>=======================
> >I'm missing a bit of info here, what limitations need to be addressed?
> 
> all above mentioned outstanding issues can be addressed by the flat
> hierarchy.

But you also introduce new issues. A huge directory to browse would be
major concern. Users like doing directory listings and that is one of
the worst workloads we have on Gluster :-/

> >>* Instead of creating the whole directory under trash, we will rename
> >>   the file and place it directly under trash directory (of course with
> >>   appended time stamp).
> >>* Directory hierarchy can be stored via either of the following two
> >>   approaches:
> >>	(a) File name will contain the whole path with time stamp
> >>	    appended
> >>	(b) Store whole hierarchy as an xattr
> >If this is needed, definitely go with (b). Filenames have a limit, and
> >the full path (directories + filename + timestamp) could surely hit
> >that.
> 
> Thanks for the suggestion.
> 
> >>Other enhancements
> >>==================
> >Have these been filed as bugs/RFEs? If not, please do so and include a
> >good description of the work that is needed. Maybe others in the Gluster
> >community are interested in providing patches, and details on what to do
> >is very helpful.
> 
> Sure. We will file different RFE's as soon as possible and sent it in
> different mail.

Thanks!

> >Thanks,
> >Niels
> >
> >>* Create the trash directory only
> >>when trash xlator is enabled.
> >>* Operations such as unlink, rename etc
> >>will be prevented on trash
> >>   directory only when trash xlator is
> >>enabled.
> >>* A new trash helper translator on client side(loaded only when
> >>trash
> >>   is enabled) to resolve split brain issues with truncation of
> >>files.
> >>* Restore files from trash with the help of an explicit setfattr
> >>call.
> >>
> >>Thanks & Regards,
> >>-Anoop C S
> >>-Jiffin Tony Thottan
> >>_______________________________________________
> >>Gluster-devel mailing list
> >>Gluster-devel at gluster.org
> >>http://www.gluster.org/mailman/listinfo/gluster-devel
> >>
> 
> --
> Jiffin
> >>
> >>
> >>_______________________________________________
> >>Gluster-devel mailing list
> >>Gluster-devel at gluster.org
> >>http://www.gluster.org/mailman/listinfo/gluster-devel
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20150819/32c8958c/attachment.sig>