[Gluster-devel] Lease-lock Design notes

Tue Jul 21 09:19:35 UTC 2015

Hi Shyam, 

Please find my reply inline.

Rgards,
Poornima

----- Original Message -----
> From: "Ira Cooper" <icooper at redhat.com>
> To: "Shyam" <srangana at redhat.com>
> Cc: "Gluster Devel" <gluster-devel at gluster.org>
> Sent: Saturday, July 18, 2015 4:09:30 AM
> Subject: Re: [Gluster-devel] Lease-lock Design notes
> 
> 1. Yes, it is intentional.  The internals of gluster should be able to
> lease-lks.  We discussed using them in the read ahead translator and the
> write behind translator.
> 2. This has been discussed, and proposed, but there is actually a need for a
> lease fop also, because clients can request the "renewal" or "reinstatement"
> of a lease.  (Actually for Samba having it all be one call and atomic is
> very interesting.)
> 3. This I can't answer... I haven't been in the most recent discussions.  But
> the intent of this work, when I started, was to be useful to the whole of
> Gluster, not just Samba or Ganesha.
> 
> Thanks,
> 
> -Ira
> 
> ----- Original Message -----
> > Hi,
> > 
> > I have some questions or gaps in understanding the current lease design,
> > request some clarification on the same.
> > 
> > 1) Is there a notion of a *gluster* client holding a lease on a file/dir?
Yes, a lease is granted to a gluster client on a file, dir is not yet implemented, but is on cards.
The lease can be requestd from any xlator, and is granted for the whole client.

> > - As per your Barcelona gluster summit presentation the client side
> > caching needs this notion
> > - DHT/EC/AFR can leverage such a lease to prevent eager or any form of
> > locking when attempting to hold consistency of data being operated on
> >    - Please note that this requires not each xlator requesting and
> > holding the lease, but a *gluster* client holding the lease, assuming
> > that one can do local in memory locking to prevent different
> > fd/connection/applications performing operations on the same file
> > against the *same* gluster client and not have to coordinate this with
> > the brick
> > 
> > I see some references to this in the design but wanted to understand if
> > the thought is there.
The initial thought of having leases in Gluster was to support Multiprotocol access,
aprt from this another use case we saw was, having a Gluster client cache xlator
which caches taking leases. Yes, DHT/EC/AFR also could leverage leases, but i m not
sure if leases can replace eager locks.

> > 
> > IOW, if an NFS client requests a lease, Ganesha requests the lease from
> > Gluster, in which case the gfapi client that requested the lease first
> > gets the lease, and then re-leases it to Ganesha, now Ganesha is free to
> > lease it to any client on its behalf and recall leases etc. as the case
> > may be and the gluster client does not care. When another gluster client
> > (due to Ganesha or otherwise (say self heal or rebalance)) attempts to
> > get a lease, that is when the lease is broken all across.
> > 
> > Is my understanding right, is the design along these lines?
Yes, that is right, any conflicts between its(ganesha/Samba) clients should be
resolved by the Ganesha/Samba server. Gluster server will handle the conflicts
across gluster clients.

> > 
> > 2) Why is the lease not piggy backed with the open? Why 2 network FOPs
> > instead of 1 in this case? How can we compound this lease with other
> > requests? Or do you have thoughts around the same?
> > 
> > Isn't NFS and SMB requesting leases with its open calls
Our initial thought was to overload lk call to request for lease, and also
support open+lease. But the problems with fd based leases were:
 - Doesn't go well with the NFS world, where handles are created and
   delegations are granted followed by multiple open/read/write/close
   on that fd. Hence lease is more conveniently associated with handles
   than fds, in NFS.
 - Anonymous fds are used by NFSV3 and pnfs to maintain statelessness,
   aonymous fds means there is no open fd on the file, the backend opens
   read/writes to the file and closes. These anonfds makes it harder to get
   the lease conflict check right, and we may break leases when it is not
   necessary.
 - The lease might have to live longer than fd (handle based leases, i.e.
   when persistent/durable handles will be used instead of fd).
That said, we could even now overload open call to request for lease,
but it will be associated with the inode and the client and not the fd,
for the above said reasons.

> > 
> > 3) If there were discussions around some designs that were rejected,
> > could you also update the document with the same, so the one can
> > understand the motivation behind the current manner of implementing leases.
Yes sure, we are updating the leases.md, will send it at the earliest.

> > 
> > Thanks,
> > Shyam
> > 
> > On 04/16/2015 07:37 AM, Soumya Koduri wrote:
> > > Hi,
> > >
> > > Below link contains the lease-lock design notes (as per the latest
> > > discussion we had). Thanks to everyone involved (CC'ed).
> > >
> > > http://www.gluster.org/community/documentation/index.php/Features/Upcall-infrastructure#delegations.2Flease-locks
> > >
> > >
> > > Kindly go through the same and provide us your inputs.
> > >
> > > Thanks,
> > > Soumya
> > > _______________________________________________
> > > Gluster-devel mailing list
> > > Gluster-devel at gluster.org
> > > http://www.gluster.org/mailman/listinfo/gluster-devel
> > _______________________________________________
> > Gluster-devel mailing list
> > Gluster-devel at gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-devel
> > 
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
> 