[GEDI] [RH-nfs-ganesha] gfapi: possible approaches to get lk_owner support

[Frank Filz]

On 10/04/2017 06:37 AM, Soumya Koduri wrote:
>
> Hi,
>
> On 10/04/2017 06:47 PM, Frank Filz wrote:
>> On 10/04/2017 05:36 AM, Matt Benjamin wrote:
>>> Hi Soumya,
>>>
>>> On Wed, Oct 4, 2017 at 7:33 AM, Soumya Koduri <skoduri at redhat.com>
>>> wrote:
>>>
>>>> As far as the locks are concerned, looks like when we close a fd,
>>>> server
>>>> expects us to pass on lkowner as well and it cleans up locks with
>>>> only that
>>>> lkowner. That way we are right now safe to use glfs_dup and set
>>>> another
>>>> lkowner on the duplicate glfd returned.
>>>>
>>>> But if in case in future there comes any requirement to close all
>>>> the locks
>>>> tied to an fd, then the locks taken on both the glfd's shall be lost.
>>> I've been trying to understand this precise point in the discussion.
>>> Are you describing the potential effect of holding POSIX fcntl locks
>>> in the server (as opposed to OFD locks, which I'd hope it used now, if
>>> it uses such locks at all!), or something else?
>>>
>>> This discussion has been very closely tied to details of the
>>> structures, sorry about the basic semantic questions.
>>>
>> Yea, I was going to say something about OFD locks. Definitely Gluster
>> should investigate using OFD locks on the underlying filesystems so it
>> never has to worry about losing locks. I'd be curious what it actually
>> does on that side to manage the locks since the Gluster daemons should
>> have the same issues Ganesha had about being a single process and
>> therefore single lock owner. Does Gluster have to run on non-Linux?
>
> Gluster doesn't have support of OFD locks but from what I understood
> the behavior seems to be almost similar (from today's tests). Gluster
> native client/Linux_FUSE_client sends lock_owners as part of lock
> requests and gluster server differentiates those lk_requests bases on
> that lk_owner set. Our effort (by this patch) is to have the same
> capability via gfapi as well.
>
> Currently the only 2 cases where gluster server is releasing all the
> locks tied to a fd are -
> * network disconnect - where it needs to flush all the fds taken by
> the client and in turn flushes the locks taken on that fd
> * fd_destroy - this should be called when the last reference on that
> fd is released.
>
> glfs_dup() doesn't actually act like a fd_dup (maybe buggy) but just
> takes an extra reference on the existing fd. My only point is that
> since Gluster doesn't support OFD locks yet, I do not know if there
> shall be any other case (IN FUTURE) where in we need to flush the
> locks tied to a fd which may break our current assumption with
> glfs_dup and there is no standard written on how glfs_dup should/will
> actually behave which applications can rely on.
>
> Maybe this concern is pointless as things work as expected at the
> moment. We should probably keep an eye of any such changes. Will
> include a upstream regression test case in the patch set using
> glfs_dup which should let us know of any failures later. 
How does the server actually do the locks? Does it use fcntl locks to
the underlying real filesystem or are the locks instantiated some other way?

We should make sure any behaviors we rely on are documented, preferably
in the code.

My suggestion of OFD locks was to potentially use the OFD lock defines
(don't you use the fcntl structure to define locks?) so that we can
indicate the behavior we expect of them. On the other hand, one might
argue convincingly that glfs_dup SHOULD effectively create a new glfd
that is the same "open file descriptor" - it IS using the same fd over
on the server...

I just want to make sure the semantics we use are well documented and
defined. POSIX has some weird semantics, but at least for the most part
they are defined, and at least documented somewhere...

Frank