[GEDI] [RH-nfs-ganesha] gfapi: possible approaches to get lk_owner support

[Soumya Koduri]

Hi,

On 10/04/2017 06:47 PM, Frank Filz wrote:
> On 10/04/2017 05:36 AM, Matt Benjamin wrote:
>> Hi Soumya,
>>
>> On Wed, Oct 4, 2017 at 7:33 AM, Soumya Koduri <skoduri at redhat.com> wrote:
>>
>>> As far as the locks are concerned, looks like when we close a fd, server
>>> expects us to pass on lkowner as well and it cleans up locks with only that
>>> lkowner. That way we are right now safe to use glfs_dup and set another
>>> lkowner on the duplicate glfd returned.
>>>
>>> But if in case in future there comes any requirement to close all the locks
>>> tied to an fd, then the locks taken on both the glfd's shall be lost.
>> I've been trying to understand this precise point in the discussion.
>> Are you describing the potential effect of holding POSIX fcntl locks
>> in the server (as opposed to OFD locks, which I'd hope it used now, if
>> it uses such locks at all!), or something else?
>>
>> This discussion has been very closely tied to details of the
>> structures, sorry about the basic semantic questions.
>>
> Yea, I was going to say something about OFD locks. Definitely Gluster
> should investigate using OFD locks on the underlying filesystems so it
> never has to worry about losing locks. I'd be curious what it actually
> does on that side to manage the locks since the Gluster daemons should
> have the same issues Ganesha had about being a single process and
> therefore single lock owner. Does Gluster have to run on non-Linux?

Gluster doesn't have support of OFD locks but from what I understood the 
behavior seems to be almost similar (from today's tests). Gluster native 
client/Linux_FUSE_client sends lock_owners as part of lock requests and 
gluster server differentiates those lk_requests bases on that lk_owner 
set. Our effort (by this patch) is to have the same capability via gfapi 
as well.

Currently the only 2 cases where gluster server is releasing all the 
locks tied to a fd are -
* network disconnect - where it needs to flush all the fds taken by the 
client and in turn flushes the locks taken on that fd
* fd_destroy - this should be called when the last reference on that fd 
is released.

glfs_dup() doesn't actually act like a fd_dup (maybe buggy) but just 
takes an extra reference on the existing fd. My only point is that since 
Gluster doesn't support OFD locks yet, I do not know if there shall be 
any other case (IN FUTURE) where in we need to flush the locks tied to a 
fd which may break our current assumption with glfs_dup and there is no 
standard written on how glfs_dup should/will actually behave which 
applications can rely on.

Maybe this concern is pointless as things work as expected at the 
moment. We should probably keep an eye of any such changes. Will include 
a upstream regression test case in the patch set using glfs_dup which 
should let us know of any failures later.

@Jiffin,

may be the test case which you had written could be added to our ganesha 
CentOS-CI regression tests.

Thanks,
Soumya
> 
> We also need to push very hard against ever implementing the broken
> POSIX behavior, and if we do, allow a bypass of it with OFD locks
> (though then we might need a new glfs_dup that creates a new "open file
> description" in case in the process of making glfapi work more like
> POSIX (unfortunate in this circumstance) it is also expected that
> glfs_dup would share the open file description and thus lock owner or
> something crazy like that...
> 
> Frank
>