[GEDI] [RH-nfs-ganesha] gfapi: possible approaches to get lk_owner support
skoduri at redhat.com
Wed Oct 4 13:37:49 UTC 2017
On 10/04/2017 06:47 PM, Frank Filz wrote:
> On 10/04/2017 05:36 AM, Matt Benjamin wrote:
>> Hi Soumya,
>> On Wed, Oct 4, 2017 at 7:33 AM, Soumya Koduri <skoduri at redhat.com> wrote:
>>> As far as the locks are concerned, looks like when we close a fd, server
>>> expects us to pass on lkowner as well and it cleans up locks with only that
>>> lkowner. That way we are right now safe to use glfs_dup and set another
>>> lkowner on the duplicate glfd returned.
>>> But if in case in future there comes any requirement to close all the locks
>>> tied to an fd, then the locks taken on both the glfd's shall be lost.
>> I've been trying to understand this precise point in the discussion.
>> Are you describing the potential effect of holding POSIX fcntl locks
>> in the server (as opposed to OFD locks, which I'd hope it used now, if
>> it uses such locks at all!), or something else?
>> This discussion has been very closely tied to details of the
>> structures, sorry about the basic semantic questions.
> Yea, I was going to say something about OFD locks. Definitely Gluster
> should investigate using OFD locks on the underlying filesystems so it
> never has to worry about losing locks. I'd be curious what it actually
> does on that side to manage the locks since the Gluster daemons should
> have the same issues Ganesha had about being a single process and
> therefore single lock owner. Does Gluster have to run on non-Linux?
Gluster doesn't have support of OFD locks but from what I understood the
behavior seems to be almost similar (from today's tests). Gluster native
client/Linux_FUSE_client sends lock_owners as part of lock requests and
gluster server differentiates those lk_requests bases on that lk_owner
set. Our effort (by this patch) is to have the same capability via gfapi
Currently the only 2 cases where gluster server is releasing all the
locks tied to a fd are -
* network disconnect - where it needs to flush all the fds taken by the
client and in turn flushes the locks taken on that fd
* fd_destroy - this should be called when the last reference on that fd
glfs_dup() doesn't actually act like a fd_dup (maybe buggy) but just
takes an extra reference on the existing fd. My only point is that since
Gluster doesn't support OFD locks yet, I do not know if there shall be
any other case (IN FUTURE) where in we need to flush the locks tied to a
fd which may break our current assumption with glfs_dup and there is no
standard written on how glfs_dup should/will actually behave which
applications can rely on.
Maybe this concern is pointless as things work as expected at the
moment. We should probably keep an eye of any such changes. Will include
a upstream regression test case in the patch set using glfs_dup which
should let us know of any failures later.
may be the test case which you had written could be added to our ganesha
CentOS-CI regression tests.
> We also need to push very hard against ever implementing the broken
> POSIX behavior, and if we do, allow a bypass of it with OFD locks
> (though then we might need a new glfs_dup that creates a new "open file
> description" in case in the process of making glfapi work more like
> POSIX (unfortunate in this circumstance) it is also expected that
> glfs_dup would share the open file description and thus lock owner or
> something crazy like that...
More information about the integration