[Gluster-users] NFS mounts with glusterd on localhost - reliable or not?

Wed Jul 18 22:42:39 UTC 2012

This is another reason why it there *really* should be a way with the CLI to
have an option that GlusterNFS be enabled on  only one machine in a cluster
(I've asked for that before). Our cluster has a private network.  Inside the
cluster private network I use native glusterfs mounts.  Outside the private
network, I use an nfs mount exported by a gateway machine.  For us it is only
necessary to have GlusterNFS running on that gateway machine.

Having the glusterfs file system mounted nolock I can live with but but it is
intolerable if the home directories from outside the cluster that are mounted
with automount in the gluster require nolock.  I might be able to tolerate this
it if this is the situation is only on the gateway machine, not every machine
in the cluster.

In the the past I used a hack to create a peer that had only Kernel NFS by
making sure that the Kernel NFS came up first on boot.  Is that still true?

Bill Sebok      Computer Software Manager, Univ. of Maryland, Astronomy
	Internet: wls at astro.umd.edu	URL: http://furo.astro.umd.edu/

On Wed, Jul 18, 2012 at 01:56:04AM -0400, Rajesh Amaravathi wrote:
> Let me elucidate with an example:
> 
> host1: GlusterNfs server, say "vol"
> 
> host2: Kernel Nfs export: say, "export"
> 
> Assuming host1 and host2 are not peers, i.e, host2 does NOT have any Gluster Nfs servers running,
> Lets assume for some reason, "export" needs to be mounted on host1.
> 
> This is not possible *with locking* since glusterNfs already has the portmap registration.
> Even if it is mounted, then the kernel nfs client starts the kernel NLM v4, which will override
> the portmap registration of NLMv4 on host1, so Gluster NFS' NLM implementation wouldn't work.
> 
> However, you can mount "export" on host1 with "-o nolock" option, since the kernel nfs client
> then does not attempt to spawn its own NLM.
> 
> Also, the only way host1 can mount "vol" via nfs is by specifying "nolock" option, else
> the same conflict arises.
> 
> As to whether we can disable parts of kernel NFS (I'm assuming e.g NLM), I think
> its not really necessary since we can mount other exports with nolock option.
> If we take out NLM or disable NLM at the kernel level, then every time we need
> NLM from kernel, we need to recompile the kernel/have a secondary kernel with NLM
> and reboot, much tedious than simply killing Gluster/fuse NFS and after kernel NLM's
> work is done, restart Gluster/fuse NFS. My $0.02 :) 
>  
> 
> Regards, 
> Rajesh Amaravathi, 
> Software Engineer, GlusterFS 
> RedHat Inc. 
> 
> ----- Original Message -----
> From: "Whit Blauvelt" <whit.gluster at transpect.com>
> To: "Rajesh Amaravathi" <rajesh at redhat.com>
> Cc: "Gluster General Discussion List" <gluster-users at gluster.org>
> Sent: Tuesday, July 17, 2012 5:38:48 PM
> Subject: Re: [Gluster-users] NFS mounts with glusterd on localhost - reliable or not?
> 
> Sorry my question was too vague. What I meant to ask is if it is possible,
> since there is a conflict between the locking requests from the kernel's NFS
> and from Gluster/fuse's NFS, that the kernel might be compiled so with some
> or all of its NFS support disabled, so that then Gluster/fuse NFS-locking
> would work.
> 
> Perhaps longer term there needs to be a way to have the kernel shut its NFS
> locking attempts off, just if there is a userland NFS such as Gluster's
> running. Meanwhile can enough of NFS be taken out of a custom kernel to
> allow Gluster to lock?
> 
> Thanks,
> Whit
> 
> On Tue, Jul 17, 2012 at 08:03:03AM -0400, Rajesh Amaravathi wrote:
> > it should be possible to mount another kernel export with -o nolock option and
> > compile kernel on it. I'm just guessing when you mount with nolock option,
> > we are mounting for mostly read purposes and not for critical writes.
> > 
> > Regards, 
> > Rajesh Amaravathi, 
> > Software Engineer, GlusterFS 
> > RedHat Inc. 
> > 
> > ----- Original Message -----
> > From: "Whit Blauvelt" <whit.gluster at transpect.com>
> > To: "Rajesh Amaravathi" <rajesh at redhat.com>
> > Cc: "David Coulson" <david at davidcoulson.net>, "Gluster General Discussion List" <gluster-users at gluster.org>
> > Sent: Monday, July 16, 2012 9:56:28 PM
> > Subject: Re: [Gluster-users] NFS mounts with glusterd on localhost - reliable or not?
> > 
> > Say, is it possible to compile a kernel without whatever part of its NFS
> > support competes with Gluster's locking? 
> > 
> > Whit
> > 
> > On Fri, Jul 13, 2012 at 08:14:27AM -0400, Rajesh Amaravathi wrote:
> > > I hope you do realize that two NLM implementations of the same version
> > > cannot operate simultaneously in the same machine. I really look forward
> > > to a solution to make this work, that'd be something.
> > > 
> > > Regards, 
> > > Rajesh Amaravathi, 
> > > Software Engineer, GlusterFS 
> > > RedHat Inc. 
> > > 
> > > ----- Original Message -----
> > > From: "David Coulson" <david at davidcoulson.net>
> > > To: "Rajesh Amaravathi" <rajesh at redhat.com>
> > > Cc: "Tomasz Chmielewski" <mangoo at wpkg.org>, "Gluster General Discussion List" <gluster-users at gluster.org>
> > > Sent: Friday, July 13, 2012 5:28:04 PM
> > > Subject: Re: [Gluster-users] NFS mounts with glusterd on localhost - reliable or not?
> > > 
> > > Was that introduced by the same person who thought that binding to 
> > > sequential ports down from 1024 was a good idea?
> > > 
> > > Considering how hard RedHat was pushing Gluster at the Summit a week or 
> > > two ago, it seems like they're making it hard for people to really 
> > > implement it in any capacity other than their Storage Appliance product.
> > > 
> > > Luckily I don't need locking yet, but I suppose RedHat will be happy 
> > > when I do since I'll need to buy more GFS2 Add-Ons for my environment :-)
> > > 
> > > David
> > > 
> > > On 7/13/12 7:49 AM, Rajesh Amaravathi wrote:
> > > > Actually, if you want to mount *any* nfs volumes(of Gluster) OR
> > > > exports (of kernel-nfs-server), you cannot do it with locking on
> > > > a system where a glusterfs(nfs process) is running(since 3.3.0).
> > > > However, if its ok to mount without locking, then you should be
> > > > able to do it on localhost.
> > > >
> > > > Regards,
> > > > Rajesh Amaravathi,
> > > > Software Engineer, GlusterFS
> > > > RedHat Inc.
> > > >
> > > > ----- Original Message -----
> > > > From: "David Coulson" <david at davidcoulson.net>
> > > > To: "Tomasz Chmielewski" <mangoo at wpkg.org>
> > > > Cc: "Rajesh Amaravathi" <rajesh at redhat.com>, "Gluster General Discussion List" <gluster-users at gluster.org>
> > > > Sent: Friday, July 13, 2012 3:16:38 PM
> > > > Subject: Re: [Gluster-users] NFS mounts with glusterd on localhost - reliable or not?
> > > >
> > > >
> > > > On 7/13/12 5:29 AM, Tomasz Chmielewski wrote:
> > > >> Killing the option to use NFS mounts on localhost is certainly quite
> > > >> the opposite to my performance needs!
> > > >>
> > > > He was saying you can't run kernel NFS server and gluster NFS server at
> > > > the same time, on the same host. There is nothing stopping you from
> > > > mounting localhost:/volume on all your boxes. That is exactly how our
> > > > 3.2.5 and 3.3.0 environments access volumes for the performance reasons
> > > > you identified.
> > > 
> > > 
> > > _______________________________________________
> > > Gluster-users mailing list
> > > Gluster-users at gluster.org
> > > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users

-- 
--- Bill Sebok