[Gluster-users] Unable to start nfs server
David Coulson
david at davidcoulson.net
Tue Mar 6 00:58:54 UTC 2012
I poke around at this today and figured it out - Sort of.
When I did 'gluster volume set named nfs.register-with-portmap on'
originally, I only had rpcbind running on two of my four servers.
Gluster nfs started up on all four, but obviously only two correctly
connected with rpcbind/portmap. Seems that if rpcbind is not running
when you set 'register-with-portmap on', even when you cycle gluster it
still doesn't work.
So, I started up rpcbind, did a 'register-with-portmap off', followed by
'register-with-portmap on' and it works now.
When I diffed the before and after nfs-server.vol files, I see this:
[root at dresproddns01 ~]# diff nfs-vol-1 /etc/glusterd/nfs/nfs-server.vol
143c143
< option rpc.register-with-portmap off
---
> option rpc.register-with-portmap on
Apparently if rpcbind is not running, the option does not get enabled
properly. There is an error in nfs.log, but it's hard to find especially
if the node you manage the cluster from isn't the node with the issue.
It isn't clear either that it's broken even if you cycle gluster (and
even though the gluster volume configuration says 'register-with-portmap
on'. Does the 'gluster volume set' command have the ability to get
success/fail information back from each node? It also appears that
'register-with-portmap' is applied to all volumes, even if you just
enable it on one - Is there a cluster-wide place to 'set' options?
[2012-03-05 19:51:22.517368] E
[rpcsvc.c:2771:nfs_rpcsvc_program_register_portmap] 0-nfsrpc: Could not
register with portmap
[2012-03-05 19:51:22.517420] E
[rpcsvc.c:2861:nfs_rpcsvc_program_register] 0-nfsrpc: portmap
registration of program failed
[2012-03-05 19:51:22.517428] E
[rpcsvc.c:2874:nfs_rpcsvc_program_register] 0-nfsrpc: Program
registration failed: MOUNT3, Num: 100005, Ver: 3, Port: 38465
David
On 3/5/12 2:05 PM, Bryan Whitehead wrote:
> Is selinux running? iptables?
>
> Can you http://pastie.org/ the nfs.log in /var/log/glusterfs ?
>
> On Mon, Mar 5, 2012 at 3:59 AM, David Coulson <david at davidcoulson.net
> <mailto:david at davidcoulson.net>> wrote:
>
> Yep.
>
> [root at dresproddns01 ~]# service glusterd stop
> Stopping glusterd: [ OK ]
>
> [root at dresproddns01 ~]# ps ax | grep nfs
> 120494 pts/0 S+ 0:00 grep nfs
>
> 2167119 ? S 0:00 [nfsiod]
> [root at dresproddns01 ~]# service rpcbind stop
> Stopping rpcbind: [ OK ]
>
> [root at dresproddns01 ~]# rpcinfo -p
> rpcinfo: can't contact portmapper: RPC: Remote system error - No
> such file or directory
> [root at dresproddns01 ~]# service rpcbind start
> Starting rpcbind: [ OK ]
> [root at dresproddns01 ~]# service glusterd start
> Starting glusterd: [ OK ]
>
> [root at dresproddns01 ~]# rpcinfo -p
> program vers proto port service
> 100000 4 tcp 111 portmapper
> 100000 3 tcp 111 portmapper
> 100000 2 tcp 111 portmapper
> 100000 4 udp 111 portmapper
> 100000 3 udp 111 portmapper
> 100000 2 udp 111 portmapper
>
> Note that I waited a short while between the last two steps. FYI,
> this is RHEL6 (the two systems that work are RHEL6 too, so I'm not
> sure it matters much).
>
>
> On 3/5/12 3:27 AM, Bryan Whitehead wrote:
>> did you start portmap service before you started gluster?
>>
>> On Sun, Mar 4, 2012 at 11:53 AM, David Coulson
>> <david at davidcoulson.net <mailto:david at davidcoulson.net>> wrote:
>>
>> I've four systems with multiple 4-way replica volumes. I'm
>> migrating a number of volumes from Fuse to NFS for
>> performance reasons.
>>
>> My first two hosts seem to work nicely, but the other two
>> won't start the NFS services properly. I looked through the
>> nfs.log, but it doesn't give any indication of why it did not
>> register with rpcbind. I'm presuming I've got a
>> misconfiguration on two of the systems, but there isn't a
>> clear indication of what is not working.
>>
>> Here is an example from a host which does not work:
>>
>> [root at dresproddns01 ~]# rpcinfo -p
>> program vers proto port service
>> 100000 4 tcp 111 portmapper
>> 100000 3 tcp 111 portmapper
>> 100000 2 tcp 111 portmapper
>> 100000 4 udp 111 portmapper
>> 100000 3 udp 111 portmapper
>> 100000 2 udp 111 portmapper
>> [root at dresproddns01 ~]# ps ax | grep nfs
>> 2167119 ? S 0:00 [nfsiod]
>> 2738268 ? Ssl 0:00
>> /opt/glusterfs/3.2.5/sbin/glusterfs -f
>> /etc/glusterd/nfs/nfs-server.vol -p
>> /etc/glusterd/nfs/run/nfs.pid -l /var/log/glusterfs/nfs.log
>> 2934228 pts/0 S+ 0:00 grep nfs
>> [root at dresproddns01 ~]# netstat -ntlp | grep 2738268
>> tcp 0 0 0.0.0.0:38465
>> <http://0.0.0.0:38465>
>> 0.0.0.0:* LISTEN 2738268/glusterfs
>> tcp 0 0 0.0.0.0:38466
>> <http://0.0.0.0:38466>
>> 0.0.0.0:* LISTEN 2738268/glusterfs
>> tcp 0 0 0.0.0.0:38467
>> <http://0.0.0.0:38467>
>> 0.0.0.0:* LISTEN 2738268/glusterfs
>>
>> [root at dresproddns01 ~]# gluster volume info svn
>>
>> Volume Name: svn
>> Type: Replicate
>> Status: Started
>> Number of Bricks: 4
>> Transport-type: tcp
>> Bricks:
>> Brick1: rhesproddns01:/gluster/svn
>> Brick2: rhesproddns02:/gluster/svn
>> Brick3: dresproddns01:/gluster/svn
>> Brick4: dresproddns02:/gluster/svn
>> Options Reconfigured:
>> performance.client-io-threads: 1
>> performance.flush-behind: on
>> network.ping-timeout: 5
>> performance.stat-prefetch: 1
>> nfs.disable: off
>> nfs.register-with-portmap: on
>> auth.allow: 10.250.53.*,10.252.248.*,169.254.*,127.0.0.1
>> performance.cache-size: 256Mb
>> performance.write-behind-window-size: 128Mb
>>
>> Only obvious difference with a host which does work is this:
>>
>> [root at rhesproddns01 named]# rpcinfo -p
>> program vers proto port service
>> 100000 4 tcp 111 portmapper
>> 100000 3 tcp 111 portmapper
>> 100000 2 tcp 111 portmapper
>> 100000 4 udp 111 portmapper
>> 100000 3 udp 111 portmapper
>> 100000 2 udp 111 portmapper
>> 100005 3 tcp 38465 mountd
>> 100005 1 tcp 38466 mountd
>> 100003 3 tcp 38467 nfs
>>
>>
>> Any ideas where to look for errors?
>>
>>
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org <mailto:Gluster-users at gluster.org>
>> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20120305/c214d338/attachment.html>
More information about the Gluster-users
mailing list