[Bugs] [Bug 1213304] New: nfs-ganesha: using features.enable command the nfs-ganesha process does come up on all four nodes
bugzilla at redhat.com
bugzilla at redhat.com
Mon Apr 20 10:01:17 UTC 2015
https://bugzilla.redhat.com/show_bug.cgi?id=1213304
Bug ID: 1213304
Summary: nfs-ganesha: using features.enable command the
nfs-ganesha process does come up on all four nodes
Product: GlusterFS
Version: 3.7.0
Component: nfs
Severity: high
Assignee: bugs at gluster.org
Reporter: saujain at redhat.com
CC: bugs at gluster.org, gluster-bugs at redhat.com
Description of problem:
gluster features.ganesha enable cli is used to set up the pcs cluster for
nfs-ganesha and bring up nfs-ganesha process.
So this time I am trying out the things with 4 node cluster of glusterfs.
All four nodes are suppose to be part of the nfs-ganesha cluster as well.
So effectively the four nodes in consideration should have nfs-ganesha process
post completion of the cli command, but nfs-ganesha does not come up on all
nodes every time.
Here are the logs of the issue seen from latest execution,
[root at nfs1 ~]# gluster features.ganesha enable
Enabling NFS-Ganesha requires Gluster-NFS to bedisabled across the trusted
pool. Do you still want to continue? (y/n) y
Error : Request timed out
node 1,
#####################################
[root at nfs1 ~]# ps -eaf | grep nfs
root 5338 6760 0 14:57 pts/0 00:00:00 grep nfs
[root at nfs1 ~]# pcs status
Cluster name: ganesha-ha-2
Last updated: Mon Apr 20 14:58:03 2015
Last change: Mon Apr 20 12:28:04 2015
Stack: cman
Current DC: nfs1 - partition with quorum
Version: 1.1.11-97629de
4 Nodes configured
22 Resources configured
Online: [ nfs1 nfs2 nfs3 nfs4 ]
Full list of resources:
Clone Set: nfs_start-clone [nfs_start]
nfs_start (ocf::heartbeat:ganesha_nfsd): FAILED nfs3 (unmanaged)
nfs_start (ocf::heartbeat:ganesha_nfsd): FAILED nfs1 (unmanaged)
nfs_start (ocf::heartbeat:ganesha_nfsd): FAILED nfs2 (unmanaged)
Stopped: [ nfs4 ]
nfs1-dead_ip-1 (ocf::heartbeat:Dummy): Started nfs4
Clone Set: nfs-mon-clone [nfs-mon]
Started: [ nfs1 nfs2 nfs3 nfs4 ]
Clone Set: nfs-grace-clone [nfs-grace]
Started: [ nfs1 nfs2 nfs3 nfs4 ]
nfs1-cluster_ip-1 (ocf::heartbeat:IPaddr): Started nfs3
nfs1-trigger_ip-1 (ocf::heartbeat:Dummy): Started nfs3
nfs2-cluster_ip-1 (ocf::heartbeat:IPaddr): Started nfs2
nfs2-trigger_ip-1 (ocf::heartbeat:Dummy): Started nfs2
nfs3-cluster_ip-1 (ocf::heartbeat:IPaddr): Started nfs3
nfs3-trigger_ip-1 (ocf::heartbeat:Dummy): Started nfs3
nfs4-cluster_ip-1 (ocf::heartbeat:IPaddr): Started nfs3
nfs4-trigger_ip-1 (ocf::heartbeat:Dummy): Started nfs3
nfs4-dead_ip-1 (ocf::heartbeat:Dummy): Started nfs1
Failed actions:
nfs_start_stop_0 on nfs3 'unknown error' (1): call=20, status=Timed Out,
last-rc-change='Mon Apr 20 12:27:09 2015', queued=0ms, exec=40002ms
nfs_start_stop_0 on nfs3 'unknown error' (1): call=20, status=Timed Out,
last-rc-change='Mon Apr 20 12:27:09 2015', queued=0ms, exec=40002ms
nfs_start_stop_0 on nfs1 'unknown error' (1): call=20, status=Timed Out,
last-rc-change='Mon Apr 20 12:27:09 2015', queued=0ms, exec=40001ms
nfs_start_stop_0 on nfs1 'unknown error' (1): call=20, status=Timed Out,
last-rc-change='Mon Apr 20 12:27:09 2015', queued=0ms, exec=40001ms
nfs_start_stop_0 on nfs2 'unknown error' (1): call=20, status=Timed Out,
last-rc-change='Mon Apr 20 12:27:09 2015', queued=0ms, exec=40002ms
nfs_start_stop_0 on nfs2 'unknown error' (1): call=20, status=Timed Out,
last-rc-change='Mon Apr 20 12:27:09 2015', queued=0ms, exec=40002ms
node 2,
##########################################
[root at nfs2 ~]# ps -eaf | grep nfs
root 5260 16826 0 14:58 pts/0 00:00:00 grep nfs
root 6216 1 0 12:27 ? 00:00:05 /usr/bin/ganesha.nfsd -L
/var/log/ganesha.log -f /etc/ganesha/ganesha.conf -N NIV_EVENT -p
/var/run/ganesha.nfsd.pid
[root at nfs2 ~]# pcs status
Cluster name: ganesha-ha-2
Last updated: Mon Apr 20 14:58:49 2015
Last change: Mon Apr 20 12:28:04 2015
Stack: cman
Current DC: nfs1 - partition with quorum
Version: 1.1.11-97629de
4 Nodes configured
22 Resources configured
Online: [ nfs1 nfs2 nfs3 nfs4 ]
Full list of resources:
Clone Set: nfs_start-clone [nfs_start]
nfs_start (ocf::heartbeat:ganesha_nfsd): FAILED nfs3 (unmanaged)
nfs_start (ocf::heartbeat:ganesha_nfsd): FAILED nfs1 (unmanaged)
nfs_start (ocf::heartbeat:ganesha_nfsd): FAILED nfs2 (unmanaged)
Stopped: [ nfs4 ]
nfs1-dead_ip-1 (ocf::heartbeat:Dummy): Started nfs4
Clone Set: nfs-mon-clone [nfs-mon]
Started: [ nfs1 nfs2 nfs3 nfs4 ]
Clone Set: nfs-grace-clone [nfs-grace]
Started: [ nfs1 nfs2 nfs3 nfs4 ]
nfs1-cluster_ip-1 (ocf::heartbeat:IPaddr): Started nfs3
nfs1-trigger_ip-1 (ocf::heartbeat:Dummy): Started nfs3
nfs2-cluster_ip-1 (ocf::heartbeat:IPaddr): Started nfs2
nfs2-trigger_ip-1 (ocf::heartbeat:Dummy): Started nfs2
nfs3-cluster_ip-1 (ocf::heartbeat:IPaddr): Started nfs3
nfs3-trigger_ip-1 (ocf::heartbeat:Dummy): Started nfs3
nfs4-cluster_ip-1 (ocf::heartbeat:IPaddr): Started nfs3
nfs4-trigger_ip-1 (ocf::heartbeat:Dummy): Started nfs3
nfs4-dead_ip-1 (ocf::heartbeat:Dummy): Started nfs1
Failed actions:
nfs_start_stop_0 on nfs3 'unknown error' (1): call=20, status=Timed Out,
last-rc-change='Mon Apr 20 12:27:09 2015', queued=0ms, exec=40002ms
nfs_start_stop_0 on nfs3 'unknown error' (1): call=20, status=Timed Out,
last-rc-change='Mon Apr 20 12:27:09 2015', queued=0ms, exec=40002ms
nfs_start_stop_0 on nfs1 'unknown error' (1): call=20, status=Timed Out,
last-rc-change='Mon Apr 20 12:27:09 2015', queued=0ms, exec=40001ms
nfs_start_stop_0 on nfs1 'unknown error' (1): call=20, status=Timed Out,
last-rc-change='Mon Apr 20 12:27:09 2015', queued=0ms, exec=40001ms
nfs_start_stop_0 on nfs2 'unknown error' (1): call=20, status=Timed Out,
last-rc-change='Mon Apr 20 12:27:09 2015', queued=0ms, exec=40002ms
nfs_start_stop_0 on nfs2 'unknown error' (1): call=20, status=Timed Out,
last-rc-change='Mon Apr 20 12:27:09 2015', queued=0ms, exec=40002ms
node 3,
#############################################
[root at nfs3 ~]# ps -eaf | grep nfs
root 20901 18085 0 14:59 pts/0 00:00:00 grep nfs
root 26369 1 0 12:27 ? 00:00:05 /usr/bin/ganesha.nfsd -L
/var/log/ganesha.log -f /etc/ganesha/ganesha.conf -N NIV_EVENT -p
/var/run/ganesha.nfsd.pid
[root at nfs3 ~]# pcs status
Cluster name: ganesha-ha-2
Last updated: Mon Apr 20 14:59:22 2015
Last change: Mon Apr 20 12:28:04 2015
Stack: cman
Current DC: nfs1 - partition with quorum
Version: 1.1.11-97629de
4 Nodes configured
22 Resources configured
Online: [ nfs1 nfs2 nfs3 nfs4 ]
Full list of resources:
Clone Set: nfs_start-clone [nfs_start]
nfs_start (ocf::heartbeat:ganesha_nfsd): FAILED nfs3 (unmanaged)
nfs_start (ocf::heartbeat:ganesha_nfsd): FAILED nfs1 (unmanaged)
nfs_start (ocf::heartbeat:ganesha_nfsd): FAILED nfs2 (unmanaged)
Stopped: [ nfs4 ]
nfs1-dead_ip-1 (ocf::heartbeat:Dummy): Started nfs4
Clone Set: nfs-mon-clone [nfs-mon]
Started: [ nfs1 nfs2 nfs3 nfs4 ]
Clone Set: nfs-grace-clone [nfs-grace]
Started: [ nfs1 nfs2 nfs3 nfs4 ]
nfs1-cluster_ip-1 (ocf::heartbeat:IPaddr): Started nfs3
nfs1-trigger_ip-1 (ocf::heartbeat:Dummy): Started nfs3
nfs2-cluster_ip-1 (ocf::heartbeat:IPaddr): Started nfs2
nfs2-trigger_ip-1 (ocf::heartbeat:Dummy): Started nfs2
nfs3-cluster_ip-1 (ocf::heartbeat:IPaddr): Started nfs3
nfs3-trigger_ip-1 (ocf::heartbeat:Dummy): Started nfs3
nfs4-cluster_ip-1 (ocf::heartbeat:IPaddr): Started nfs3
nfs4-trigger_ip-1 (ocf::heartbeat:Dummy): Started nfs3
nfs4-dead_ip-1 (ocf::heartbeat:Dummy): Started nfs1
Failed actions:
nfs_start_stop_0 on nfs3 'unknown error' (1): call=20, status=Timed Out,
last-rc-change='Mon Apr 20 12:27:09 2015', queued=0ms, exec=40002ms
nfs_start_stop_0 on nfs3 'unknown error' (1): call=20, status=Timed Out,
last-rc-change='Mon Apr 20 12:27:09 2015', queued=0ms, exec=40002ms
nfs_start_stop_0 on nfs1 'unknown error' (1): call=20, status=Timed Out,
last-rc-change='Mon Apr 20 12:27:09 2015', queued=0ms, exec=40001ms
nfs_start_stop_0 on nfs1 'unknown error' (1): call=20, status=Timed Out,
last-rc-change='Mon Apr 20 12:27:09 2015', queued=0ms, exec=40001ms
nfs_start_stop_0 on nfs2 'unknown error' (1): call=20, status=Timed Out,
last-rc-change='Mon Apr 20 12:27:09 2015', queued=0ms, exec=40002ms
nfs_start_stop_0 on nfs2 'unknown error' (1): call=20, status=Timed Out,
last-rc-change='Mon Apr 20 12:27:09 2015', queued=0ms, exec=40002ms
node 4,
######################################
[root at nfs4 ~]# ps -eaf | grep nfs
root 16073 27004 0 04:12 pts/0 00:00:00 grep nfs
[root at nfs4 ~]# pcs status
Cluster name: ganesha-ha-2
Last updated: Mon Apr 20 04:13:00 2015
Last change: Mon Apr 20 01:41:11 2015
Stack: cman
Current DC: nfs1 - partition with quorum
Version: 1.1.11-97629de
4 Nodes configured
22 Resources configured
Online: [ nfs1 nfs2 nfs3 nfs4 ]
Full list of resources:
Clone Set: nfs_start-clone [nfs_start]
nfs_start (ocf::heartbeat:ganesha_nfsd): FAILED nfs3 (unmanaged)
nfs_start (ocf::heartbeat:ganesha_nfsd): FAILED nfs1 (unmanaged)
nfs_start (ocf::heartbeat:ganesha_nfsd): FAILED nfs2 (unmanaged)
Stopped: [ nfs4 ]
nfs1-dead_ip-1 (ocf::heartbeat:Dummy): Started nfs4
Clone Set: nfs-mon-clone [nfs-mon]
Started: [ nfs1 nfs2 nfs3 nfs4 ]
Clone Set: nfs-grace-clone [nfs-grace]
Started: [ nfs1 nfs2 nfs3 nfs4 ]
nfs1-cluster_ip-1 (ocf::heartbeat:IPaddr): Started nfs3
nfs1-trigger_ip-1 (ocf::heartbeat:Dummy): Started nfs3
nfs2-cluster_ip-1 (ocf::heartbeat:IPaddr): Started nfs2
nfs2-trigger_ip-1 (ocf::heartbeat:Dummy): Started nfs2
nfs3-cluster_ip-1 (ocf::heartbeat:IPaddr): Started nfs3
nfs3-trigger_ip-1 (ocf::heartbeat:Dummy): Started nfs3
nfs4-cluster_ip-1 (ocf::heartbeat:IPaddr): Started nfs3
nfs4-trigger_ip-1 (ocf::heartbeat:Dummy): Started nfs3
nfs4-dead_ip-1 (ocf::heartbeat:Dummy): Started nfs1
Failed actions:
nfs_start_stop_0 on nfs3 'unknown error' (1): call=20, status=Timed Out,
last-rc-change='Mon Apr 20 12:27:09 2015', queued=0ms, exec=40002ms
nfs_start_stop_0 on nfs3 'unknown error' (1): call=20, status=Timed Out,
last-rc-change='Mon Apr 20 12:27:09 2015', queued=0ms, exec=40002ms
nfs_start_stop_0 on nfs1 'unknown error' (1): call=20, status=Timed Out,
last-rc-change='Mon Apr 20 12:27:09 2015', queued=0ms, exec=40001ms
nfs_start_stop_0 on nfs1 'unknown error' (1): call=20, status=Timed Out,
last-rc-change='Mon Apr 20 12:27:09 2015', queued=0ms, exec=40001ms
nfs_start_stop_0 on nfs2 'unknown error' (1): call=20, status=Timed Out,
last-rc-change='Mon Apr 20 12:27:09 2015', queued=0ms, exec=40002ms
nfs_start_stop_0 on nfs2 'unknown error' (1): call=20, status=Timed Out,
last-rc-change='Mon Apr 20 12:27:09 2015', queued=0ms, exec=40002ms
Version-Release number of selected component (if applicable):
glusterfs-3.7dev-0.1017.git7fb85e3.el6.x86_64
nfs-ganesha-2.2-0.rc8.el6.x86_64
How reproducible:
most of the times.
Expected results:
nfs-ganesha is suppose to come up on all nodes.
Additional info:
--
You are receiving this mail because:
You are on the CC list for the bug.
You are the assignee for the bug.
More information about the Bugs
mailing list