[Gluster-users] nfs-ganesha volume null errors
Soumya Koduri
skoduri at redhat.com
Tue Mar 22 13:18:09 UTC 2016
On 03/21/2016 02:16 AM, ML Wong wrote:
> Hello Soumya,
> Thanks for answering my questions.
> Question 1) I am still puzzled what VOL is still referring to. Is that a
> variable/parameter that i can specify somewhere in the ganesha-ha.conf?
> Any pointers will be very much appreciated.
No it doesn't refer to any volume as it is a global option. The log
message is misleading.
>
> 1) Those 3 test systems do not have firewalld running and SELinux
> running. And i also verify corosync.conf is now empty.
> # sestatus
> SELinux status: disabled
>
> # firewall-cmd --zone=public --list-all
> FirewallD is not running
>
> # ls -al /etc/corosync/corosync.conf
> -rw-r--r-- 1 root root 0 Mar 20 12:54 /etc/corosync/corosync.conf
>
> 2) I also do not find pacemaker.log under /var/log, but i found the
> following. Will these be the same:
> # ls -al /var/log/pcsd/pcsd.log
> -rw-r--r--. 1 root root 162322 Mar 20 13:26 /var/log/pcsd/pcsd.log
>
> In any case, that log is full of the following:
> +++
> I, [2016-03-20T13:33:34.982311 #939] INFO -- : Running:
> /usr/sbin/corosync-cmapctl totem.cluster_name
> I, [2016-03-20T13:33:34.982459 #939] INFO -- : CIB USER: hacluster,
> groups:
> I, [2016-03-20T13:33:34.985984 #939] INFO -- : Return Value: 1
> +++
There should be pacemaker.log too.. Which version of pacemaker are you
using?
>
> 3) /var/log/messages - it does not look ganesha passing the logs to this
> file. But i see /var/log/ganesha.log - which i found out logging seem
> to be sent to there from /etc/sysconfig/ganesha (OPTIONS="-L
> /var/log/ganesha.log -f /etc/ganesha/ganesha.conf -N NIV_FULL_DEBUG""
>
> After it failed to acquire the volume, the server will be filled with
> the following in "ganesha.log", but the other 2 nodes in the cluster do
> not have anything logged in ganesha.log. The other nodes have "E
> [MSGID: 106062] [glusterd-op-sm.c:3728:glusterd_op_ac_unlock]
> 0-management: Unable to acquire volname" logged in the
> "etc-glusterfs-glusterd.vol.log"
> +++
> 20/03/2016 13:37:32 : epoch 56ef059d : mlw-fusion1 :
> ganesha.nfsd-5215[dbus_heartbeat] gsh_dbus_thread :DBUS :F_DBG :top of
> poll loop
> 20/03/2016 13:37:32 : epoch 56ef059d : mlw-fusion1 :
> ganesha.nfsd-5215[dbus_heartbeat] gsh_dbus_thread :RW LOCK :F_DBG
> :Acquired mutex 0x7fd38e3fe080 (&dbus_bcast_lock) at
> /builddir/build/BUILD/nfs-ganesha-2.3.0/src/dbus/dbus_server.c:689
> 20/03/2016 13:37:32 : epoch 56ef059d : mlw-fusion1 :
> ganesha.nfsd-5215[dbus_heartbeat] gsh_dbus_thread :RW LOCK :F_DBG
> :Released mutex 0x7fd38e3fe080 (&dbus_bcast_lock) at
> /builddir/build/BUILD/nfs-ganesha-2.3.0/src/dbus/dbus_server.c:739
> +++
>
'/var/log/messages' is where cluster-setup related errors are logged.
Probably to debug you could try below steps -
* bring up nfs-ganesha server on all the nodes
#systemctl start nfs-ganesha
* Check if nfs-ganesha is successfully started
* On one of the nodes,
# cd '/usr/libexec/ganesha'
# bash -x ./ganesha.sh --setup /etc/ganesha
This would throw the errors returned by the script on the console during
cluster setup.
Please give a try and let me know if you see any errors.
Thanks,
Soumya
> Testing Environment: Running CentOS Linux release 7.2.1511, glusterfs
> 3.7.8 (glusterfs-server-3.7.8-2.el7.x86_64),
> nfs-ganesha-gluster-2.3.0-1.el7.x86_64
>
>
> On Mon, Mar 14, 2016 at 2:05 AM, Soumya Koduri <skoduri at redhat.com
> <mailto:skoduri at redhat.com>> wrote:
>
> Hi,
>
>
> On 03/14/2016 04:06 AM, ML Wong wrote:
>
> Running CentOS Linux release 7.2.1511, glusterfs 3.7.8
> (glusterfs-server-3.7.8-2.el7.x86_64),
> nfs-ganesha-gluster-2.3.0-1.el7.x86_64
>
> 1) Ensured the connectivity between gluster nodes by using PING
> 2) Disabled NetworkManager (Loaded: loaded
> (/usr/lib/systemd/system/NetworkManager.service; disabled)
> 3) Gluster 'gluster_shared_storage' is created by using (gluster
> volume
> set all cluster.enable-shared-storage enable), and are all
> mounted under
> /run/gluster/shared_storage, and nfs-ganesha directory is also
> created
> after the feature being enabled
> 4) Emtpy out /etc/ganesha/ganesha.conf (have tested ganesha
> running as a
> stand-alone NFS server)
> 5) Installed pacemaker, corosync, and resource-agents
> 6) Reset 'hacluster' system-user password to be the same:
> # pcs cluster auth -u hacluster mlw-fusion1
> mlw-fusion2 mlw-fusion3
> Password:
> mlw-fusion2: Authorized
> mlw-fusion3: Authorized
> mlw-fusion1: Authorized
> 7) IPv6 is enabled - (IPV6INIT=yes in
> /etc/sysconfig/network-scripts/ifcfg-en*)
> 8) Started pcsd, and corosync
> 9) Created /var/lib/glusterd/nfs/secret.pem, and transfer to the
> other 2
> nodes
> # ssh -i secret.pem root at mlw-fusion3 "echo helloworld"
> helloworld
> 9) Transfer the following ganesha-ha.conf to the other nodes in the
> cluster, but change the HA_VOL_SERVER value accordingly to
> mlw-fusion2,
> and mlw-fusion3
>
> HA_NAME="ganesha-ha-01"
> HA_VOL_SERVER="mlw-fusion1"
> HA_CLUSTER_NODES="mlw-fusion1,mlw-fusion2,mlw-fusion3"
> VIP_mlw_fusion1="192.168.30.201"
> VIP_mlw_fusion2="192.168.30.202"
> VIP_mlw_fusion3="192.168.30.203"
>
> Question 1) As i am new to nfs-ganesha, pacemaker, corosync, i was
> mostly puzzled by the error message found in the
> 'etc-glusterfs-glusterd.vol.log'. It seems like it will show the
> below
> message regardless of what i have done to troubleshoot - So,
> what Volume
> does these error messages are referring to?
>
>
> Just a guess. Since this option is not tied to any particular
> volume, it may have thrown (null) in the error message. Could you
> check '/var/log/messages' and '/var/log/pacemaker.log' for the
> errors/warnings. Since they are running RHEL 7, please check if
> there are denials from selinux or firewalld.
>
> Is that referring to the
>
> HA_VOL_NAME in the /usr/libexec/ganesha/ganesha-ha.sh? Do I need to
> change any of the 4 HA_* variables inside ganesha-ha.sh?
> HA_NUM_SERVERS=0
> HA_SERVERS=""
> HA_CONFDIR="/etc/ganesha"
> HA_VOL_NAME="gluster_shared_storage"
> HA_VOL_MNT="/run/gluster/shared_storage"
>
> No. You need not change any of these variables.
>
>
> E [MSGID: 106123] [glusterd-syncop.c:1407:gd_commit_op_phase]
> 0-management: Commit of operation 'Volume (null)' failed on
> localhost :
> Failed to set up HA config for NFS-Ganesha. Please check the log
> file
> for details
>
> Question 2) do I really have to start corosync before enabling
> nfs-ganesha?
>
> No. The setup automatically starts pacemaker and corosync services.
>
> Thanks,
> Soumya
>
>
> Any help will be appreciated!!!
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org <mailto:Gluster-users at gluster.org>
> http://www.gluster.org/mailman/listinfo/gluster-users
>
>
More information about the Gluster-users
mailing list