[Bugs] [Bug 1376874] New: RFE : move ganesha related configuration into shared storage
bugzilla at redhat.com
bugzilla at redhat.com
Fri Sep 16 17:41:55 UTC 2016
https://bugzilla.redhat.com/show_bug.cgi?id=1376874
Bug ID: 1376874
Summary: RFE : move ganesha related configuration into shared
storage
Product: GlusterFS
Version: 3.9
Component: ganesha-nfs
Keywords: Triaged
Assignee: jthottan at redhat.com
Reporter: jthottan at redhat.com
CC: bugs at gluster.org, jthottan at redhat.com,
kkeithle at redhat.com, ndevos at redhat.com,
skoduri at redhat.com, sraj at redhat.com
Depends On: 1355956
Blocks: 1261866
A bug fixes for above feature
+++ This bug was initially created as a clone of Bug #1355956 +++
Description of problem:
In the present design, we maintain a copy of the below database (required for
NFS-Ganesha setup) on all the nodes of the Gluster Storage pool -
* /etc/ganesha/ganesha-ha.conf (with the details about the nodes forming
NFS-Ganesha cluster and their VIPs to be configured)
* /etc/ganesha/ganesha.conf (contains configuration parameters and the volumes
list to be exported, used by NFS-Ganesha process while starting)
* /etc/ganesha/exports/ganesha.$vol.conf (config file of a volume created when
a volume is being exported by NFS-Ganesha)
* /etc/ganesha/exports/.export_added (each volume should have unique ExportID.
This file is to keep track of the last ExportID used)
Outstanding Issues:
Since each node shall have a local copy, we have few issues with respect to
synchronizing them across the nodes. Few of them are outlined below:
* When a node reboots, depending on the order the services ‘pacemaker’ and
‘glusterd’ starts, we may run into a case, where in glusterd couldn’t sync the
export files on this node.
* There could be case where in ‘.export_added’ could go out of sync across the
nodes thus resulting in different ExportIDs for the same volume exported across
different nfs-ganesha heads. This shall result in NFS mounts throwing “Stale
File handle error” post failover
* When a node is down while performing “refresh-config”, the export config of
that volume doesn’t get synced when it comes back up.
* As one of the pre-requisites, ganesha-ha.conf has to be copied to all the
nodes in the gluster cluster before enabling nfs-ganesha (setup).
* When any changes are made to main “/etc/ganesha.conf” file, it has to be
manually copied to all the nodes in the cluster.
* When “refresh-config” is performed, the script syncs up the volume config
only across the NFS-Ganesha cluster but not to all the nodes in the Gluster
storage pool
Version-Release number of selected component (if applicable):
mainline
Additional info:
The existing users shall be impacted while doing upgrade. We may need to
provide scripts for the smooth upgrade to move the older config data to
shared_storage and also few of the setup pre-requisites need to be
removed/modified accordingly.
--- Additional comment from Vijay Bellur on 2016-07-13 05:55:25 EDT ---
REVIEW: http://review.gluster.org/14906 (glusterd/ganesha : Move
ganesha-ha.conf and ganesha.conf to shared storage) posted (#1) for review on
master by jiffin tony Thottan (jthottan at redhat.com)
--- Additional comment from Vijay Bellur on 2016-07-13 05:55:28 EDT ---
REVIEW: http://review.gluster.org/14907 (Revert "glusterd-ganesha : copy
ganesha export configuration files during reboot") posted (#1) for review on
master by jiffin tony Thottan (jthottan at redhat.com)
--- Additional comment from Vijay Bellur on 2016-07-13 05:55:31 EDT ---
REVIEW: http://review.gluster.org/14908 (glusterd/ganesha : create export
configuration file in shared storage) posted (#1) for review on master by
jiffin tony Thottan (jthottan at redhat.com)
--- Additional comment from Vijay Bellur on 2016-07-13 05:55:35 EDT ---
REVIEW: http://review.gluster.org/14909 (ganesha/scripts : Modifying
ganesha-ha.sh for share storage related changes) posted (#1) for review on
master by jiffin tony Thottan (jthottan at redhat.com)
--- Additional comment from Vijay Bellur on 2016-07-15 05:24:44 EDT ---
REVIEW: http://review.gluster.org/14908 (glusterd/ganesha : create export
configuration file in shared storage) posted (#2) for review on master by
jiffin tony Thottan (jthottan at redhat.com)
--- Additional comment from Vijay Bellur on 2016-07-15 05:24:47 EDT ---
REVIEW: http://review.gluster.org/14909 (ganesha/scripts : Modifying
ganesha-ha.sh for share storage related changes) posted (#2) for review on
master by jiffin tony Thottan (jthottan at redhat.com)
--- Additional comment from Vijay Bellur on 2016-07-15 05:24:50 EDT ---
REVIEW: http://review.gluster.org/14906 (glusterd/ganesha : Move
ganesha-ha.conf and ganesha.conf to shared storage) posted (#2) for review on
master by jiffin tony Thottan (jthottan at redhat.com)
--- Additional comment from Vijay Bellur on 2016-07-15 05:24:54 EDT ---
REVIEW: http://review.gluster.org/14907 (Revert "glusterd-ganesha : copy
ganesha export configuration files during reboot") posted (#2) for review on
master by jiffin tony Thottan (jthottan at redhat.com)
--- Additional comment from Vijay Bellur on 2016-08-08 05:25:14 EDT ---
REVIEW: http://review.gluster.org/14908 (glusterd/ganesha : create export
configuration file in shared storage) posted (#3) for review on master by
jiffin tony Thottan (jthottan at redhat.com)
--- Additional comment from Vijay Bellur on 2016-08-08 05:25:17 EDT ---
REVIEW: http://review.gluster.org/14909 (ganesha/scripts : Modifying
ganesha-ha.sh for share storage related changes) posted (#3) for review on
master by jiffin tony Thottan (jthottan at redhat.com)
--- Additional comment from Vijay Bellur on 2016-08-08 05:25:22 EDT ---
REVIEW: http://review.gluster.org/14906 (glusterd/ganesha : Move
ganesha-ha.conf and ganesha.conf to shared storage) posted (#3) for review on
master by jiffin tony Thottan (jthottan at redhat.com)
--- Additional comment from Vijay Bellur on 2016-08-08 05:25:26 EDT ---
REVIEW: http://review.gluster.org/14907 (Revert "glusterd-ganesha : copy
ganesha export configuration files during reboot") posted (#3) for review on
master by jiffin tony Thottan (jthottan at redhat.com)
--- Additional comment from Worker Ant on 2016-08-25 04:37:43 EDT ---
REVIEW: http://review.gluster.org/14908 (glusterd/ganesha : create export
configuration file in shared storage) posted (#4) for review on master by
jiffin tony Thottan (jthottan at redhat.com)
--- Additional comment from Worker Ant on 2016-08-25 04:37:47 EDT ---
REVIEW: http://review.gluster.org/14909 (ganesha/scripts : Modifying
ganesha-ha.sh for share storage related changes) posted (#4) for review on
master by jiffin tony Thottan (jthottan at redhat.com)
--- Additional comment from Worker Ant on 2016-08-25 04:37:51 EDT ---
REVIEW: http://review.gluster.org/14906 (glusterd/ganesha : Move
ganesha-ha.conf and ganesha.conf to shared storage) posted (#4) for review on
master by jiffin tony Thottan (jthottan at redhat.com)
--- Additional comment from Worker Ant on 2016-08-25 04:37:55 EDT ---
REVIEW: http://review.gluster.org/14907 (Revert "glusterd-ganesha : copy
ganesha export configuration files during reboot") posted (#4) for review on
master by jiffin tony Thottan (jthottan at redhat.com)
--- Additional comment from Worker Ant on 2016-08-25 09:56:01 EDT ---
COMMIT: http://review.gluster.org/14906 committed in master by Kaleb KEITHLEY
(kkeithle at redhat.com)
------
commit 76726da0e86077a8f3a59c02a47fcf2e3994218f
Author: Jiffin Tony Thottan <jthottan at redhat.com>
Date: Tue Jul 12 15:44:23 2016 +0530
glusterd/ganesha : Move ganesha-ha.conf and ganesha.conf to shared storage
Currently all the ganesha related configuration files(ganesha.conf,
ganesha-ha.conf, export files, etc) is stored locally at /etc/ganesha
on a every node in ganesha cluster. Usually we end up in two issues by
doing so :
* difficult in modifiying ganesha related conf file
* diffciult to maintain consistency of conf file across ganesha cluster
To tackle this, we plan to move all the ganesha configuration to shared
storage. As a first step in this patch ganesha.conf and ganesha-ha.conf
move to shared storage. Here actual ganesha.conf will resides in shared
stoarge and symlinks will be created in /etc/ganesha when the option
"gluster nfs-ganesha enable" is executed and remove those during the
"disable" part.
Modified prerequisites to done before running globaloption:
* enable shared storage
* create nfs-ganesha folder in shared storage
* create ganesha.conf and ganesha-ha.conf in it
More details can be found at http://review.gluster.org/#/c/15105/
Change-Id: Ifabb6c5db50061f077a03932940190af74e2ca7f
BUG: 1355956
Signed-off-by: Jiffin Tony Thottan <jthottan at redhat.com>
Reviewed-on: http://review.gluster.org/14906
Reviewed-by: soumya k <skoduri at redhat.com>
NetBSD-regression: NetBSD Build System <jenkins at build.gluster.org>
Smoke: Gluster Build System <jenkins at build.gluster.org>
Reviewed-by: Kaleb KEITHLEY <kkeithle at redhat.com>
CentOS-regression: Gluster Build System <jenkins at build.gluster.org>
--- Additional comment from Worker Ant on 2016-08-26 07:46:10 EDT ---
COMMIT: http://review.gluster.org/14907 committed in master by Kaleb KEITHLEY
(kkeithle at redhat.com)
------
commit a769be2c0d4227fead7348e769bd1ae49e267971
Author: Jiffin Tony Thottan <jthottan at redhat.com>
Date: Tue Jul 12 17:23:03 2016 +0530
Revert "glusterd-ganesha : copy ganesha export configuration files during
reboot"
This reverts commit f71e2fa49af185779b9f43e146effd122d4e9da0.
Reason:
As part of sync up node reboot this patch copies ganesha export conf file
from a source node. This change is no more require if the export files are
available in shared storage.
Change-Id: Id9c1ae78377bbd7d5d80aa1c14f534e30feaae97
BUG: 1355956
Signed-off-by: Jiffin Tony Thottan <jthottan at redhat.com>
Reviewed-on: http://review.gluster.org/14907
Reviewed-by: soumya k <skoduri at redhat.com>
Smoke: Gluster Build System <jenkins at build.gluster.org>
CentOS-regression: Gluster Build System <jenkins at build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins at build.gluster.org>
Reviewed-by: Kaleb KEITHLEY <kkeithle at redhat.com>
--- Additional comment from Worker Ant on 2016-08-26 07:47:22 EDT ---
COMMIT: http://review.gluster.org/14909 committed in master by Kaleb KEITHLEY
(kkeithle at redhat.com)
------
commit 2f44a979132a43c1cf12ed2b9e9e94dbe1311c9e
Author: Jiffin Tony Thottan <jthottan at redhat.com>
Date: Wed Jul 13 12:09:43 2016 +0530
ganesha/scripts : Modifying ganesha-ha.sh for share storage related changes
Currently the ganesha related configurations are "scp"ied for operations
like
add, delete, refresh-config in ganesha-ha.sh. This is no more required
since
all the conf files are available in shared storage and every node can
directly
access them from shared storage.
More details can be found at http://review.gluster.org/#/c/15105/
Change-Id: Ic025eb4dc246db61d6fbe969ca60214751fbf3ba
BUG: 1355956
Signed-off-by: Jiffin Tony Thottan <jthottan at redhat.com>
Reviewed-on: http://review.gluster.org/14909
NetBSD-regression: NetBSD Build System <jenkins at build.gluster.org>
CentOS-regression: Gluster Build System <jenkins at build.gluster.org>
Reviewed-by: soumya k <skoduri at redhat.com>
Smoke: Gluster Build System <jenkins at build.gluster.org>
Reviewed-by: Kaleb KEITHLEY <kkeithle at redhat.com>
--- Additional comment from Worker Ant on 2016-08-26 07:47:55 EDT ---
COMMIT: http://review.gluster.org/14908 committed in master by Kaleb KEITHLEY
(kkeithle at redhat.com)
------
commit 4786150ebe36154ec0762a951612f66f68a7e95c
Author: Jiffin Tony Thottan <jthottan at redhat.com>
Date: Wed Jul 13 11:38:10 2016 +0530
glusterd/ganesha : create export configuration file in shared storage
This is the second patch which moves export related configuration for
a volume into shared storage. The main change includes in scripts
create-export-ganesha.sh, dbus-send.sh and the handling of volume set
command "ganesha.enable". The manipulation of EXPORT_ID move from
dbus-send.sh to create-export-ganesha.sh.
In volume set handling following has performed
stage | commit
----------------------------------------------------------
1.) gluster v set <volname> ganesha.enable on
None | create export file
| in node where cli executed,
| thne export volume via dbus
2.) gluster v set <volname> ganesha.enable off
unexport volume via dbus | remove export file from the
| shared storage
-----------------------------------------------------------
More details can be found at http://review.gluster.org/#/c/15105/
Change-Id: Ia8b0e89bc8fff24b0bc5d20a538a89212894a8e4
BUG: 1355956
Signed-off-by: Jiffin Tony Thottan <jthottan at redhat.com>
Reviewed-on: http://review.gluster.org/14908
NetBSD-regression: NetBSD Build System <jenkins at build.gluster.org>
CentOS-regression: Gluster Build System <jenkins at build.gluster.org>
Reviewed-by: soumya k <skoduri at redhat.com>
Smoke: Gluster Build System <jenkins at build.gluster.org>
Reviewed-by: Kaleb KEITHLEY <kkeithle at redhat.com>
--- Additional comment from Worker Ant on 2016-08-30 09:11:32 EDT ---
REVIEW: http://review.gluster.org/15355 (ganesha/glustrd : Correct the path for
ganesha conf dir in Makefile) posted (#1) for review on master by jiffin tony
Thottan (jthottan at redhat.com)
--- Additional comment from Worker Ant on 2016-09-16 01:45:31 EDT ---
REVIEW: http://review.gluster.org/15355 (ganesha/glusterd : Correct the path
for ganesha conf dir in Makefile) posted (#2) for review on master by jiffin
tony Thottan (jthottan at redhat.com)
--- Additional comment from Worker Ant on 2016-09-16 10:22:54 EDT ---
COMMIT: http://review.gluster.org/15355 committed in master by Atin Mukherjee
(amukherj at redhat.com)
------
commit 7e18e16f9f62ed95acded0fb3f2a8784087f59c0
Author: Jiffin Tony Thottan <jthottan at redhat.com>
Date: Tue Aug 30 18:36:51 2016 +0530
ganesha/glusterd : Correct the path for ganesha conf dir in Makefile
The value for ganesha conf dir in Makefile.am is mistakenly entered as
following :
DCONFDIR=\"/$(runstatedir)/gluster/shared_storage/nfs-ganesha\"
Here value for runstatedir is seems to "NULL" which results wrong path
for ganesha configuration directory
Change-Id: I0b7ebd8e2503de0cb79b601553c4405d0d1fd711
BUG: 1355956
Signed-off-by: Jiffin Tony Thottan <jthottan at redhat.com>
Reviewed-on: http://review.gluster.org/15355
NetBSD-regression: NetBSD Build System <jenkins at build.gluster.org>
Reviewed-by: soumya k <skoduri at redhat.com>
Reviewed-by: Niels de Vos <ndevos at redhat.com>
CentOS-regression: Gluster Build System <jenkins at build.gluster.org>
Smoke: Gluster Build System <jenkins at build.gluster.org>
Referenced Bugs:
https://bugzilla.redhat.com/show_bug.cgi?id=1261866
[Bug 1261866] HA : All the nodes in HA-cluster should be up to perform any
of ganesha-ha.sh related operations
https://bugzilla.redhat.com/show_bug.cgi?id=1355956
[Bug 1355956] RFE : move ganesha related configuration into shared storage
--
You are receiving this mail because:
You are on the CC list for the bug.
Unsubscribe from this bug https://bugzilla.redhat.com/token.cgi?t=x1ZloWvCOH&a=cc_unsubscribe
More information about the Bugs
mailing list