[Gluster-users] NFS REEXPORT FAIL IN NFS SIDE

[Martin Gumbau]

Hi, I have the next escenario:

GLUSTERFS SERVER
sirius00(glusterfs server) + sirius01(glusterfs server)
(identical SO, HW, configuration + glusterfs-1.3.12 + fuse of SuSE SLES
10 SP2)
	|
	|
	|

REEXPORTER GLUSTERFS-NFS
sirius99(glusterfs client-afr + nfs server)
(glusterfs-1.3.12 + fuse-2.7.3glfs10)
	|
	|
	|
CLIENT NFS
sirius199(nfs client)
***********************************************************************
PROBLEM: when I mount nfs-client reexported glusterfs-client_nfs-server
volumns all the directories in nfs-client shown the same information.
But in reexpoter machine I can acces to differents volumns without
problems mounted with glusterfs.
***********************************************************************

I have proved two OPTIONS with the same result
##################################################################
OPTION-1 (1 Glusterfs server – 2 glusterfs clients + 2 nfsexports – 2
nfsimports)
##################################################################

GLUSTERFS SERVER SIDE (in two servers the same config files)
### Command to leave up the gluster server 
$ glusterfs -f /etc/glusterfs/glusterfs-server.vol
--------------------------------------------------------------------
# file: /etc/glusterfs/glusterfs-server.vol
volume kimik
	type storage/posix
	option directory /cfs/kimik/
end-volume

volume tekla
	type storage/posix
	option directory /cfs/tekla/
end-volume

volume server
type protocol/server
	option transport-type tcp/server
	option auth.ip.kimik.allow * # Allow access to "kimik" volume
	option auth.ip.tekla.allow * # Allow access to "tekla" volume
	subvolumes kimik tekla 
end-volume
--------------------------------------------------------------------

REEXPORTER SERVER SIDE
### Command to mount glusterfs volumns in client side
$ glusterfs --direct-io-mode=DISABLE
-l/var/log/glusterfs/gfscli-kimik.log -LDEBUG
-f /etc/glusterfs/glusterfs-client-kimik.vol /kimik

$ glusterfs --direct-io-mode=DISABLE
-l/var/log/glusterfs/gfscli-tekla.log -LDEBUG
-f /etc/glusterfs/glusterfs-client-tekla.vol /tekla

$ cat /etc/glusterfs/glusterfs-client-kimik.vol 
--------------------------------------------------------------------
volume sirius00
	type protocol/client
	option transport-type tcp/client
	option remote-host 10.3.1.200
	option remote-subvolume kimik
end-volume

volume sirius01
	type protocol/client
	option transport-type tcp/client
	option remote-host 10.3.1.201
	option remote-subvolume kimik
end-volume

volume kimik
	type cluster/afr
	subvolumes sirius00 sirius01
end-volume

--------------------------------------------------------------------
$ cat /etc/glusterfs/glusterfs-client-tekla.vol
--------------------------------------------------------------------
volume sirius00
	type protocol/client
	option transport-type tcp/client
	option remote-host 10.3.1.200
	option remote-subvolume tekla
end-volume

volume sirius01
	type protocol/client
	option transport-type tcp/client
	option remote-host 10.3.1.201
	option remote-subvolume tekla
end-volume

volume tekla
	type cluster/afr
	subvolumes sirius00 sirius01
end-volume
--------------------------------------------------------------------

NFSSERVER-SIDE
$cat /etc/exports
/kimik
*(sync,no_subtree_check,no_auth_nlm,no_acl,fsid=10,anonuid=1002,anongid=100,rw)
/tekla
*(sync,no_subtree_check,no_auth_nlm,no_acl,fsid=10,anonuid=1002,anongid=100,rw)

NFSCLIENT SIDE
$ mount sirius99:/kimik /mnt/kimik
$ mount sirius99:/tekla /mnt/tekla

##################################################################
OPTION-2 (2 Glusterfs server – 2 glusterfs clients + 2 nfsexports – 2
nfsimports)
##################################################################

GLUSTERFS SERVER SIDE
###Leave up two servers in different ports
$ glusterfs -f /etc/glusterfs/glusterfs-server-kimik.vol
$ glusterfs -f /etc/glusterfs/glusterfs-server-tekla.vol

$cat /etc/glusterfs/glusterfs-server-kimik.vol
--------------------------------------------------------------------
# file: /etc/glusterfs/glusterfs-server.vol
volume kimik
	type storage/posix
	option directory /cfs/kimik/
end-volume

volume server
	type protocol/server
	option transport-type tcp/server
	option listen-port 6996
	option auth.ip.kimik.allow *
	subvolumes kimik
end-volume

--------------------------------------------------------------------
$cat /etc/glusterfs/glusterfs-server-tekla.vol
--------------------------------------------------------------------

# file: /etc/glusterfs/glusterfs-server.vol
volume tekla
	type storage/posix
	option directory /cfs/tekla/
end-volume

volume server
	type protocol/server
	option transport-type tcp/server
	option listen-port 6997
	option auth.ip.tekla.allow *
	subvolumes tekla
end-volume

--------------------------------------------------------------------

REEXPORTER SIDE
### Command
glusterfs --direct-io-mode=DISABLE -l/var/log/glusterfs/gfscli-kimik.log
-LDEBUG -f /etc/glusterfs/glusterfs-client-kimik.vol /kimik

$ cat /etc/glusterfs/glusterfs-client-kimik.vol 
-----------------------------------------------------------
volume sirius00
	type protocol/client
	option transport-type tcp/client
	option remote-host 10.3.1.200
	option remote-port 6996
	option remote-subvolume kimik
end-volume

volume sirius01
	type protocol/client
	option transport-type tcp/client
	option remote-host 10.3.1.201
	option remote-port 6996
	option remote-subvolume kimik
end-volume

volume kimik
	type cluster/afr
	subvolumes sirius00 sirius01
end-volume
-------------------------------------------------------------------

$ cat /root/scripts/glusterstart-tekla
glusterfs --direct-io-mode=DISABLE -l/var/log/glusterfs/gfscli-tekla.log
-LDEBUG -f /etc/glusterfs/glusterfs-client-tekla.vol /tekla

$ cat /etc/glusterfs/glusterfs-client-tekla.vol 
--------------------------------------------------------------------
volume sirius00
	type protocol/client
	option transport-type tcp/client
	option remote-host 10.3.1.200
	option remote-port 6997
	option remote-subvolume tekla
end-volume

volume sirius01
	type protocol/client
	option transport-type tcp/client
	option remote-host 10.3.1.201
	option remote-port 6997
	option remote-subvolume tekla
end-volume

volume tekla
	type cluster/afr
	subvolumes sirius00 sirius01
end-volume
--------------------------------------------------------------------

NFSSERVER & CLIENT (the same that OPTION 1)

-- 
Martin Gumbau - mgumbau at iciq.es
Responsable Informatic
dels grups teorics
de Quimica Computacional
del ICIQ