[Gluster-users] vfs_gluster broken

[Anoop C S]

On Thu, 2018-09-20 at 14:58 -0600, Terry McGuire wrote:
> > On Sep 19, 2018, at 06:37, Anoop C S <anoopcs at autistici.org> wrote:
> > 
> > On Wed, 2018-09-12 at 10:37 -0600, Terry McGuire wrote:
> > > > Can you please attach the output of `testparm -s` so as to look through how Samba is setup?
> > 
> > I have a setup where I could browse and work with a GlusterFS volume share made available to
> > Windows
> > via vfs_glusterfs module on CentOS 7.5.1804 with glusterfs-3.10.12-1.el7 and samba-4.7.1-
> > 9.el7_5.
> > 
> > What am I missing? Are there any specific operation that leads to abnormal behaviours?
> 
> It seems that, when using smbclient to keep things simple, the first command in the share’s root
> works, even if it’s a write.  The following commands, even if it’s just an ls, fails.
> 
> One difference that might be making the difference is that my gluster volume is distributed-
> dispersed.  This seems also to break the vfs_fruit module.
> 
> 
> > > From our test server (“nomodule-nofruit” is currently the only well-behaved share):
> > > 
> > > root at mfsuat-01 ~]#testparm -s
> > > Load smb config files from /etc/samba/smb.conf
> > > rlimit_max: increasing rlimit_max (1024) to minimum Windows limit (16384)
> > > Processing section "[share1]"
> > > Processing section "[share2]"
> > > Processing section "[nomodule]"
> > > Processing section "[nomodule-nofruit]"
> > > Processing section "[module]"
> > > Processing section "[IPC$]"
> > > WARNING: No path in service IPC$ - making it unavailable!
> > > NOTE: Service IPC$ is flagged unavailable.
> > 
> > On an unrelated note:
> > 
> > I don't think your intention to make [IPC$] unavailable using the 'available' parameter would
> > work
> > at all.
> 
> I assume this IPC$ stuff is just an artifact of how I built my smb.conf file.  I put "vfs objects
> = glusterfs” into the global stanza, so it didn’t need to be added to each share stanza, but
> I discovered that this broke IPC$ in such a way as to make no share accessible.  To fix this, I
> added a share stanza for IPC$ that includes only “vfs objects =".  Maybe that’s dumb, but it
> worked, and, I guess, results in what we see here.

Ok. First things first. Better move "vfs objects" parameter from [global] section to those share
sections where it is required. This will help you avoid problems in future when you add non-
glusterfs shares(and you forget adding empty "vfs objects" line).

> > > Loaded services file OK.
> > > idmap range not specified for domain '*'
> > > ERROR: Invalid idmap range for domain *!
> > 
> > On an unrelated note:
> > 
> > Why haven't you specified range for default configuration? I think it is a must to set range for
> > the
> > default configuration.
> 
> I guess because I’m clueless enough not to know that I should.  We’re authenticating samba users
> from Windows Active Directory, and I’m really clueless about that.  We got things to the point
> that they were working, and called it a day.

Hm.. we will take that up later.

> > > WARNING: You have some share names that are longer than 12 characters.
> > > These may not be accessible to some older clients.
> > > (Eg. Windows9x, WindowsMe, and smbclient prior to Samba 3.0.)
> > > WARNING: some services use vfs_fruit, others don't. Mounting them in conjunction on OS X
> > > clients
> > > results in undefined behaviour.
> > > 
> > > Server role: ROLE_DOMAIN_MEMBER
> > > 
> > > # Global parameters
> > > [global]
> > > 	log file = /var/log/samba/log.%m
> > > 	map to guest = Bad User
> > > 	max log size = 50
> > > 	realm = XXXX.AD.UALBERTA.CA
> > > 	security = ADS
> > > 	workgroup = STS
> > > 	glusterfs:volume = mfs1
> > > 	idmap config * : backend = tdb
> > > 	access based share enum = Yes
> > > 	force create mode = 0777
> > > 	force directory mode = 0777
> > > 	include = /mfsmount/admin/etc/mfs/smb_shares.conf
> > > 	kernel share modes = No
> > > 	read only = No
> > > 	smb encrypt = desired
> > > 	vfs objects = glusterfs
> > > [share1]
> > > 	path = /share1
> > > 	valid users = @mfs-sa1 at xxxx.ad.ualberta.ca
> > > [share2]
> > > 	path = /share2
> > > 	valid users = @mfs-test-group at xxxx.ad.ualberta.ca
> > 
> > Oh.. you are sharing sub-directories which is also fine.
> 
> We *only* share subdirs of the gluster volume, never the volume itself.
> 
> 
> > > [nomodule]
> > > 	kernel share modes = Yes
> > > 	path = /mfsmount/share1
> > > 	valid users = @mfs-sa1 at xxxx.ad.ualberta.ca
> > > 	vfs objects = fruit streams_xattr
> > 
> > Interesting..
> > 
> > Even this FUSE mounted GlusterFS share is not behaving normal? What errors do you see in
> > glusterfs
> > fuse mount log(/var/log/glusterfs/mfsmount-.log) while accessing this share?
> 
> This one isn’t affected by the current samba issue, but it’s affected by another issue.  (I
> actually posted about it a while ago - subject “vfs_fruit and extended attributes”- and I think
> you responded, but it didn’t go anywhere.)  We discovered that the vfs_fruit module seems to
> interact poorly with our distributed-dispersed gluster volume.  To learn this, we made a test
> replicated volume and it worked fine.

We can look into issue with vfs_fruit + FUSE mount + disperse volume(interesting though..) after we
finish debugging the current problem.

> Ironically, we decided we could live without vfs_fruit when we discovered vfs_glusterfs.  Now we
> have neither.  :-(
> 
> 
> > > [nomodule-nofruit]
> > > 	kernel share modes = Yes
> > > 	path = /mfsmount/share1
> > > 	valid users = @mfs-sa1 at xxxx.ad.ualberta.ca
> > > 	vfs objects = 
> > > 
> > > 
> > > [module]
> > > 	path = /share1
> > > 	valid users = @mfs-sa1 at xxxx.ad.ualberta.ca
> > > [IPC$]
> > > 	available = No
> > > 	vfs objects = 
> > 
> > You may remove the whole [IPC$] section.
> 
> IPC$ stuff explained above.

Base on your explanation I would repeat my previous request to remove [IPC$] and follow what I
suggested above on moving "vfs objects" to individual shares.

Isn't shares [share1] and [module] same?