[Gluster-users] gluster or fuse and group rights

[Charles Weber]

I have added new groups to AD/Winbind/LDAP IDMAP setup. We have used this setup with CO5X and recently CO6X for some years. It works fine with usual caveats to let us keep AD account SID to UID and GID mappings on SMB and NFS clients.

Set up:
2 x DL380 with ~90TB in cluster mirror pair
CentOS 6.3 with samba version 3.5.10-125.el6.x86_64 with Gluster 3.4qa4.
I started with gluster 3.31and have tried various versions of sernet samba.  

We have a setup with CO5X and recently CO6X for some years using AD/Winbind/LDAP IDMAP. It works fine with usual caveats to let us keep AD account SID to UID and GID mappings on SMB and NFS clients. Backing filesystem is XFS, which again I have used for many years on large scale file servers.


Everything works fine on our cluster test setupuntil I created some new groups on AD and then tried to apply rights with the new groups.
I can apply ownership or ACLs using old or new groups to the raw brick or to the mounted gluster volume. I can see the groups via "id uname".  Old group membership rights work. New group membership rights work on the brick but do not work on the cluster volume.

So I simplified by shutting down 1 server and wiping all volumes. I formatted brick and created simple volume:
gluster volume create myorg_vol 192.168.255.254:/bricks/myorg_brk
Here is mount:
/dev/mapper/myorg_vg-myorg_lv on /bricks/myorg_brk type xfs (rw,allocsize=4096)
192.168.255.254:/myorg_vol on /gluster/myorg type fuse.glusterfs (rw,allow_other,max_read=131072)

Here is example of problem:
1. on brick
mkdir /bricks/myorg_brk/cwtest	 				### make test dir on brick, set group and rights on dir
chgrp nugrp /bricks/myorg_brk/cwtest   			
chmod 770 /bricks/myorg_brk/cutest
su - nugrpmem								### server setup with authconfig to allow AD user to login and auto create home
cd  /bricks/myorg_brk/cwtest					### cd to test dir on brick as normal user in group nugrp. Make new dir. this all works and has appropriate rights.
mkdir newtest								

This works exactly as you think it should.

2. on gluster but with old group
mkdir /gluster/myorg/cwtest	 				### make test dir on cluster, set group and rights on dir
chgrp oldgrp /bricks/myorg/cwtest   			
chmod 770 /bricks/myorg/cwtest
su - oldgrpmem								### server setup with authconfig to allow AD user to login and auto create home
cd  /bricks/myorg/cwtest					### cd to test dir on gluster as normal user in group oldgrp. Make new dir. this all works and has appropriate rights.
mkdir newtest								

This works exactly as you think it should.

3. on gluster with new group
mkdir /gluster/myorg/cwtest	 				### make test dir on gluster, set group and rights on dir
chgrp nugrp /bricks/myorg/cwtest   			
chmod 770 /bricks/myorg/cutest
su - nugrpmem								### server setup with authconfig to allow AD user to login and auto create home
cd  /bricks/myorg/cwtest					### cd to test dir on gluster as normal user in group nugrp. Try to make dir, but it fails every time.
mkdir newtest								

I cannot do anything that relies on group membership in the test directory with several new groups I created. Here is sample getfacl for a test dir.
# file: cwtest
# owner: root
# group: domain\nugrp
user::rwx
group::rwx
other::---



The only obvious thing I can think of is we have a large range of GIDs. It ranges from 15000-3475000 in Samba IDMAP settings.
So an old group has a GID of 150005 and a recently created group might be 2063042. I have had multiple reboots and versions of samba and gluster to verify this result. I have also verified that the new groups work fine on our existing non-gluster servers.

Any bright ideas are welcome.

Chuck