[Gluster-users] GlusterFS 3.7.2 and ACL

Jüri Palis jyri.palis at gmail.com
Thu Jul 30 12:52:20 UTC 2015


Hi,

Thanks Niels, your hints about those two options did the trick although I had to enable both of them and I had to add nscd (sssd provides user identities) to this mix as well. 

Now back to the problem with ACL’s. Is your test setup something like this: GlusterFS 3.7.2 replicated volume on Centos/RHEL 7 and client or clients accessing GlusterFS volumes by NFS protocol, correct?

# gluster volume info acltest
Volume Name: acltest
Type: Replicate
Volume ID: 9e0de3f5-45ba-4612-a4f1-16bc5d1eb985
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: vfs-node-01:/data/gfs/acltest/brick0/brick
Brick2: vfs-node-02:/data/gfs/acltest/brick0/brick
Options Reconfigured:
server.manage-gids: on
nfs.server-aux-gids: on
performance.readdir-ahead: on
server.event-threads: 32
performance.cache-size: 2GB
storage.linux-aio: on
nfs.disable: off
performance.write-behind-window-size: 1GB
performance.nfs.io-cache: on
performance.nfs.write-behind-window-size: 250MB
performance.nfs.stat-prefetch: on
performance.nfs.read-ahead: on
performance.nfs.io-threads: on
cluster.readdir-optimize: on
network.remote-dio: on
auth.allow: 10.1.1.32,10.1.1.42
diagnostics.latency-measurement: on
diagnostics.count-fop-hits: on
nfs.rpc-auth-allow: 10.1.1.32,10.1.1.42
nfs.trusted-sync: on

Maybe there is a way to increase verbosity of nfs server which could help me to trace this problem. I did not find any good hints for increasing verbosity of nfs server in documentation.

Regards,
J.

On 30 Jul 2015, at 10:09, Jiffin Tony Thottan <jthottan at redhat.com> wrote:

> 
> 
> On 29/07/15 20:14, Niels de Vos wrote:
>> On Wed, Jul 29, 2015 at 05:22:31PM +0300, Jüri Palis wrote:
>>> Hi,
>>> 
>>> Another issue with NFS and sec=sys mode. As we all know there is a
>>> limit of 15 security ids involved when running NFS in sec=sys mode.
>>> This limit makes effective and granular usage of ACL assigned through
>>> groups almost unusable. One way to overcome this limit is to use
>>> kerberised NFS but GlusterFS does not natively support this access
>>> mode . Another option, at least  according to one email thread, states
>>> that  GlusterFS has an option server.manage-gids which should mitigate
>>> this limit and raise it to 90 something.  Is this the option, which
>>> can be used for increasing sec=sys limit. Sadly documentation does not
>>> have clear description about this option, what exactly this option
>>> does and how it should be used.
>> server.manage-gids is an option to resolve the groups of a uid in the
>> brick process. You probably need to also use the nfs.server-aux-gids
>> option so that the NFS-server resolves the gids of the uid accessing the
>> NFS-server.
>> 
>> The nfs.server-aux-gids option is used to overcome the
>> AUTH_SYS/AUTH_UNIX limit of (I thought 32?) groups.
>> 
>> The server.manage-gids option is used to overcome the GlusterFS protocol
>> limit of ~93 groups.
>> 
>> If your users do not belong to 90+ groups, you would not need to set the
>> server.manage-gids option, and nfs.server-aux-gids might be sufficient.
>> 
>> HTH,
>> Niels
>> 
>>> J.
>>> 
>>> 
>>> On 29 Jul 2015, at 16:16, Jiffin Tony Thottan <jthottan at redhat.com> wrote:
>>> 
>>>> 
>>>> On 29/07/15 18:04, Jüri Palis wrote:
>>>>> Hi,
>>>>> 
>>>>> setfacl for dir on local filesystem:
>>>>> 
>>>>> 1. set acl setfacl -m g:x_meie_sec-test02:rx test
>>>>> 2. get acl
>>>>> 
>>>>> # getfacl test
>>>>> user::rwx
>>>>> group::r-x
>>>>> group:x_meie_sec-test02:r-x
>>>>> mask::r-x
>>>>> other::r-x
>>>>> 
>>>>> setfacl for dir on GlusterFS volume which is NFS mounted to client system
>>>>> 
>>>>> 1. same command is used for setting ACE, no error is returned by that command
>>>>> 2. get acl
>>>>> 
>>>>> #getfacl test
>>>>> user::rwx
>>>>> group::r-x
>>>>> other::---
>>>>> 
>>>>> 
>>>>> If I use ordinary file as a target on GlusterFS like this
>>>>> 
>>>>> setfacl -m g:x_meie_sec-test02:rw dummy
>>>>> 
>>>>> then ACE entry is set for file dummy stored on GlusterFS
>>>>> 
>>>>> # getfacl dummy
>>>>> user::rw-
>>>>> group::r--
>>>>> group:x_meie_sec-test02:rw-
>>>>> mask::rw-
>>>>> other::—
>>>>> 
>>>>> So, as you can see setting ACLs for files works but does not work for directories.
>>>>> 
>>>>> This all is happening on CentOS7, running GlusterFS 3.7.2
>>>> Hi Jyri,
>>>> 
>>>> It seems there are couple of issues ,
>>>> 
>>>> 1.)  when u set a named group acl for file/directory, it clears the permission of others too.
>>>> 2.)  named group acl is not working properly for directories ,
>>>> 
>>>> I will try the same on my setup and share my findings.
>>>> --
>>>> Jiffin
> 
> In my setup (glusterfs 3.7.2 and RHEL 7.1 client) it worked properly
> 
> I followed the same steps mentioned by you.
> #cd /mnt
> # mkdir dir
> # touch file
> # getfacl file
> # file: file
> # owner: root
> # group: root
> user::rw-
> group::r--
> other::r--
> 
> # getfacl dir
> # file: dir
> # owner: root
> # group: root
> user::rwx
> group::r-x
> other::r-x
> 
> # setfacl -m g:gluster:rw file
> # getfacl file
> # file: file
> # owner: root
> # group: root
> user::rw-
> group::r--
> group:gluster:rw-
> mask::rw-
> other::r--
> 
> setfacl -m g:gluster:r-x dir
> getfacl dir
> # file: dir
> # owner: root
> # group: root
> user::rwx
> group::r-x
> group:gluster:r-x
> mask::r-x
> other::r-x
> 
> 
> So can u share the following  information from the server.
> 1.) gluster vol info
> 2.) nfs.log (nfs-server log)
> 3.) brick logs
> 
> and also can u try the same on fuse mount(gluster native mount).
> 
> --
> Jiffin
> 
>>>>> J.
>>>>> On 29 Jul 2015, at 15:16, Jiffin Thottan <jthottan at redhat.com> wrote:
>>>>> 
>>>>>> ----- Original Message -----
>>>>>> From: "Jüri Palis" <jyri.palis at gmail.com>
>>>>>> To: gluster-users at gluster.org
>>>>>> Sent: Wednesday, July 29, 2015 4:19:20 PM
>>>>>> Subject: [Gluster-users] GlusterFS 3.7.2 and ACL
>>>>>> 
>>>>>> Hi
>>>>>> 
>>>>>> Setup:
>>>>>> GFS 3.7.2, NFS is used for host access
>>>>>> 
>>>>>> Problem:
>>>>>> POSIX ACL work correctly when ACLs are applied to files but do not work when ACLs are applied to directories on GFS volumes.
>>>>>> 
>>>>>> How can I debug this issue more deeply?
>>>>>> 
>>>>>> Can you please explain the issue with more details, i.e what exactly not working properly , is it setting acl or any functionality issue, in which client?
>>>>>> __
>>>>>> Jiffin
>>>>>> 
>>>>>> Regards,
>>>>>> Jyri
>>>>>> _______________________________________________
>>>>>> Gluster-users mailing list
>>>>>> Gluster-users at gluster.org
>>>>>> http://www.gluster.org/mailman/listinfo/gluster-users
>>>>> _______________________________________________
>>>>> Gluster-users mailing list
>>>>> Gluster-users at gluster.org
>>>>> http://www.gluster.org/mailman/listinfo/gluster-users
>>>> _______________________________________________
>>>> Gluster-users mailing list
>>>> Gluster-users at gluster.org
>>>> http://www.gluster.org/mailman/listinfo/gluster-users
>>> _______________________________________________
>>> Gluster-users mailing list
>>> Gluster-users at gluster.org
>>> http://www.gluster.org/mailman/listinfo/gluster-users
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-users
> 
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150730/a82be035/attachment.html>


More information about the Gluster-users mailing list