[Gluster-users] GlusterFS 3.7.2 and ACL

Soumya Koduri skoduri at redhat.com
Thu Jul 30 13:03:10 UTC 2015



On 07/30/2015 06:22 PM, Jüri Palis wrote:
> Hi,
>
> Thanks Niels, your hints about those two options did the trick although
> I had to enable both of them and I had to add nscd (sssd provides user
> identities) to this mix as well.
>
> Now back to the problem with ACL’s. Is your test setup something like
> this: GlusterFS 3.7.2 replicated volume on Centos/RHEL 7 and client or
> clients accessing GlusterFS volumes by NFS protocol, correct?
>
As Jiffin had suggested, did you try the same command on GlusterFS 
Native mount?

Log levels can be increased to TRACE/DEBUG mode using the command 
'gluster vol set <volname> diagnostics.client-log-level [TRACE,DEBUG]'

Also please capture a packet trace on the server-side using the command 
- 'tcpdump -i any -s 0 -w /var/tmp/nfs-acl.pcap tcp and not port 22'

Verify the packets sent by Gluster-NFS process to the brick process to 
set the ACL.

Thanks,
Soumya

> # gluster volume info acltest
> Volume Name: acltest
> Type: Replicate
> Volume ID: 9e0de3f5-45ba-4612-a4f1-16bc5d1eb985
> Status: Started
> Number of Bricks: 1 x 2 = 2
> Transport-type: tcp
> Bricks:
> Brick1: vfs-node-01:/data/gfs/acltest/brick0/brick
> Brick2: vfs-node-02:/data/gfs/acltest/brick0/brick
> Options Reconfigured:
> server.manage-gids: on
> nfs.server-aux-gids: on
> performance.readdir-ahead: on
> server.event-threads: 32
> performance.cache-size: 2GB
> storage.linux-aio: on
> nfs.disable: off
> performance.write-behind-window-size: 1GB
> performance.nfs.io-cache: on
> performance.nfs.write-behind-window-size: 250MB
> performance.nfs.stat-prefetch: on
> performance.nfs.read-ahead: on
> performance.nfs.io-threads: on
> cluster.readdir-optimize: on
> network.remote-dio: on
> auth.allow: 10.1.1.32,10.1.1.42
> diagnostics.latency-measurement: on
> diagnostics.count-fop-hits: on
> nfs.rpc-auth-allow: 10.1.1.32,10.1.1.42
> nfs.trusted-sync: on
>
> Maybe there is a way to increase verbosity of nfs server which could
> help me to trace this problem. I did not find any good hints for
> increasing verbosity of nfs server in documentation.
>
> Regards,
> J.
>
> On 30 Jul 2015, at 10:09, Jiffin Tony Thottan <jthottan at redhat.com
> <mailto:jthottan at redhat.com>> wrote:
>
>>
>>
>> On 29/07/15 20:14, Niels de Vos wrote:
>>> On Wed, Jul 29, 2015 at 05:22:31PM +0300, Jüri Palis wrote:
>>>> Hi,
>>>>
>>>> Another issue with NFS and sec=sys mode. As we all know there is a
>>>> limit of 15 security ids involved when running NFS in sec=sys mode.
>>>> This limit makes effective and granular usage of ACL assigned through
>>>> groups almost unusable. One way to overcome this limit is to use
>>>> kerberised NFS but GlusterFS does not natively support this access
>>>> mode . Another option, at least  according to one email thread, states
>>>> that  GlusterFS has an option server.manage-gids which should mitigate
>>>> this limit and raise it to 90 something.  Is this the option, which
>>>> can be used for increasing sec=sys limit. Sadly documentation does not
>>>> have clear description about this option, what exactly this option
>>>> does and how it should be used.
>>> server.manage-gids is an option to resolve the groups of a uid in the
>>> brick process. You probably need to also use the nfs.server-aux-gids
>>> option so that the NFS-server resolves the gids of the uid accessing the
>>> NFS-server.
>>>
>>> The nfs.server-aux-gids option is used to overcome the
>>> AUTH_SYS/AUTH_UNIX limit of (I thought 32?) groups.
>>>
>>> The server.manage-gids option is used to overcome the GlusterFS protocol
>>> limit of ~93 groups.
>>>
>>> If your users do not belong to 90+ groups, you would not need to set the
>>> server.manage-gids option, and nfs.server-aux-gids might be sufficient.
>>>
>>> HTH,
>>> Niels
>>>
>>>> J.
>>>>
>>>>
>>>> On 29 Jul 2015, at 16:16, Jiffin Tony Thottan <jthottan at redhat.com
>>>> <mailto:jthottan at redhat.com>> wrote:
>>>>
>>>>>
>>>>> On 29/07/15 18:04, Jüri Palis wrote:
>>>>>> Hi,
>>>>>>
>>>>>> setfacl for dir on local filesystem:
>>>>>>
>>>>>> 1. set acl setfacl -m g:x_meie_sec-test02:rx test
>>>>>> 2. get acl
>>>>>>
>>>>>> # getfacl test
>>>>>> user::rwx
>>>>>> group::r-x
>>>>>> group:x_meie_sec-test02:r-x
>>>>>> mask::r-x
>>>>>> other::r-x
>>>>>>
>>>>>> setfacl for dir on GlusterFS volume which is NFS mounted to client
>>>>>> system
>>>>>>
>>>>>> 1. same command is used for setting ACE, no error is returned by
>>>>>> that command
>>>>>> 2. get acl
>>>>>>
>>>>>> #getfacl test
>>>>>> user::rwx
>>>>>> group::r-x
>>>>>> other::---
>>>>>>
>>>>>>
>>>>>> If I use ordinary file as a target on GlusterFS like this
>>>>>>
>>>>>> setfacl -m g:x_meie_sec-test02:rw dummy
>>>>>>
>>>>>> then ACE entry is set for file dummy stored on GlusterFS
>>>>>>
>>>>>> # getfacl dummy
>>>>>> user::rw-
>>>>>> group::r--
>>>>>> group:x_meie_sec-test02:rw-
>>>>>> mask::rw-
>>>>>> other::—
>>>>>>
>>>>>> So, as you can see setting ACLs for files works but does not work
>>>>>> for directories.
>>>>>>
>>>>>> This all is happening on CentOS7, running GlusterFS 3.7.2
>>>>> Hi Jyri,
>>>>>
>>>>> It seems there are couple of issues ,
>>>>>
>>>>> 1.)  when u set a named group acl for file/directory, it clears the
>>>>> permission of others too.
>>>>> 2.)  named group acl is not working properly for directories ,
>>>>>
>>>>> I will try the same on my setup and share my findings.
>>>>> --
>>>>> Jiffin
>>
>> In my setup (glusterfs 3.7.2 and RHEL 7.1 client) it worked properly
>>
>> I followed the same steps mentioned by you.
>> #cd /mnt
>> # mkdir dir
>> # touch file
>> # getfacl file
>> # file: file
>> # owner: root
>> # group: root
>> user::rw-
>> group::r--
>> other::r--
>>
>> # getfacl dir
>> # file: dir
>> # owner: root
>> # group: root
>> user::rwx
>> group::r-x
>> other::r-x
>>
>> # setfacl -m g:gluster:rw file
>> # getfacl file
>> # file: file
>> # owner: root
>> # group: root
>> user::rw-
>> group::r--
>> group:gluster:rw-
>> mask::rw-
>> other::r--
>>
>> setfacl -m g:gluster:r-x dir
>> getfacl dir
>> # file: dir
>> # owner: root
>> # group: root
>> user::rwx
>> group::r-x
>> group:gluster:r-x
>> mask::r-x
>> other::r-x
>>
>>
>> So can u share the following  information from the server.
>> 1.) gluster vol info
>> 2.) nfs.log (nfs-server log)
>> 3.) brick logs
>>
>> and also can u try the same on fuse mount(gluster native mount).
>>
>> --
>> Jiffin
>>
>>>>>> J.
>>>>>> On 29 Jul 2015, at 15:16, Jiffin Thottan <jthottan at redhat.com
>>>>>> <mailto:jthottan at redhat.com>> wrote:
>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>> From: "Jüri Palis" <jyri.palis at gmail.com
>>>>>>> <mailto:jyri.palis at gmail.com>>
>>>>>>> To: gluster-users at gluster.org <mailto:gluster-users at gluster.org>
>>>>>>> Sent: Wednesday, July 29, 2015 4:19:20 PM
>>>>>>> Subject: [Gluster-users] GlusterFS 3.7.2 and ACL
>>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> Setup:
>>>>>>> GFS 3.7.2, NFS is used for host access
>>>>>>>
>>>>>>> Problem:
>>>>>>> POSIX ACL work correctly when ACLs are applied to files but do
>>>>>>> not work when ACLs are applied to directories on GFS volumes.
>>>>>>>
>>>>>>> How can I debug this issue more deeply?
>>>>>>>
>>>>>>> Can you please explain the issue with more details, i.e what
>>>>>>> exactly not working properly , is it setting acl or any
>>>>>>> functionality issue, in which client?
>>>>>>> __
>>>>>>> Jiffin
>>>>>>>
>>>>>>> Regards,
>>>>>>> Jyri
>>>>>>> _______________________________________________
>>>>>>> Gluster-users mailing list
>>>>>>> Gluster-users at gluster.org <mailto:Gluster-users at gluster.org>
>>>>>>> http://www.gluster.org/mailman/listinfo/gluster-users
>>>>>> _______________________________________________
>>>>>> Gluster-users mailing list
>>>>>> Gluster-users at gluster.org <mailto:Gluster-users at gluster.org>
>>>>>> http://www.gluster.org/mailman/listinfo/gluster-users
>>>>> _______________________________________________
>>>>> Gluster-users mailing list
>>>>> Gluster-users at gluster.org <mailto:Gluster-users at gluster.org>
>>>>> http://www.gluster.org/mailman/listinfo/gluster-users
>>>> _______________________________________________
>>>> Gluster-users mailing list
>>>> Gluster-users at gluster.org <mailto:Gluster-users at gluster.org>
>>>> http://www.gluster.org/mailman/listinfo/gluster-users
>>> _______________________________________________
>>> Gluster-users mailing list
>>> Gluster-users at gluster.org <mailto:Gluster-users at gluster.org>
>>> http://www.gluster.org/mailman/listinfo/gluster-users
>>
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org <mailto:Gluster-users at gluster.org>
>> http://www.gluster.org/mailman/listinfo/gluster-users
>
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>


More information about the Gluster-users mailing list