[Gluster-users] 32 group limit

[Niels de Vos]

On Mon, May 12, 2014 at 08:49:47AM -0700, Harshavardhana wrote:
> Hey Tom,
> 
> http://review.gluster.org/7501 - this fixes your issues, its currently
> on the master branch should be backported to release-3.5 branch.
> Copying Niels he can explain more.
> 
>     rpc: implement server.manage-gids for group resolving on the bricks
> 
>     The new volume option 'server.manage-gids' can be enabled in
>     environments where a user belongs to more than the current absolute
>     maximum of 93 groups. This option triggers the following behavior:
> 
>     1. The AUTH_GLUSTERFS structure sent by GlusterFS clients (fuse, nfs or
>        libgfapi) will contain only one (1) auxiliary group, instead of
>        a full list. This reduces network usage and prevents problems in
>        encoding the AUTH_GLUSTERFS structure which should fit in 400 bytes.
>     2. The single group in the RPC Calls received by the server is replaced
>        by resolving the groups server-side. Permission checks and similar in
>        lower xlators are applied against the full list of groups where the
>        user belongs to, and not the single auxiliary group that the client
>        sent.

Indeed, this change makes it possible for any GlusterFS client 
implementation (fuse, nfs, libgfapi, ..) to use more than 93 groups. The 
current limit of 93 comes from an RPC protocol restriction.

> On Mon, May 12, 2014 at 8:24 AM, Tom Young <tom.young at corvidtec.com> 
> wrote:
> > Hi,
> >
> > Can anyone tell me how I can increase the number of groups that 
> > gluster
> > recognizes from the default 32, to 96?  I’ve heard that we can set gluster
> > to use 96 groups, but I’m not certain how to enable/configure that.  Thank
> > you!

The 32 limit you are hitting is caused by FUSE. The Linux kernel module 
provides the groups of the process that accesses the FUSE-mountpoint 
through /proc/$PID/status (line starting with 'Groups:'). The kernel 
does not pass more groups than 32, this limit is hardcoded in the FUSE 
kernel module.

As Harsha pointed out, change http://review.gluster.org/7501 will make 
it possible to solve this problem for you. When there is a release with 
this change you should be able do do:

  # gluster volume set $VOLUME server.manage-gids on

https://bugzilla.redhat.com/1096425 has been filed to get this change 
backported and included in a 3.5 update. You can follow that bug (add 
yourself on CC if you have an account) and get informed when a release 
is available.

Do let me know if you have any further questions,
Niels