[Gluster-devel] 3.5.1-beta2 Problems with suid and sgid bits on directories

[Anders Blomdell]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 2014-06-24 22:26, Shyamsundar Ranganathan wrote:
> ----- Original Message -----
>> From: "Anders Blomdell" <anders.blomdell at control.lth.se>
>> To: "Niels de Vos" <ndevos at redhat.com>
>> Cc: "Shyamsundar Ranganathan" <srangana at redhat.com>, "Gluster Devel" <gluster-devel at gluster.org>, "Susant Palai"
>> <spalai at redhat.com>
>> Sent: Tuesday, June 24, 2014 4:09:52 AM
>> Subject: Re: [Gluster-devel] 3.5.1-beta2 Problems with suid and sgid bits on directories
>>
> On 2014-06-23 12:03, Niels de Vos wrote:
>>>> On Tue, Jun 17, 2014 at 11:49:26AM -0400, Shyamsundar Ranganathan wrote:
>>>>> You maybe looking at the problem being fixed here, [1].
>>>>>
>>>>> On a lookup attribute mismatch was not being healed across
>>>>> directories, and this patch attempts to address the same. Currently
>>>>> the version of the patch does not heal the S_ISUID and S_ISGID bits,
>>>>> which is work in progress (but easy enough to incorporate and test
>>>>> based on the patch at [1]).
>>>>>
>>>>> On a separate note, add-brick just adds a brick to the cluster, the
>>>>> lookup is where the heal (or creation of the directory across all sub
>>>>> volumes in DHT xlator) is being done.
>>>>
>>>> I assume that this is not a regression between 3.5.0 and 3.5.1? If that
>>>> is the case, we can pull the fix in 3.5.2 because 3.5.1 really should
>>>> not get delayed much longer.
> No, it does not work in 3.5.0 either :-(
> 
>> I ran these tests using your scripts and observed similar behavior and need to dig into this a little further to understand how to make this work reliably.
This might be a root cause, probably should be resolved first:

https://bugzilla.redhat.com/show_bug.cgi?id=1113050

> 
> 
> 
> The proposed patch does not work as intended, with the following hieararchy
> 
>    755    0:   0 /mnt/gluster
>   2777    0:1000 /mnt/gluster/test
>   2755 1000:1000 /mnt/gluster/test/dir1
>   2755 1000:1000 /mnt/gluster/test/dir1/dir2
> 
> In the (approx 25%) of cases where my test-script does trigger a
> self heal on disk2, 10% ends up with (giving access error on client):
> 
>      0    0:   0 /data/disk2/gluster/test
>    755 1000:1000 /data/disk2/gluster/test/dir1
>    755 1000:1000 /data/disk2/gluster/test/dir1/dir2
> or
> 
>   2777    0:1000 /data/disk2/gluster/test
>      0    0:   0 /data/disk2/gluster/test/dir1
>    755 1000:1000 /data/disk2/gluster/test/dir1/dir2
> 
> or
> 
>   2777    0:1000 /data/disk2/gluster/test
>   2755 1000:1000 /data/disk2/gluster/test/dir1
>      0    0:   0 /data/disk2/gluster/test/dir1/dir2
> 
> 
> and 73% ends up with either partially healed directories
> (/data/disk2/gluster/test/dir1/dir2 or
>  /data/disk2/gluster/test/dir1 missing) or the sgid bit
> [randomly] set on some of the directories.
> 
> Since I don't even understand how to reliably trigger
> a self-heal of the directories, I'm currently clueless
> to the reason for this behaviour.
> 
> Soo, I think that the comment from susant in
> http://review.gluster.org/#/c/6983/3/xlators/cluster/dht/src/dht-common.c:
> 
>   susant palai		Jun 13 9:04 AM
> 
>    I think we dont have to worry about that.
>    Rebalance does not interfere with directory SUID/GID/STICKY bits.
> 
> unfortunately is wrong :-(, and I'm on too deep water to understand how to
> fix this at the moment.
> 
>
> Currently in the test case rebalance is not run, so the above comment in relation to rebalance is sort of different that what is observed. Just a note.
I stand corrected :-) 
So far only self-heal has interfered.
> 
> 
> 
> N.B: with 00777 flags on the /mnt/gluster/test directory
> I have not been able to trigger any unreadable directories
> 
> /Anders
> 
>>>>
>>>> Thanks,
>>>> Niels
>>>>
>>>>>
>>>>> Shyam
>>>>>
>>>>> [1] http://review.gluster.org/#/c/6983/
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Anders Blomdell" <anders.blomdell at control.lth.se>
>>>>>> To: "Gluster Devel" <gluster-devel at gluster.org>
>>>>>> Sent: Tuesday, June 17, 2014 10:53:52 AM
>>>>>> Subject: [Gluster-devel] 3.5.1-beta2 Problems with suid and sgid bits on
>>>>>> 	directories
>>>>>>
>>>> With a glusterfs-3.5.1-0.3.beta2.fc20.x86_64 with a reverted
>>>> 3dc56cbd16b1074d7ca1a4fe4c5bf44400eb63ff (due to local lack of IPv4
>>>> addresses), I get
>>>> weird behavior if I:
>>>>
>>>> 1. Create a directory with suid/sgid/sticky bit set (/mnt/gluster/test)
>>>> 2. Make a subdirectory of #1 (/mnt/gluster/test/dir1)
>>>> 3. Do an add-brick
>>>>
>>>> Before add-brick
>>>>
>>>>    755 /mnt/gluster
>>>>   7775 /mnt/gluster/test
>>>>   2755 /mnt/gluster/test/dir1
>>>>
>>>> After add-brick
>>>>
>>>>    755 /mnt/gluster
>>>>   1775 /mnt/gluster/test
>>>>    755 /mnt/gluster/test/dir1
>>>>
>>>> On the server it looks like this:
>>>>
>>>>   7775 /data/disk1/gluster/test
>>>>   2755 /data/disk1/gluster/test/dir1
>>>>   1775 /data/disk2/gluster/test
>>>>    755 /data/disk2/gluster/test/dir1
>>>>
>>>> Filed as bug:
>>>>
>>>>   https://bugzilla.redhat.com/show_bug.cgi?id=1110262
>>>>
>>>> If somebody can point me to where the logic of add-brick is placed, I can
>>>> give
>>>> it a shot (a find/grep on mkdir didn't immediately point me to the right
>>>> place).
>>>>
>>>>

/Anders

- -- 
Anders Blomdell                  Email: anders.blomdell at control.lth.se
Department of Automatic Control
Lund University                  Phone:    +46 46 222 4625
P.O. Box 118                     Fax:      +46 46 138118
SE-221 00 Lund, Sweden

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJTqq+gAAoJENZYyvaDG8NcCjQH/3ewRS0It6zG5uQeeA9QDpOH
l3ed1O+Xrrsz1l5woSXLuSq7z72QQ9g24Z9L3/J9IKVSu8GNoIZZ0qtgKJfB/k0W
TTNcuR4Z1ao+g8bW8Ec8gsT/NdMJtWxJPClDydzWTGi1LxgxTUD7LzyGX/51RsP6
e+t4uZSb0INF1WYqChnZ5S0ZxYyBm5RvFm8gj91sx5MdLOu7Bvr/LuGK+5uE6DOG
S9G7LHW0bJ8udlwQmjl/106rJp6nkd2FibAeCBWicxlDUK8HbsJgqm3NI3xauj8A
+nwh1bUKgLXlZzyc0W5+vd2X19zygvYGICc0nZuKPqkZlFsuN37sb2r5A3o4ig4=
=jh/D
-----END PGP SIGNATURE-----