[Bugs] [Bug 1543016] New: dht_lookup_unlink_of_false_linkto_cbk fails with " Permission denied"

[bugzilla at redhat.com]

https://bugzilla.redhat.com/show_bug.cgi?id=1543016

            Bug ID: 1543016
           Summary: dht_lookup_unlink_of_false_linkto_cbk fails with
                    "Permission denied"
           Product: GlusterFS
           Version: 3.12
         Component: distribute
          Severity: high
          Assignee: bugs at gluster.org
          Reporter: nbalacha at redhat.com
                CC: amukherj at redhat.com, aspandey at redhat.com,
                    atoborek at redhat.com, bturner at redhat.com,
                    bugs at gluster.org, jahernan at redhat.com,
                    jthottan at redhat.com, nbalacha at redhat.com,
                    pkarampu at redhat.com, pousley at redhat.com,
                    rabhat at redhat.com, rhinduja at redhat.com,
                    rhs-bugs at redhat.com, storage-qa-internal at redhat.com,
                    tdesala at redhat.com
        Depends On: 1540664, 1542318



+++ This bug was initially created as a clone of Bug #1542318 +++

+++ This bug was initially created as a clone of Bug #1540664 +++

Description of problem:

[2018-01-30 19:41:53.630949] I [MSGID: 109045]
[dht-common.c:2391:dht_lookup_cbk] 0-vol1-dht: linkfile not having link subvol
for /data/file1
[2018-01-30 19:41:53.672524] W [MSGID: 114031]
[client-rpc-fops.c:630:client3_3_unlink_cbk] 0-vol1-client-6: remote operation
failed [Permission denied]
[2018-01-30 19:41:53.672638] W [MSGID: 114031]
[client-rpc-fops.c:630:client3_3_unlink_cbk] 0-vol1-client-7: remote operation
failed [Permission denied]
[2018-01-30 19:41:53.672632] W [MSGID: 114031]
[client-rpc-fops.c:630:client3_3_unlink_cbk] 0-vol1-client-8: remote operation
failed [Permission denied]
[2018-01-30 19:41:53.672828] W [MSGID: 114031]
[client-rpc-fops.c:630:client3_3_unlink_cbk] 0-vol1-client-11: remote operation
failed [Permission denied]
[2018-01-30 19:41:53.672839] I [MSGID: 109069]
[dht-common.c:1353:dht_lookup_unlink_of_false_linkto_cbk] 0-vol1-dht:
lookup_unlink returned with op_ret -> -1 and op-errno -> 13 for /data/file1
[2018-01-30 19:41:53.672868] W [MSGID: 112199]
[nfs3-helpers.c:3515:nfs3_log_newfh_res] 0-nfs-nfsv3: /data/file1 => (XID:
52dbaffa, LOOKUP: NFS: 5(I/O error), POSIX: 5(Input/output error)), FH:
exportid 00000000-0000-0000-0000-000000000000, gfid
00000000-0000-0000-0000-000000000000, mountid
00000000-0000-0000-0000-000000000000 [Resource temporarily unavailable]

Actual results:

Bricks are unable to fully heal.

Expected results:

Normal operation.

Additional info:

--- Additional comment from Nithya Balachandran on 2018-02-06 04:48:10 EST ---

Unprivileged users do not have permissions to delete stale linkto files.
Failure to unlink the file causes DHT to return EIO.


Steps to reproduce this issue:
=============================

1. gluster volume create vol1 server1:/bricks/brick1/vol1-{1,2}
2. gluster volume start vol1                                                    
3. mount -t glusterfs -s server1:/vol1 /mnt/fuse1                               

# create a file for testing                                                     
4. dd if=/dev/urandom of=/mnt/fuse1/FILE-1 count=1 bs=16k                       
5. gluster volume set vol1 performance.parallel-readdir on

#Rename the file in order to create a linkto file                        
6. mv vol1/FILE-1 vol1/FILE-2    

#Force the linkto xattr values to become stale by changing the dht subvols in
the graph                               
7. gluster volume set vol1 performance.parallel-readdir off 

Fuse mount as a non-privileged user

8. echo user_allow_other >> /etc/fuse.conf                                      
9. gluster volume set vol1 allow-insecure on  

Create a user user1
10. useradd user1     
11: su user1                  
12. cd /home/user1/
13. mkdir mymount

#Mount the volume as user1
14: glusterfs --log-level=INFO --log-file=my_user_accessable_logfile.log
--volfile-id=vol1 --volfile-server=server1 ~/mymount

15: cd mymount/
16. ls FILE-2

Results:
=========
ls: cannot access FILE-2: Input/output error

--- Additional comment from Worker Ant on 2018-02-06 04:53:36 EST ---

REVIEW: https://review.gluster.org/19508 (cluster/dht: Unlink linkto files as
root) posted (#1) for review on master by N Balachandran

--- Additional comment from Nithya Balachandran on 2018-02-06 08:32:53 EST ---

See Joe Julian's blog for details on how to FUSE mount the volume as a
non-privileged user.

https://joejulian.name/post/mounting-a-glusterfs-volume-as-an-unprivileged-user/

--- Additional comment from Worker Ant on 2018-02-07 09:33:32 EST ---

COMMIT: https://review.gluster.org/19508 committed in master by "N
Balachandran" <nbalacha at redhat.com> with a commit message- cluster/dht: Unlink
linkto files as root

Non-privileged users cannot delete linkto
files. However the failure to unlink a stale linkto
causes DHT to fail the lookup with EIO and hence
prevent access to the file.

Change-Id: Id295362d41e52263790694602f36f1219f0646a2
BUG: 1542318
Signed-off-by: N Balachandran <nbalacha at redhat.com>


Referenced Bugs:

https://bugzilla.redhat.com/show_bug.cgi?id=1540664
[Bug 1540664] Files are unavailable on the mount point
https://bugzilla.redhat.com/show_bug.cgi?id=1542318
[Bug 1542318] dht_lookup_unlink_of_false_linkto_cbk fails with "Permission
denied"
-- 
You are receiving this mail because:
You are on the CC list for the bug.
You are the assignee for the bug.