[Gluster-users] I/O error repaired only by owner or root access

[Dan Bretherton]

Dear All-
Several users are having a lot of trouble reading files belonging to 
other users.  Here is an example.

[sms05dab at jupiter ~]$ ls -l /users/gcs/WORK/ORCA1/ORCA1-R07-MEAN/Ctl
ls: /users/gcs/WORK/ORCA1/ORCA1-R07-MEAN/Ctl: Input/output error

The corresponding nfs.log messages are shown below.

[2013-02-21 12:11:39.204659] W [nfs3.c:727:nfs3svc_getattr_stat_cbk] 
0-nfs: fe2ba5b8: /gorgon/users/gcs/WORK/ORCA1/ORCA1-R07-MEAN => -1 
(Invalid argument)
[2013-02-21 12:11:39.204778] W [nfs3-helpers.c:3389:nfs3_log_common_res] 
0-nfs-nfsv3: XID: fe2ba5b8, GETATTR: NFS: 22(Invalid argument for 
operation), POSIX: 22(Invalid argument)
[2013-02-21 12:11:39.215345] I 
[dht-common.c:954:dht_lookup_everywhere_cbk] 0-nemo2-dht: deleting stale 
linkfile /gorgon/users/gcs/WORK/ORCA1/ORCA1-R07-MEAN on nemo2-replicate-0
[2013-02-21 12:11:39.225674] W 
[client3_1-fops.c:592:client3_1_unlink_cbk] 0-nemo2-client-1: remote 
operation failed: Permission denied
[2013-02-21 12:11:39.225786] W 
[client3_1-fops.c:592:client3_1_unlink_cbk] 0-nemo2-client-0: remote 
operation failed: Permission denied
[2013-02-21 12:11:39.681029] W 
[client3_1-fops.c:258:client3_1_mknod_cbk] 0-nemo2-client-18: remote 
operation failed: Permission denied. Path: 
/gorgon/users/gcs/WORK/ORCA1/ORCA1-R07-MEAN 
(1662aa0a-d43b-4c2e-9be9-407eb7a89e85)
[2013-02-21 12:11:39.681400] W 
[client3_1-fops.c:258:client3_1_mknod_cbk] 0-nemo2-client-19: remote 
operation failed: Permission denied. Path: 
/gorgon/users/gcs/WORK/ORCA1/ORCA1-R07-MEAN 
(1662aa0a-d43b-4c2e-9be9-407eb7a89e85)
[2013-02-21 12:11:39.682268] W [nfs3.c:1627:nfs3svc_readlink_cbk] 0-nfs: 
2ca5b8: /gorgon/users/gcs/WORK/ORCA1/ORCA1-R07-MEAN => -1 (Invalid argument)
[2013-02-21 12:11:39.682338] W 
[nfs3-helpers.c:3403:nfs3_log_readlink_res] 0-nfs-nfsv3: XID: 2ca5b8, 
READLINK: NFS: 22(Invalid argument for operation), POSIX: 22(Invalid 
argument), target: (null)

I managed to access the same directory as the owner (or any user with 
write access including root) without any trouble, and after that access 
from my normal user account was fine as well.  The permissions on the 
directory allowed read access by everyone, but the "Permission denied" 
messages in nfs.log indicate that some sort of operation is not being 
allowed when the directory is accessed by other users.   I have seen 
this happen with files and directories, and with the GlusterFS native 
client and NFS.

I presume this is a bug; I would be grateful if someone could confirm 
this.  I would file a bug report, but the trouble is that I don't know 
how to reproduce the problem that causes the I/O error in the first 
place.  It only happens with some files and directories, not all.  Would 
a bug report without any way to reproduce the error be any use, and can 
anyone suggest a way to dig deeper (eg looking at xattrs) next time I 
come across an example?

-Dan.

--
Dan Bretherton
ESSC Computer System Manager
Department of Meteorology
Harry Pitt Building, 3 Earley Gate
University of Reading
Reading, RG6 7BE (or RG6 6AL for postal service deliveries)
UK
Tel. +44 118 378 5205, Fax: +44 118 378 6413
-- 
## Please sponsor me to run in VSO's 30km Race to the Eye ##
##        http://www.justgiving.com/DanBretherton         ##