[Gluster-devel] NFS SETATTR call with a truncate and chmod 440 fails
Michael Brown
michael at netdirect.ca
Tue Apr 23 17:35:10 UTC 2013
I've run across another problem - this one I'm pretty sure is a problem
with Gluster. I've opened
https://bugzilla.redhat.com/show_bug.cgi?id=955753.
I'm using Oracle DNFS still and it's erroring out on some of its logfiles:
ARC3: Error 19508 Closing archive log file
'/db/flash_recovery_area/ALTUS/archivelog/2013_04_22/o1_mf_1_1093__1366653401581181_.arc'
Gluster is reporting:
[2013-04-22 13:57:22.073354] W
[client3_1-fops.c:707:client3_1_truncate_cbk] 0-gv0-client-9: remote
operation failed: Permission denied
[2013-04-22 13:57:22.073496] W
[client3_1-fops.c:707:client3_1_truncate_cbk] 0-gv0-client-8: remote
operation failed: Permission denied
[2013-04-22 13:57:22.073805] W [nfs3.c:889:nfs3svc_truncate_cbk] 0-nfs:
8b534455:
/fleming1/db0/ALTUS_flash/archivelog/2013_04_22/.o1_mf_1_1093__1366653401581181_.arc
=> -1 (Permission denied)
[2013-04-22 13:57:22.082594] E [nfs3.c:3408:nfs3_remove_resume]
0-nfs-nfsv3: Unable to resolve FH: (192.168.10.3:46391) gv0 :
82c4c5ec-f3ad-4074-ac66-c5a455146d71
Immediately prior to this, that file has attributes:
Regular File mode:0640 uid:500 gid:1000, size: 476959744
The actual NFS RPC causing this error is [1]. Briefly:
Remote Procedure Call, Type:Call XID:0x8b534455
Network File System, SETATTR Call FH:0x5c191ad8
new_attributes
mode: value follows
set_it: value follows (1)
Mode: 0440, S_IRUSR, S_IRGRP
size: value follows
set_it: value follows (1)
size: 476959744
In other words, a "truncate" and "chmod 440" in the same call.
Gluster is replying with [2]:
Remote Procedure Call, Type:Reply XID:0x8b534455
Network File System, SETATTR Reply Error:NFS3ERR_ACCES
Status: NFS3ERR_ACCES (13)
What's happening is that gluster is processing the mode change before
the truncate, causing the truncate to fail.
Incidentally, this also causes gluster to think that these files need
healing:
Gathering Heal info on volume gv0 has been successful
...
Brick fearless1:/export/bricks/500117310007a7ec/glusterdata
/fleming1/db0/ALTUS_flash/archivelog/2013_04_22/.o1_mf_1_1093__1366653401581181_.arc
...
Brick fearless2:/export/bricks/500117310007a74c/glusterdata
/fleming1/db0/ALTUS_flash/archivelog/2013_04_22/.o1_mf_1_1093__1366653401581181_.arc
So, arguably gluster should be doing the truncate before the chmod.
Perhaps the Most Correct thing is to always chmod last if removing
permissions. That's a longer discussion :p
[1] Full RPC Call
Remote Procedure Call, Type:Call XID:0x8b534455
Fragment header: Last fragment, 172 bytes
1... .... .... .... .... .... .... .... = Last Fragment: Yes
.000 0000 0000 0000 0000 0000 1010 1100 = Fragment Length: 172
XID: 0x8b534455 (2337490005)
Message Type: Call (0)
RPC Version: 2
Program: NFS (100003)
Program Version: 3
Procedure: SETATTR (2)
[The reply to this request is in frame 293325]
Credentials
Flavor: AUTH_UNIX (1)
Length: 52
Stamp: 0xabcdefab
Machine Name: fleming1.netdirect.ca
length: 21
contents: fleming1.netdirect.ca
fill bytes: opaque data
UID: 500
GID: 1000
Auxiliary GIDs
GID: 1000
GID: 1030
Verifier
Flavor: AUTH_NULL (0)
Length: 0
Network File System, SETATTR Call FH:0x5c191ad8
[Program Version: 3]
[V3 Procedure: SETATTR (2)]
object
length: 36
[hash (CRC-32): 0x5c191ad8]
[Name: .o1_mf_1_1093__1366653401581181_.arc]
[Full Name:
192.168.10.1:/gv0/fleming1/db0/ALTUS_flash/archivelog/2013_04_22/.o1_mf_1_1093__1366653401581181_.arc]
decode type as: unknown
filehandle: 3a4f474c20117b487f884f169490a0349afacf71e16a95fc...
new_attributes
mode: value follows
set_it: value follows (1)
Mode: 0440, S_IRUSR, S_IRGRP
.... .... .... .... .... 0... .... .... = S_ISUID: No
.... .... .... .... .... .0.. .... .... = S_ISGID: No
.... .... .... .... .... ..0. .... .... = S_ISVTX: No
.... .... .... .... .... ...1 .... .... = S_IRUSR: Yes
.... .... .... .... .... .... 0... .... = S_IWUSR: No
.... .... .... .... .... .... .0.. .... = S_IXUSR: No
.... .... .... .... .... .... ..1. .... = S_IRGRP: Yes
.... .... .... .... .... .... ...0 .... = S_IWGRP: No
.... .... .... .... .... .... .... 0... = S_IXGRP: No
.... .... .... .... .... .... .... .0.. = S_IROTH: No
.... .... .... .... .... .... .... ..0. = S_IWOTH: No
.... .... .... .... .... .... .... ...0 = S_IXOTH: No
uid: no value
set_it: no value (0)
gid: no value
set_it: no value (0)
size: value follows
set_it: value follows (1)
size: 476959744
atime: don't change
set_it: don't change (0)
mtime: don't change
set_it: don't change (0)
guard: no value
check: no value (0)
[2] Full Reply
Ethernet II, Src: Ibm_36:f7:d0 (5c:f3:fc:36:f7:d0), Dst:
IntelCor_38:e7:58 (00:1e:67:38:e7:58)
Internet Protocol Version 4, Src: 192.168.10.1 (192.168.10.1), Dst:
192.168.10.3 (192.168.10.3)
Transmission Control Protocol, Src Port: 38467 (38467), Dst Port: 46391
(46391), Seq: 1230671698, Ack: 2230824272, Len: 40
Remote Procedure Call, Type:Reply XID:0x8b534455
Fragment header: Last fragment, 36 bytes
1... .... .... .... .... .... .... .... = Last Fragment: Yes
.000 0000 0000 0000 0000 0000 0010 0100 = Fragment Length: 36
XID: 0x8b534455 (2337490005)
Message Type: Reply (1)
[Program: NFS (100003)]
[Program Version: 3]
[Procedure: SETATTR (2)]
Reply State: accepted (0)
[This is a reply to a request in frame 293324]
[Time from request: 0.001547000 seconds]
Verifier
Flavor: AUTH_NULL (0)
Length: 0
Accept State: RPC executed successfully (0)
Network File System, SETATTR Reply Error:NFS3ERR_ACCES
[Program Version: 3]
[V3 Procedure: SETATTR (2)]
Status: NFS3ERR_ACCES (13)
obj_wcc
before
attributes_follow: no value (0)
after
attributes_follow: no value (0)
--
Michael Brown | `One of the main causes of the fall of
Systems Consultant | the Roman Empire was that, lacking zero,
Net Direct Inc. | they had no way to indicate successful
?: +1 519 883 1172 x5106 | termination of their C programs.' - Firth
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-devel/attachments/20130423/69a554f1/attachment-0001.html>
More information about the Gluster-devel
mailing list