[Gluster-devel] NFS SETATTR call with a truncate and chmod 440 fails

Michael Brown michael at netdirect.ca
Tue Apr 23 17:35:10 UTC 2013


I've run across another problem - this one I'm pretty sure is a problem
with Gluster. I've opened
https://bugzilla.redhat.com/show_bug.cgi?id=955753.

I'm using Oracle DNFS still and it's erroring out on some of its logfiles:
ARC3: Error 19508 Closing archive log file
'/db/flash_recovery_area/ALTUS/archivelog/2013_04_22/o1_mf_1_1093__1366653401581181_.arc'

Gluster is reporting:
[2013-04-22 13:57:22.073354] W
[client3_1-fops.c:707:client3_1_truncate_cbk] 0-gv0-client-9: remote
operation failed: Permission denied
[2013-04-22 13:57:22.073496] W
[client3_1-fops.c:707:client3_1_truncate_cbk] 0-gv0-client-8: remote
operation failed: Permission denied
[2013-04-22 13:57:22.073805] W [nfs3.c:889:nfs3svc_truncate_cbk] 0-nfs:
8b534455:
/fleming1/db0/ALTUS_flash/archivelog/2013_04_22/.o1_mf_1_1093__1366653401581181_.arc
=> -1 (Permission denied)
[2013-04-22 13:57:22.082594] E [nfs3.c:3408:nfs3_remove_resume]
0-nfs-nfsv3: Unable to resolve FH: (192.168.10.3:46391) gv0 :
82c4c5ec-f3ad-4074-ac66-c5a455146d71

Immediately prior to this, that file has attributes:
Regular File mode:0640 uid:500 gid:1000, size: 476959744

The actual NFS RPC causing this error is [1]. Briefly:
Remote Procedure Call, Type:Call XID:0x8b534455
Network File System, SETATTR Call FH:0x5c191ad8
    new_attributes
        mode: value follows
            set_it: value follows (1)
            Mode: 0440, S_IRUSR, S_IRGRP
        size: value follows
            set_it: value follows (1)
            size: 476959744

In other words, a "truncate" and "chmod 440" in the same call.

Gluster is replying with [2]:
Remote Procedure Call, Type:Reply XID:0x8b534455
Network File System, SETATTR Reply  Error:NFS3ERR_ACCES
    Status: NFS3ERR_ACCES (13)

What's happening is that gluster is processing the mode change before
the truncate, causing the truncate to fail.

Incidentally, this also causes gluster to think that these files need
healing:
Gathering Heal info on volume gv0 has been successful
...
Brick fearless1:/export/bricks/500117310007a7ec/glusterdata
/fleming1/db0/ALTUS_flash/archivelog/2013_04_22/.o1_mf_1_1093__1366653401581181_.arc
...
Brick fearless2:/export/bricks/500117310007a74c/glusterdata
/fleming1/db0/ALTUS_flash/archivelog/2013_04_22/.o1_mf_1_1093__1366653401581181_.arc

So, arguably gluster should be doing the truncate before the chmod.
Perhaps the Most Correct thing is to always chmod last if removing
permissions. That's a longer discussion :p

[1] Full RPC Call
Remote Procedure Call, Type:Call XID:0x8b534455
    Fragment header: Last fragment, 172 bytes
        1... .... .... .... .... .... .... .... = Last Fragment: Yes
        .000 0000 0000 0000 0000 0000 1010 1100 = Fragment Length: 172
    XID: 0x8b534455 (2337490005)
    Message Type: Call (0)
    RPC Version: 2
    Program: NFS (100003)
    Program Version: 3
    Procedure: SETATTR (2)
    [The reply to this request is in frame 293325]
    Credentials
        Flavor: AUTH_UNIX (1)
        Length: 52
        Stamp: 0xabcdefab
        Machine Name: fleming1.netdirect.ca
            length: 21
            contents: fleming1.netdirect.ca
            fill bytes: opaque data
        UID: 500
        GID: 1000
        Auxiliary GIDs
            GID: 1000
            GID: 1030
    Verifier
        Flavor: AUTH_NULL (0)
        Length: 0
Network File System, SETATTR Call FH:0x5c191ad8
    [Program Version: 3]
    [V3 Procedure: SETATTR (2)]
    object
        length: 36
        [hash (CRC-32): 0x5c191ad8]
        [Name: .o1_mf_1_1093__1366653401581181_.arc]
        [Full Name:
192.168.10.1:/gv0/fleming1/db0/ALTUS_flash/archivelog/2013_04_22/.o1_mf_1_1093__1366653401581181_.arc]
        decode type as: unknown
        filehandle: 3a4f474c20117b487f884f169490a0349afacf71e16a95fc...
    new_attributes
        mode: value follows
            set_it: value follows (1)
            Mode: 0440, S_IRUSR, S_IRGRP
                .... .... .... .... .... 0... .... .... = S_ISUID: No
                .... .... .... .... .... .0.. .... .... = S_ISGID: No
                .... .... .... .... .... ..0. .... .... = S_ISVTX: No
                .... .... .... .... .... ...1 .... .... = S_IRUSR: Yes
                .... .... .... .... .... .... 0... .... = S_IWUSR: No
                .... .... .... .... .... .... .0.. .... = S_IXUSR: No
                .... .... .... .... .... .... ..1. .... = S_IRGRP: Yes
                .... .... .... .... .... .... ...0 .... = S_IWGRP: No
                .... .... .... .... .... .... .... 0... = S_IXGRP: No
                .... .... .... .... .... .... .... .0.. = S_IROTH: No
                .... .... .... .... .... .... .... ..0. = S_IWOTH: No
                .... .... .... .... .... .... .... ...0 = S_IXOTH: No
        uid: no value
            set_it: no value (0)
        gid: no value
            set_it: no value (0)
        size: value follows
            set_it: value follows (1)
            size: 476959744
        atime: don't change
            set_it: don't change (0)
        mtime: don't change
            set_it: don't change (0)
    guard: no value
        check: no value (0)

[2] Full Reply
Ethernet II, Src: Ibm_36:f7:d0 (5c:f3:fc:36:f7:d0), Dst:
IntelCor_38:e7:58 (00:1e:67:38:e7:58)
Internet Protocol Version 4, Src: 192.168.10.1 (192.168.10.1), Dst:
192.168.10.3 (192.168.10.3)
Transmission Control Protocol, Src Port: 38467 (38467), Dst Port: 46391
(46391), Seq: 1230671698, Ack: 2230824272, Len: 40
Remote Procedure Call, Type:Reply XID:0x8b534455
    Fragment header: Last fragment, 36 bytes
        1... .... .... .... .... .... .... .... = Last Fragment: Yes
        .000 0000 0000 0000 0000 0000 0010 0100 = Fragment Length: 36
    XID: 0x8b534455 (2337490005)
    Message Type: Reply (1)
    [Program: NFS (100003)]
    [Program Version: 3]
    [Procedure: SETATTR (2)]
    Reply State: accepted (0)
    [This is a reply to a request in frame 293324]
    [Time from request: 0.001547000 seconds]
    Verifier
        Flavor: AUTH_NULL (0)
        Length: 0
    Accept State: RPC executed successfully (0)
Network File System, SETATTR Reply  Error:NFS3ERR_ACCES
    [Program Version: 3]
    [V3 Procedure: SETATTR (2)]
    Status: NFS3ERR_ACCES (13)
    obj_wcc
        before
            attributes_follow: no value (0)
        after
            attributes_follow: no value (0)

-- 
Michael Brown               | `One of the main causes of the fall of
Systems Consultant          | the Roman Empire was that, lacking zero,
Net Direct Inc.             | they had no way to indicate successful
?: +1 519 883 1172 x5106    | termination of their C programs.' - Firth

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-devel/attachments/20130423/69a554f1/attachment-0001.html>


More information about the Gluster-devel mailing list