[Gluster-users] NFS problem

anthony garnier sokar6012 at hotmail.com
Fri Jun 10 07:30:56 UTC 2011


Hi,
Don't know if it's going to help you but I noticed something : 
The issue is on the "chown" cmd, but strangely when I'm using Bash shell  I don't have this error, when using KSH shell => the error come back
In the two case, cmd is executed as root.

Cmd that I'm running : tar   cf -  .   |   (cd /users99/ylalXXX/save ; tar xf -)


> Date: Fri, 10 Jun 2011 12:53:02 +0530
> From: shehjart at gluster.com
> To: christopher.anderlik at xidras.com
> CC: gluster-users at gluster.org; sokar6012 at hotmail.com
> Subject: Re: [Gluster-users] NFS problem
> 
> We'll need the crash stack trace also.
> 
> 
> Christopher Anderlik wrote:
> > here are our logs when nfs is crashing....
> > 
> > 
> > 
> > 
> > 
> > [2011-06-10 08:54:14.900049] D [nfs3-helpers.c:2424:nfs3_log_common_res] 
> > 0-nfs-nfsv3: XID: f8851fc2, ACCESS: NFS: 0(Call completed 
> > successfully.), POSIX: 0(Success)
> > [2011-06-10 08:54:14.902002] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: f9851fc2, Ver: 2, Program: 100003, ProgVers: 3, Proc: 1
> > [2011-06-10 08:54:14.902037] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - GETATTR
> > [2011-06-10 08:54:14.902062] D 
> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: f9851fc2, 
> > GETATTR: args: FH: hashcount 3, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > 74c48fb3-d065-462b-83a9-e4558b042465
> > [2011-06-10 08:54:14.920579] D 
> > [afr-transaction.c:976:afr_post_nonblocking_inodelk_cbk] 
> > 0-ksc-replicate-0: Non blocking inodelks done. Proceeding to FOP
> > [2011-06-10 08:54:14.921099] D [client-lk.c:442:delete_granted_locks_fd] 
> > 0-ksc-client-0: Number of locks cleared=0
> > [2011-06-10 08:54:14.921155] D [client-lk.c:442:delete_granted_locks_fd] 
> > 0-ksc-client-1: Number of locks cleared=0
> > [2011-06-10 08:54:14.932629] D [nfs3-helpers.c:2424:nfs3_log_common_res] 
> > 0-nfs-nfsv3: XID: f9851fc2, GETATTR: NFS: 0(Call completed 
> > successfully.), POSIX: 0(Success)
> > [2011-06-10 08:54:14.932863] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: fa851fc2, Ver: 2, Program: 100003, ProgVers: 3, Proc: 4
> > [2011-06-10 08:54:14.932890] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - ACCESS
> > [2011-06-10 08:54:14.932907] D 
> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: fa851fc2, 
> > ACCESS: args: FH: hashcount 3, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > 74c48fb3-d065-462b-83a9-e4558b042465
> > [2011-06-10 08:54:14.961700] D [socket.c:193:__socket_rwv] 
> > 0-ksc-client-0: EOF from peer 10.0.1.198:24031
> > [2011-06-10 08:54:14.961741] W 
> > [socket.c:1494:__socket_proto_state_machine] 0-ksc-client-0: reading 
> > from socket failed. Error (Transport endpoint is not connected), peer 
> > (10.0.1.198:24031)
> > [2011-06-10 08:54:14.961757] D [socket.c:1768:socket_event_handler] 
> > 0-transport: disconnecting now
> > [2011-06-10 08:54:14.961858] E [rpc-clnt.c:338:saved_frames_unwind] 
> > (-->/opt/glusterfs/3.2.0/lib64/libgfrpc.so.0(rpc_clnt_notify+0x158) 
> > [0x7f140e8c0acc] 
> > (-->/opt/glusterfs/3.2.0/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x101) 
> > [0x7f140e8c006a] 
> > (-->/opt/glusterfs/3.2.0/lib64/libgfrpc.so.0(saved_frames_destroy+0x1c) 
> > [0x7f140e8bfb78]))) 0-ksc-client-0: forced unwinding frame 
> > type(GlusterFS 3.1) op(SETATTR(38)) called at 2011-06-10 08:54:14.920644
> > [2011-06-10 08:54:14.961880] I 
> > [client3_1-fops.c:1640:client3_1_setattr_cbk] 0-ksc-client-0: remote 
> > operation failed: Transport endpoint is not connected
> > [2011-06-10 08:54:14.961880] I 
> > [client3_1-fops.c:1640:client3_1_setattr_cbk] 0-ksc-client-0: remote 
> > operation failed: Transport endpoint is not connected
> > [2011-06-10 08:54:14.961940] I [client.c:1883:client_rpc_notify] 
> > 0-ksc-client-0: disconnected
> > [2011-06-10 08:54:14.988468] W [socket.c:204:__socket_rwv] 
> > 0-ksc-client-1: readv failed (Connection reset by peer)
> > [2011-06-10 08:54:14.988490] W 
> > [socket.c:1494:__socket_proto_state_machine] 0-ksc-client-1: reading 
> > from socket failed. Error (Connection reset by peer), peer 
> > (10.0.1.199:24027)
> > [2011-06-10 08:54:14.988501] D [socket.c:1768:socket_event_handler] 
> > 0-transport: disconnecting now
> > [2011-06-10 08:54:14.988501] D [socket.c:1768:socket_event_handler] 
> > 0-transport: disconnecting now
> > [2011-06-10 08:54:14.988551] E [rpc-clnt.c:338:saved_frames_unwind] 
> > (-->/opt/glusterfs/3.2.0/lib64/libgfrpc.so.0(rpc_clnt_notify+0x158) 
> > [0x7f140e8c0acc] 
> > (-->/opt/glusterfs/3.2.0/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x101) 
> > [0x7f140e8c006a] 
> > (-->/opt/glusterfs/3.2.0/lib64/libgfrpc.so.0(saved_frames_destroy+0x1c) 
> > [0x7f140e8bfb78]))) 0-ksc-client-1: forced unwinding frame 
> > type(GlusterFS 3.1) op(SETATTR(38)) called at 2011-06-10 08:54:14.920655
> > [2011-06-10 08:54:14.988568] I 
> > [client3_1-fops.c:1640:client3_1_setattr_cbk] 0-ksc-client-1: remote 
> > operation failed: Transport endpoint is not connected
> > [2011-06-10 08:54:14.988568] I 
> > [client3_1-fops.c:1640:client3_1_setattr_cbk] 0-ksc-client-1: remote 
> > operation failed: Transport endpoint is not connected
> > [2011-06-10 08:54:14.988599] D [client.c:77:client_submit_request] 
> > 0-ksc-client-0: connection in disconnected state
> > [2011-06-10 08:54:14.988630] W [client3_1-fops.c:4379:client3_1_xattrop] 
> > 0-ksc-client-0: failed to send the fop: Transport endpoint is not connected
> > [2011-06-10 08:54:14.988657] D [name.c:157:client_fill_address_family] 
> > 0-ksc-client-1: address-family not specified, guessing it to be inet/inet6
> > [2011-06-10 08:54:14.991719] D [common-utils.c:151:gf_resolve_ip6] 
> > 0-resolver: returning ip-10.0.1.199 (port-24007) for hostname: 
> > 10.0.1.199 and port: 24007
> > [2011-06-10 08:54:14.991786] I [socket.c:2272:socket_submit_request] 
> > 0-ksc-client-1: not connected (priv->connected = 0)
> > [2011-06-10 08:54:14.991803] W [rpc-clnt.c:1411:rpc_clnt_submit] 
> > 0-ksc-client-1: failed to submit rpc-request (XID: 0x70357x Program: 
> > GlusterFS 3.1, ProgVers: 310, Proc: 33) to rpc-transport (ksc-client-1)
> > [2011-06-10 08:54:14.991819] D [afr-lk-common.c:409:transaction_lk_op] 
> > 0-ksc-replicate-0: lk op is for a transaction
> > [2011-06-10 08:54:14.991833] D [client.c:77:client_submit_request] 
> > 0-ksc-client-0: connection in disconnected state
> > [2011-06-10 08:54:14.991845] W [client3_1-fops.c:4735:client3_1_inodelk] 
> > 0-ksc-client-0: failed to send the fop: Transport endpoint is not connected
> > [2011-06-10 08:54:14.991862] W [rpc-clnt.c:1411:rpc_clnt_submit] 
> > 0-ksc-client-1: failed to submit rpc-request (XID: 0x70358x Program: 
> > GlusterFS 3.1, ProgVers: 310, Proc: 29) to rpc-transport (ksc-client-1)
> > [2011-06-10 08:54:14.991876] I 
> > [client3_1-fops.c:1226:client3_1_inodelk_cbk] 0-ksc-client-1: remote 
> > operation failed: Transport endpoint is not connected
> > [2011-06-10 08:54:14.991904] D [nfs3-helpers.c:2477:nfs3_log_newfh_res] 
> > 0-nfs-nfsv3: XID: c71a69ff, CREATE: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected), FH: hashcount 4, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > f3e21f99-0aed-4042-8b81-16a804497b5b
> > [2011-06-10 08:54:14.991945] D [client.c:129:client_submit_request] 
> > 0-ksc-client-1: rpc_clnt_submit failed
> > [2011-06-10 08:54:14.991967] D [client.c:129:client_submit_request] 
> > 0-ksc-client-1: rpc_clnt_submit failed
> > [2011-06-10 08:54:14.992014] E [rpc-clnt.c:338:saved_frames_unwind] 
> > (-->/opt/glusterfs/3.2.0/lib64/libgfrpc.so.0(rpc_clnt_notify+0x158) 
> > [0x7f140e8c0acc] 
> > (-->/opt/glusterfs/3.2.0/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x101) 
> > [0x7f140e8c006a] 
> > (-->/opt/glusterfs/3.2.0/lib64/libgfrpc.so.0(saved_frames_destroy+0x1c) 
> > [0x7f140e8bfb78]))) 0-ksc-client-1: forced unwinding frame 
> > type(GlusterFS 3.1) op(RELEASE(41)) called at 2011-06-10 08:54:14.921180
> > [2011-06-10 08:54:14.992046] E [rpc-clnt.c:338:saved_frames_unwind] 
> > (-->/opt/glusterfs/3.2.0/lib64/libgfrpc.so.0(rpc_clnt_notify+0x158) 
> > [0x7f140e8c0acc] 
> > (-->/opt/glusterfs/3.2.0/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x101) 
> > [0x7f140e8c006a] 
> > (-->/opt/glusterfs/3.2.0/lib64/libgfrpc.so.0(saved_frames_destroy+0x1c) 
> > [0x7f140e8bfb78]))) 0-ksc-client-1: forced unwinding frame 
> > type(GlusterFS 3.1) op(STAT(1)) called at 2011-06-10 08:54:14.932952
> > [2011-06-10 08:54:14.992060] I [client3_1-fops.c:411:client3_1_stat_cbk] 
> > 0-ksc-client-1: remote operation failed: Transport endpoint is not 
> > connected
> > [2011-06-10 08:54:14.992073] D [client.c:77:client_submit_request] 
> > 0-ksc-client-0: connection in disconnected state
> > [2011-06-10 08:54:14.992091] W [client3_1-fops.c:2658:client3_1_stat] 
> > 0-ksc-client-0: failed to send the fop Transport endpoint is not connected
> > [2011-06-10 08:54:14.992101] D [afr-inode-read.c:204:afr_stat_cbk] 
> > 0-ksc-replicate-0: /pimp/htdocs/mountpoints.sh: all subvolumes tried, 
> > going out
> > [2011-06-10 08:54:14.992115] D [nfs3-helpers.c:2424:nfs3_log_common_res] 
> > 0-nfs-nfsv3: XID: fa851fc2, ACCESS: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected)
> > [2011-06-10 08:54:14.992181] I [client.c:1883:client_rpc_notify] 
> > 0-ksc-client-1: disconnected
> > [2011-06-10 08:54:14.992197] E [afr-common.c:2546:afr_notify] 
> > 0-ksc-replicate-0: All subvolumes are down. Going offline until atleast 
> > one of them comes back up.
> > [2011-06-10 08:54:15.5285] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: fb851fc2, Ver: 2, Program: 100003, ProgVers: 3, Proc: 1
> > [2011-06-10 08:54:15.5312] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - GETATTR
> > [2011-06-10 08:54:15.5312] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - GETATTR
> > [2011-06-10 08:54:15.5344] D [nfs3-helpers.c:2292:nfs3_log_common_call] 
> > 0-nfs-nfsv3: XID: fb851fc2, GETATTR: args: FH: hashcount 3, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > 74c48fb3-d065-462b-83a9-e4558b042465
> > [2011-06-10 08:54:15.5381] I [afr-inode-read.c:270:afr_stat] 
> > 0-ksc-replicate-0: /pimp/htdocs/mountpoints.sh: no child is up
> > [2011-06-10 08:54:15.5399] D [nfs3-helpers.c:2424:nfs3_log_common_res] 
> > 0-nfs-nfsv3: XID: fb851fc2, GETATTR: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected)
> > [2011-06-10 08:54:15.5783] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: fc851fc2, Ver: 2, Program: 100003, ProgVers: 3, Proc: 3
> > [2011-06-10 08:54:15.5810] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - LOOKUP
> > [2011-06-10 08:54:15.5833] D 
> > [nfs3-helpers.c:2304:nfs3_log_fh_entry_call] 0-nfs-nfsv3: XID: fc851fc2, 
> > LOOKUP: args: FH: hashcount 2, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > ff18fc56-c352-4caf-b98d-4eff71494acc, name: mountpoints.sh
> > [2011-06-10 08:54:15.5897] D [nfs3.c:1080:nfs3_fresh_lookup] 
> > 0-nfs-nfsv3: inode needs fresh lookup
> > [2011-06-10 08:54:15.5954] D [nfs3-helpers.c:2477:nfs3_log_newfh_res] 
> > 0-nfs-nfsv3: XID: fc851fc2, LOOKUP: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected), FH: hashcount 0, exportid 
> > 00000000-0000-0000-0000-000000000000, gfid 
> > 00000000-0000-0000-0000-000000000000
> > [2011-06-10 08:54:15.11858] E [socket.c:1685:socket_connect_finish] 
> > 0-ksc-client-1: connection to 10.0.1.199:24027 failed (Connection refused)
> > [2011-06-10 08:54:15.11886] D [socket.c:289:__socket_disconnect] 
> > 0-ksc-client-1: shutdown() returned -1. Transport endpoint is not connected
> > [2011-06-10 08:54:15.11921] D [socket.c:193:__socket_rwv] 
> > 0-ksc-client-1: EOF from peer 10.0.1.199:24027
> > [2011-06-10 08:54:15.11938] D 
> > [socket.c:1494:__socket_proto_state_machine] 0-ksc-client-1: reading 
> > from socket failed. Error (Transport endpoint is not connected), peer 
> > (10.0.1.199:24027)
> > [2011-06-10 08:54:15.11954] D [socket.c:1768:socket_event_handler] 
> > 0-transport: disconnecting now
> > [2011-06-10 08:54:15.158995] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: e1f47a24, Ver: 2, Program: 100003, ProgVers: 3, Proc: 1
> > [2011-06-10 08:54:15.159038] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - GETATTR
> > [2011-06-10 08:54:15.159038] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - GETATTR
> > [2011-06-10 08:54:15.159065] D 
> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: e1f47a24, 
> > GETATTR: args: FH: hashcount 6, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > 9b62d8e0-cf19-487e-b56a-9ac980294801
> > [2011-06-10 08:54:15.159105] I [afr-inode-read.c:270:afr_stat] 
> > 0-ksc-replicate-0: 
> > /vertrieb/htdocs/vertrieb_imrich/fetischtopliste.de/www/button.php: no 
> > child is up
> > [2011-06-10 08:54:15.159126] D [nfs3-helpers.c:2424:nfs3_log_common_res] 
> > 0-nfs-nfsv3: XID: e1f47a24, GETATTR: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected)
> > [2011-06-10 08:54:15.159501] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: e2f47a24, Ver: 2, Program: 100003, ProgVers: 3, Proc: 3
> > [2011-06-10 08:54:15.159528] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - LOOKUP
> > [2011-06-10 08:54:15.159550] D 
> > [nfs3-helpers.c:2304:nfs3_log_fh_entry_call] 0-nfs-nfsv3: XID: e2f47a24, 
> > LOOKUP: args: FH: hashcount 5, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > dd60aa24-78fa-4509-abee-acf32f12542f, name: button.php
> > [2011-06-10 08:54:15.159604] D [nfs3.c:1080:nfs3_fresh_lookup] 
> > 0-nfs-nfsv3: inode needs fresh lookup
> > [2011-06-10 08:54:15.159654] D [nfs3-helpers.c:2477:nfs3_log_newfh_res] 
> > 0-nfs-nfsv3: XID: e2f47a24, LOOKUP: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected), FH: hashcount 0, exportid 
> > 00000000-0000-0000-0000-000000000000, gfid 
> > 00000000-0000-0000-0000-000000000000
> > [2011-06-10 08:54:15.160014] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: e3f47a24, Ver: 2, Program: 100003, ProgVers: 3, Proc: 1
> > [2011-06-10 08:54:15.160041] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - GETATTR
> > [2011-06-10 08:54:15.160080] D 
> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: e3f47a24, 
> > GETATTR: args: FH: hashcount 5, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > dd60aa24-78fa-4509-abee-acf32f12542f
> > [2011-06-10 08:54:15.160113] I [afr-inode-read.c:270:afr_stat] 
> > 0-ksc-replicate-0: 
> > /vertrieb/htdocs/vertrieb_imrich/fetischtopliste.de/www: no child is up
> > [2011-06-10 08:54:15.160130] D [nfs3-helpers.c:2424:nfs3_log_common_res] 
> > 0-nfs-nfsv3: XID: e3f47a24, GETATTR: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected)
> > [2011-06-10 08:54:15.160532] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: e4f47a24, Ver: 2, Program: 100003, ProgVers: 3, Proc: 3
> > [2011-06-10 08:54:15.160554] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - LOOKUP
> > [2011-06-10 08:54:15.160554] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - LOOKUP
> > [2011-06-10 08:54:15.160569] D 
> > [nfs3-helpers.c:2304:nfs3_log_fh_entry_call] 0-nfs-nfsv3: XID: e4f47a24, 
> > LOOKUP: args: FH: hashcount 4, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > 434fc86c-5bed-4a38-b919-8822d38bd1a3, name: www
> > [2011-06-10 08:54:15.160611] D [nfs3.c:1080:nfs3_fresh_lookup] 
> > 0-nfs-nfsv3: inode needs fresh lookup
> > [2011-06-10 08:54:15.160657] D [nfs3-helpers.c:2477:nfs3_log_newfh_res] 
> > 0-nfs-nfsv3: XID: e4f47a24, LOOKUP: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected), FH: hashcount 0, exportid 
> > 00000000-0000-0000-0000-000000000000, gfid 
> > 00000000-0000-0000-0000-000000000000
> > [2011-06-10 08:54:15.192962] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: c81a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 1
> > [2011-06-10 08:54:15.193000] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - GETATTR
> > [2011-06-10 08:54:15.193019] D 
> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: c81a69ff, 
> > GETATTR: args: FH: hashcount 3, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > 61273392-07cb-4427-bb48-2cf869031802
> > [2011-06-10 08:54:15.193050] I [afr-inode-read.c:270:afr_stat] 
> > 0-ksc-replicate-0: /tomorrowwinners.com/typo3temp/locks: no child is up
> > [2011-06-10 08:54:15.193069] D [nfs3-helpers.c:2424:nfs3_log_common_res] 
> > 0-nfs-nfsv3: XID: c81a69ff, GETATTR: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected)
> > [2011-06-10 08:54:15.193526] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: c91a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 3
> > [2011-06-10 08:54:15.193547] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - LOOKUP
> > [2011-06-10 08:54:15.193564] D 
> > [nfs3-helpers.c:2304:nfs3_log_fh_entry_call] 0-nfs-nfsv3: XID: c91a69ff, 
> > LOOKUP: args: FH: hashcount 2, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > 96012827-9101-40a1-b540-b6c47fe49e59, name: locks
> > [2011-06-10 08:54:15.193607] D [nfs3.c:1080:nfs3_fresh_lookup] 
> > 0-nfs-nfsv3: inode needs fresh lookup
> > [2011-06-10 08:54:15.193654] D [nfs3-helpers.c:2477:nfs3_log_newfh_res] 
> > 0-nfs-nfsv3: XID: c91a69ff, LOOKUP: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected), FH: hashcount 0, exportid 
> > 00000000-0000-0000-0000-000000000000, gfid 
> > 00000000-0000-0000-0000-000000000000
> > [2011-06-10 08:54:15.193987] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: ca1a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 1
> > [2011-06-10 08:54:15.194015] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - GETATTR
> > [2011-06-10 08:54:15.194037] D 
> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: ca1a69ff, 
> > GETATTR: args: FH: hashcount 2, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > 96012827-9101-40a1-b540-b6c47fe49e59
> > [2011-06-10 08:54:15.194070] I [afr-inode-read.c:270:afr_stat] 
> > 0-ksc-replicate-0: /tomorrowwinners.com/typo3temp: no child is up
> > [2011-06-10 08:54:15.194086] D [nfs3-helpers.c:2424:nfs3_log_common_res] 
> > 0-nfs-nfsv3: XID: ca1a69ff, GETATTR: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected)
> > [2011-06-10 08:54:15.194566] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: cb1a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 3
> > [2011-06-10 08:54:15.194587] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - LOOKUP
> > [2011-06-10 08:54:15.194614] D 
> > [nfs3-helpers.c:2304:nfs3_log_fh_entry_call] 0-nfs-nfsv3: XID: cb1a69ff, 
> > LOOKUP: args: FH: hashcount 1, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > 7fecc27e-a8de-4fe5-adce-c5e00d35c7ac, name: typo3temp
> > [2011-06-10 08:54:15.194659] D [nfs3.c:1080:nfs3_fresh_lookup] 
> > 0-nfs-nfsv3: inode needs fresh lookup
> > [2011-06-10 08:54:15.194703] D [nfs3-helpers.c:2477:nfs3_log_newfh_res] 
> > 0-nfs-nfsv3: XID: cb1a69ff, LOOKUP: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected), FH: hashcount 0, exportid 
> > 00000000-0000-0000-0000-000000000000, gfid 
> > 00000000-0000-0000-0000-000000000000
> > [2011-06-10 08:54:15.396934] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: cc1a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 1
> > [2011-06-10 08:54:15.396989] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - GETATTR
> > [2011-06-10 08:54:15.396989] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - GETATTR
> > [2011-06-10 08:54:15.397017] D 
> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: cc1a69ff, 
> > GETATTR: args: FH: hashcount 1, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > 7fecc27e-a8de-4fe5-adce-c5e00d35c7ac
> > [2011-06-10 08:54:15.397064] I [afr-inode-read.c:270:afr_stat] 
> > 0-ksc-replicate-0: /tomorrowwinners.com: no child is up
> > [2011-06-10 08:54:15.397087] D [nfs3-helpers.c:2424:nfs3_log_common_res] 
> > 0-nfs-nfsv3: XID: cc1a69ff, GETATTR: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected)
> > [2011-06-10 08:54:15.397441] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: cd1a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 3
> > [2011-06-10 08:54:15.397468] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - LOOKUP
> > [2011-06-10 08:54:15.397492] D 
> > [nfs3-helpers.c:2304:nfs3_log_fh_entry_call] 0-nfs-nfsv3: XID: cd1a69ff, 
> > LOOKUP: args: FH: hashcount 0, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > 00000000-0000-0000-0000-000000000001, name: tomorrowwinners.com
> > [2011-06-10 08:54:15.397551] D [nfs3.c:1080:nfs3_fresh_lookup] 
> > 0-nfs-nfsv3: inode needs fresh lookup
> > [2011-06-10 08:54:15.397600] D [nfs3-helpers.c:2477:nfs3_log_newfh_res] 
> > 0-nfs-nfsv3: XID: cd1a69ff, LOOKUP: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected), FH: hashcount 0, exportid 
> > 00000000-0000-0000-0000-000000000000, gfid 
> > 00000000-0000-0000-0000-000000000000
> > [2011-06-10 08:54:15.397953] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: ce1a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 4
> > [2011-06-10 08:54:15.397981] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - ACCESS
> > [2011-06-10 08:54:15.398004] D 
> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: ce1a69ff, 
> > ACCESS: args: FH: hashcount 0, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > 00000000-0000-0000-0000-000000000001
> > [2011-06-10 08:54:15.398038] I [afr-inode-read.c:270:afr_stat] 
> > 0-ksc-replicate-0: /: no child is up
> > [2011-06-10 08:54:15.398055] D [nfs3-helpers.c:2424:nfs3_log_common_res] 
> > 0-nfs-nfsv3: XID: ce1a69ff, ACCESS: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected)
> > [2011-06-10 08:54:15.600726] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: cf1a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 1
> > [2011-06-10 08:54:15.600769] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - GETATTR
> > [2011-06-10 08:54:15.600769] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - GETATTR
> > [2011-06-10 08:54:15.600789] D 
> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: cf1a69ff, 
> > GETATTR: args: FH: hashcount 0, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > 00000000-0000-0000-0000-000000000001
> > [2011-06-10 08:54:15.600857] D [nfs3-helpers.c:2424:nfs3_log_common_res] 
> > 0-nfs-nfsv3: XID: cf1a69ff, GETATTR: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected)
> > [2011-06-10 08:54:15.601525] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: d01a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 4
> > [2011-06-10 08:54:15.601553] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - ACCESS
> > [2011-06-10 08:54:15.601576] D 
> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: d01a69ff, 
> > ACCESS: args: FH: hashcount 0, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > 00000000-0000-0000-0000-000000000001
> > [2011-06-10 08:54:15.601627] I [afr-inode-read.c:270:afr_stat] 
> > 0-ksc-replicate-0: /: no child is up
> > [2011-06-10 08:54:15.601646] D [nfs3-helpers.c:2424:nfs3_log_common_res] 
> > 0-nfs-nfsv3: XID: d01a69ff, ACCESS: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected)
> > [2011-06-10 08:54:15.725647] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: fd851fc2, Ver: 2, Program: 100003, ProgVers: 3, Proc: 4
> > [2011-06-10 08:54:15.725689] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - ACCESS
> > [2011-06-10 08:54:15.725689] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - ACCESS
> > [2011-06-10 08:54:15.725719] D 
> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: fd851fc2, 
> > ACCESS: args: FH: hashcount 0, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > 00000000-0000-0000-0000-000000000001
> > [2011-06-10 08:54:15.725749] I [afr-inode-read.c:270:afr_stat] 
> > 0-ksc-replicate-0: /: no child is up
> > [2011-06-10 08:54:15.725769] D [nfs3-helpers.c:2424:nfs3_log_common_res] 
> > 0-nfs-nfsv3: XID: fd851fc2, ACCESS: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected)
> > [2011-06-10 08:54:15.726209] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: fe851fc2, Ver: 2, Program: 100003, ProgVers: 3, Proc: 4
> > [2011-06-10 08:54:15.726236] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - ACCESS
> > [2011-06-10 08:54:15.726252] D 
> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: fe851fc2, 
> > ACCESS: args: FH: hashcount 0, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > 00000000-0000-0000-0000-000000000001
> > [2011-06-10 08:54:15.726276] I [afr-inode-read.c:270:afr_stat] 
> > 0-ksc-replicate-0: /: no child is up
> > [2011-06-10 08:54:15.726292] D [nfs3-helpers.c:2424:nfs3_log_common_res] 
> > 0-nfs-nfsv3: XID: fe851fc2, ACCESS: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected)
> > [2011-06-10 08:54:15.804546] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: d11a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 1
> > [2011-06-10 08:54:15.804580] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - GETATTR
> > [2011-06-10 08:54:15.804580] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - GETATTR
> > [2011-06-10 08:54:15.804606] D 
> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: d11a69ff, 
> > GETATTR: args: FH: hashcount 0, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > 00000000-0000-0000-0000-000000000001
> > [2011-06-10 08:54:15.804658] D [nfs3-helpers.c:2424:nfs3_log_common_res] 
> > 0-nfs-nfsv3: XID: d11a69ff, GETATTR: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected)
> > [2011-06-10 08:54:15.805060] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: d21a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 4
> > [2011-06-10 08:54:15.805088] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - ACCESS
> > [2011-06-10 08:54:15.805111] D 
> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: d21a69ff, 
> > ACCESS: args: FH: hashcount 0, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > 00000000-0000-0000-0000-000000000001
> > [2011-06-10 08:54:15.805143] I [afr-inode-read.c:270:afr_stat] 
> > 0-ksc-replicate-0: /: no child is up
> > [2011-06-10 08:54:15.805159] D [nfs3-helpers.c:2424:nfs3_log_common_res] 
> > 0-nfs-nfsv3: XID: d21a69ff, ACCESS: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected)
> > [2011-06-10 08:54:16.8639] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: d31a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 1
> > [2011-06-10 08:54:16.8698] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - GETATTR
> > [2011-06-10 08:54:16.20711] D [nfs3-helpers.c:2292:nfs3_log_common_call] 
> > 0-nfs-nfsv3: XID: d31a69ff, GETATTR: args: FH: hashcount 0, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > 00000000-0000-0000-0000-000000000001
> > [2011-06-10 08:54:16.20821] D [nfs3-helpers.c:2424:nfs3_log_common_res] 
> > 0-nfs-nfsv3: XID: d31a69ff, GETATTR: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected)
> > [2011-06-10 08:54:16.21517] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: d41a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 4
> > [2011-06-10 08:54:16.21556] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - ACCESS
> > [2011-06-10 08:54:16.21575] D [nfs3-helpers.c:2292:nfs3_log_common_call] 
> > 0-nfs-nfsv3: XID: d41a69ff, ACCESS: args: FH: hashcount 0, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > 00000000-0000-0000-0000-000000000001
> > [2011-06-10 08:54:16.21606] I [afr-inode-read.c:270:afr_stat] 
> > 0-ksc-replicate-0: /: no child is up
> > [2011-06-10 08:54:16.21625] D [nfs3-helpers.c:2424:nfs3_log_common_res] 
> > 0-nfs-nfsv3: XID: d41a69ff, ACCESS: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected)
> > [2011-06-10 08:54:16.225038] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: d51a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 1
> > [2011-06-10 08:54:16.225096] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - GETATTR
> > [2011-06-10 08:54:16.225096] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - GETATTR
> > [2011-06-10 08:54:16.225118] D 
> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: d51a69ff, 
> > GETATTR: args: FH: hashcount 0, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > 00000000-0000-0000-0000-000000000001
> > [2011-06-10 08:54:16.225193] D [nfs3-helpers.c:2424:nfs3_log_common_res] 
> > 0-nfs-nfsv3: XID: d51a69ff, GETATTR: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected)
> > [2011-06-10 08:54:16.225595] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: d61a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 4
> > [2011-06-10 08:54:16.225622] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - ACCESS
> > [2011-06-10 08:54:16.225639] D 
> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: d61a69ff, 
> > ACCESS: args: FH: hashcount 0, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > 00000000-0000-0000-0000-000000000001
> > [2011-06-10 08:54:16.225669] I [afr-inode-read.c:270:afr_stat] 
> > 0-ksc-replicate-0: /: no child is up
> > [2011-06-10 08:54:16.225688] D [nfs3-helpers.c:2424:nfs3_log_common_res] 
> > 0-nfs-nfsv3: XID: d61a69ff, ACCESS: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected)
> > [2011-06-10 08:54:16.243210] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: d71a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 4
> > [2011-06-10 08:54:16.243232] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - ACCESS
> > [2011-06-10 08:54:16.243232] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - ACCESS
> > [2011-06-10 08:54:16.243249] D 
> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: d71a69ff, 
> > ACCESS: args: FH: hashcount 0, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > 00000000-0000-0000-0000-000000000001
> > [2011-06-10 08:54:16.243274] I [afr-inode-read.c:270:afr_stat] 
> > 0-ksc-replicate-0: /: no child is up
> > [2011-06-10 08:54:16.243291] D [nfs3-helpers.c:2424:nfs3_log_common_res] 
> > 0-nfs-nfsv3: XID: d71a69ff, ACCESS: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected)
> > [2011-06-10 08:54:16.243670] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: d81a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 1
> > [2011-06-10 08:54:16.243697] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - GETATTR
> > [2011-06-10 08:54:16.243717] D 
> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: d81a69ff, 
> > GETATTR: args: FH: hashcount 0, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > 00000000-0000-0000-0000-000000000001
> > [2011-06-10 08:54:16.243756] D [nfs3-helpers.c:2424:nfs3_log_common_res] 
> > 0-nfs-nfsv3: XID: d81a69ff, GETATTR: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected)
> > [2011-06-10 08:54:16.428628] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: d91a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 1
> > [2011-06-10 08:54:16.428689] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - GETATTR
> > [2011-06-10 08:54:16.428689] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - GETATTR
> > [2011-06-10 08:54:16.428711] D 
> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: d91a69ff, 
> > GETATTR: args: FH: hashcount 0, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > 00000000-0000-0000-0000-000000000001
> > [2011-06-10 08:54:16.428787] D [nfs3-helpers.c:2424:nfs3_log_common_res] 
> > 0-nfs-nfsv3: XID: d91a69ff, GETATTR: NFS: 5(I/O error), POSIX: 
> > 107(Transport endpoint is not connected)
> > [2011-06-10 08:54:16.429129] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] 
> > 0-nfsrpc: RPC XID: da1a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 4
> > [2011-06-10 08:54:16.429157] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] 
> > 0-nfsrpc: Actor found: NFS3 - ACCESS
> > [2011-06-10 08:54:16.429182] D 
> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: da1a69ff, 
> > ACCESS: args: FH: hashcount 0, exportid 
> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid 
> > 00000000-0000-0000-0000-000000000001
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > Am 09.06.2011 17:09, schrieb anthony garnier:
> >> Hi,
> >>
> >> I got the same problem as Juergen,
> >> My volume is a simple replicated volume with 2 host and GlusterFS 3.2.0
> >>
> >> Volume Name: poolsave
> >> Type: Replicate
> >> Status: Started
> >> Number of Bricks: 2
> >> Transport-type: tcp
> >> Bricks:
> >> Brick1: ylal2950:/soft/gluster-data
> >> Brick2: ylal2960:/soft/gluster-data
> >> Options Reconfigured:
> >> diagnostics.brick-log-level: DEBUG
> >> network.ping-timeout: 20
> >> performance.cache-size: 512MB
> >> nfs.port: 2049
> >>
> >> I'm running this command :
> >>
> >> I get those error :
> >> tar: ./uvs00: owner not changed
> >> tar: could not stat ./uvs00/log/0906uvsGESEC.log
> >> tar: ./uvs00: group not changed
> >> tar: could not stat ./uvs00/log/0306uvsGESEC.log
> >> tar: ./uvs00/log: Input/output error
> >> cannot change back?: Unknown error 526
> >> tar: ./uvs00/log: owner not changed
> >> tar: ./uvs00/log: group not changed
> >> tar: tape blocksize error
> >>
> >> And then I tried to "ls" in gluster mount :
> >> /bin/ls: .: Input/output error
> >>
> >> only way is to restart the volume
> >>
> >>
> >> Here is the logfile in Debug mod :
> >>
> >>
> >> Given volfile:
> >> +------------------------------------------------------------------------------+ 
> >>
> >> 1: volume poolsave-client-0
> >> 2: type protocol/client
> >> 3: option remote-host ylal2950
> >> 4: option remote-subvolume /soft/gluster-data
> >> 5: option transport-type tcp
> >> 6: option ping-timeout 20
> >> 7: end-volume
> >> 8:
> >> 9: volume poolsave-client-1
> >> 10: type protocol/client
> >> 11: option remote-host ylal2960
> >> 12: option remote-subvolume /soft/gluster-data
> >> 13: option transport-type tcp
> >> 14: option ping-timeout 20
> >> 15: end-volume
> >> 16:
> >> 17: volume poolsave-replicate-0
> >> 18: type cluster/replicate
> >> 19: subvolumes poolsave-client-0 poolsave-client-1
> >> 20: end-volume
> >> 21:
> >> 22: volume poolsave-write-behind
> >> 23: type performance/write-behind
> >> 24: subvolumes poolsave-replicate-0
> >> 25: end-volume
> >> 26:
> >> 27: volume poolsave-read-ahead
> >> 28: type performance/read-ahead
> >> 29: subvolumes poolsave-write-behind
> >> 30: end-volume
> >> 31:
> >> 32: volume poolsave-io-cache
> >> 33: type performance/io-cache
> >> 34: option cache-size 512MB
> >> 35: subvolumes poolsave-read-ahead
> >> 36: end-volume
> >> 37:
> >> 38: volume poolsave-quick-read
> >> 39: type performance/quick-read
> >> 40: option cache-size 512MB
> >> 41: subvolumes poolsave-io-cache
> >> 42: end-volume
> >> 43:
> >> 44: volume poolsave-stat-prefetch
> >> 45: type performance/stat-prefetch
> >> 46: subvolumes poolsave-quick-read
> >> 47: end-volume
> >> 48:
> >> 49: volume poolsave
> >> 50: type debug/io-stats
> >> 51: option latency-measurement off
> >> 52: option count-fop-hits off
> >> 53: subvolumes poolsave-stat-prefetch
> >> 54: end-volume
> >> 55:
> >> 56: volume nfs-server
> >> 57: type nfs/server
> >> 58: option nfs.dynamic-volumes on
> >> 59: option rpc-auth.addr.poolsave.allow *
> >> 60: option nfs3.poolsave.volume-id 71e0dabf-4620-4b6d-b138-3266096b93b6
> >> 61: option nfs.port 2049
> >> 62: subvolumes poolsave
> >> 63: end-volume
> >>
> >> +------------------------------------------------------------------------------+ 
> >>
> >> [2011-06-09 16:52:23.709018] I [rpc-clnt.c:1531:rpc_clnt_reconfig] 
> >> 0-poolsave-client-0: changing
> >> port to 24014 (from 0)
> >> [2011-06-09 16:52:23.709211] I [rpc-clnt.c:1531:rpc_clnt_reconfig] 
> >> 0-poolsave-client-1: changing
> >> port to 24011 (from 0)
> >> [2011-06-09 16:52:27.716417] I 
> >> [client-handshake.c:1080:select_server_supported_programs]
> >> 0-poolsave-client-0: Using Program GlusterFS-3.1.0, Num (1298437), 
> >> Version (310)
> >> [2011-06-09 16:52:27.716650] I 
> >> [client-handshake.c:913:client_setvolume_cbk] 0-poolsave-client-0:
> >> Connected to 10.68.217.85:24014, attached to remote volume 
> >> '/soft/gluster-data'.
> >> [2011-06-09 16:52:27.716679] I [afr-common.c:2514:afr_notify] 
> >> 0-poolsave-replicate-0: Subvolume
> >> 'poolsave-client-0' came back up; going online.
> >> [2011-06-09 16:52:27.717020] I [afr-common.c:836:afr_fresh_lookup_cbk] 
> >> 0-poolsave-replicate-0: added
> >> root inode
> >> [2011-06-09 16:52:27.729719] I 
> >> [client-handshake.c:1080:select_server_supported_programs]
> >> 0-poolsave-client-1: Using Program GlusterFS-3.1.0, Num (1298437), 
> >> Version (310)
> >> [2011-06-09 16:52:27.730014] I 
> >> [client-handshake.c:913:client_setvolume_cbk] 0-poolsave-client-1:
> >> Connected to 10.68.217.86:24011, attached to remote volume 
> >> '/soft/gluster-data'.
> >> [2011-06-09 17:01:35.537084] W 
> >> [stat-prefetch.c:178:sp_check_and_create_inode_ctx]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/nfs/server.so(nfs_fop_mkdir+0x1cc) 
> >> [0x2aaaab3b88fc]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/debug/io-stats.so(io_stats_mkdir+0x151) 
> >> [0x2aaaab2948e1]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/performance/stat-prefetch.so(sp_mkdir+0xd2) 
> >>
> >> [0x2aaaab1856c2]))) 0-poolsave-stat-prefetch: stat-prefetch context is 
> >> present in inode (ino:0
> >> gfid:00000000-0000-0000-0000-000000000000) when it is supposed to be 
> >> not present
> >> [2011-06-09 17:01:35.546601] W 
> >> [stat-prefetch.c:178:sp_check_and_create_inode_ctx]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/nfs/server.so(nfs_fop_create+0x1db) 
> >> [0x2aaaab3b95bb]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/debug/io-stats.so(io_stats_create+0x165) 
> >> [0x2aaaab294ad5]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/performance/stat-prefetch.so(sp_create+0xbc) 
> >>
> >> [0x2aaaab185c9c]))) 0-poolsave-stat-prefetch: stat-prefetch context is 
> >> present in inode (ino:0
> >> gfid:00000000-0000-0000-0000-000000000000) when it is supposed to be 
> >> not present
> >> [2011-06-09 17:01:35.569755] I 
> >> [client3_1-fops.c:547:client3_1_rmdir_cbk] 0-poolsave-client-0:
> >> remote operation failed: Directory not empty
> >> [2011-06-09 17:01:35.569881] I 
> >> [client3_1-fops.c:547:client3_1_rmdir_cbk] 0-poolsave-client-1:
> >> remote operation failed: Directory not empty
> >> [2011-06-09 17:01:35.579674] W 
> >> [stat-prefetch.c:178:sp_check_and_create_inode_ctx]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/nfs/server.so(nfs_fop_mkdir+0x1cc) 
> >> [0x2aaaab3b88fc]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/debug/io-stats.so(io_stats_mkdir+0x151) 
> >> [0x2aaaab2948e1]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/performance/stat-prefetch.so(sp_mkdir+0xd2) 
> >>
> >> [0x2aaaab1856c2]))) 0-poolsave-stat-prefetch: stat-prefetch context is 
> >> present in inode (ino:0
> >> gfid:00000000-0000-0000-0000-000000000000) when it is supposed to be 
> >> not present
> >> [2011-06-09 17:01:35.587907] W 
> >> [stat-prefetch.c:178:sp_check_and_create_inode_ctx]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/nfs/server.so(nfs_fop_create+0x1db) 
> >> [0x2aaaab3b95bb]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/debug/io-stats.so(io_stats_create+0x165) 
> >> [0x2aaaab294ad5]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/performance/stat-prefetch.so(sp_create+0xbc) 
> >>
> >> [0x2aaaab185c9c]))) 0-poolsave-stat-prefetch: stat-prefetch context is 
> >> present in inode (ino:0
> >> gfid:00000000-0000-0000-0000-000000000000) when it is supposed to be 
> >> not present
> >> [2011-06-09 17:01:35.612918] W 
> >> [stat-prefetch.c:178:sp_check_and_create_inode_ctx]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/nfs/server.so(nfs_fop_create+0x1db) 
> >> [0x2aaaab3b95bb]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/debug/io-stats.so(io_stats_create+0x165) 
> >> [0x2aaaab294ad5]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/performance/stat-prefetch.so(sp_create+0xbc) 
> >>
> >> [0x2aaaab185c9c]))) 0-poolsave-stat-prefetch: stat-prefetch context is 
> >> present in inode (ino:0
> >> gfid:00000000-0000-0000-0000-000000000000) when it is supposed to be 
> >> not present
> >> [2011-06-09 17:01:35.645357] W 
> >> [stat-prefetch.c:178:sp_check_and_create_inode_ctx]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/nfs/server.so(nfs_fop_create+0x1db) 
> >> [0x2aaaab3b95bb]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/debug/io-stats.so(io_stats_create+0x165) 
> >> [0x2aaaab294ad5]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/performance/stat-prefetch.so(sp_create+0xbc) 
> >>
> >> [0x2aaaab185c9c]))) 0-poolsave-stat-prefetch: stat-prefetch context is 
> >> present in inode (ino:0
> >> gfid:00000000-0000-0000-0000-000000000000) when it is supposed to be 
> >> not present
> >> [2011-06-09 17:01:35.660873] I 
> >> [client3_1-fops.c:547:client3_1_rmdir_cbk] 0-poolsave-client-0:
> >> remote operation failed: Directory not empty
> >> [2011-06-09 17:01:35.660955] I 
> >> [client3_1-fops.c:547:client3_1_rmdir_cbk] 0-poolsave-client-1:
> >> remote operation failed: Directory not empty
> >> [2011-06-09 17:01:35.665933] I 
> >> [client3_1-fops.c:547:client3_1_rmdir_cbk] 0-poolsave-client-0:
> >> remote operation failed: Directory not empty
> >> [2011-06-09 17:01:35.666057] I 
> >> [client3_1-fops.c:547:client3_1_rmdir_cbk] 0-poolsave-client-1:
> >> remote operation failed: Directory not empty
> >> [2011-06-09 17:01:35.671199] I 
> >> [client3_1-fops.c:547:client3_1_rmdir_cbk] 0-poolsave-client-0:
> >> remote operation failed: Directory not empty
> >> [2011-06-09 17:01:35.671241] I 
> >> [client3_1-fops.c:547:client3_1_rmdir_cbk] 0-poolsave-client-1:
> >> remote operation failed: Directory not empty
> >> [2011-06-09 17:01:35.680959] W 
> >> [stat-prefetch.c:178:sp_check_and_create_inode_ctx]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/nfs/server.so(nfs_fop_create+0x1db) 
> >> [0x2aaaab3b95bb]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/debug/io-stats.so(io_stats_create+0x165) 
> >> [0x2aaaab294ad5]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/performance/stat-prefetch.so(sp_create+0xbc) 
> >>
> >> [0x2aaaab185c9c]))) 0-poolsave-stat-prefetch: stat-prefetch context is 
> >> present in inode (ino:0
> >> gfid:00000000-0000-0000-0000-000000000000) when it is supposed to be 
> >> not present
> >> [2011-06-09 17:01:35.715633] W 
> >> [stat-prefetch.c:178:sp_check_and_create_inode_ctx]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/nfs/server.so(nfs_fop_create+0x1db) 
> >> [0x2aaaab3b95bb]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/debug/io-stats.so(io_stats_create+0x165) 
> >> [0x2aaaab294ad5]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/performance/stat-prefetch.so(sp_create+0xbc) 
> >>
> >> [0x2aaaab185c9c]))) 0-poolsave-stat-prefetch: stat-prefetch context is 
> >> present in inode (ino:0
> >> gfid:00000000-0000-0000-0000-000000000000) when it is supposed to be 
> >> not present
> >> [2011-06-09 17:01:35.732798] I 
> >> [client3_1-fops.c:547:client3_1_rmdir_cbk] 0-poolsave-client-0:
> >> remote operation failed: Permission denied
> >> [2011-06-09 17:01:35.733044] I 
> >> [client3_1-fops.c:547:client3_1_rmdir_cbk] 0-poolsave-client-1:
> >> remote operation failed: Permission denied
> >> [2011-06-09 17:01:35.750009] W 
> >> [stat-prefetch.c:178:sp_check_and_create_inode_ctx]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/nfs/server.so(nfs_fop_create+0x1db) 
> >> [0x2aaaab3b95bb]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/debug/io-stats.so(io_stats_create+0x165) 
> >> [0x2aaaab294ad5]
> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/performance/stat-prefetch.so(sp_create+0xbc) 
> >>
> >> [0x2aaaab185c9c]))) 0-poolsave-stat-prefetch: stat-prefetch context is 
> >> present in inode (ino:0
> >> gfid:00000000-0000-0000-0000-000000000000) when it is supposed to be 
> >> not present
> >> [2011-06-09 17:01:35.784610] W 
> >> [socket.c:1494:__socket_proto_state_machine] 0-poolsave-client-0:
> >> reading from socket failed. Error (Transport endpoint is not 
> >> connected), peer (10.68.217.85:24014)
> >> [2011-06-09 17:01:35.784745] E [rpc-clnt.c:338:saved_frames_unwind]
> >> (-->/usr/local/lib/libgfrpc.so.0(rpc_clnt_notify+0xb9) [0x2ab58145f7f9]
> >> (-->/usr/local/lib/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x7e) 
> >> [0x2ab58145ef8e]
> >> (-->/usr/local/lib/libgfrpc.so.0(saved_frames_destroy+0xe) 
> >> [0x2ab58145eefe]))) 0-poolsave-client-0:
> >> forced unwinding frame type(GlusterFS 3.1) op(SETATTR(38)) called at 
> >> 2011-06-09 17:01:35.752080
> >> [2011-06-09 17:01:35.784770] I 
> >> [client3_1-fops.c:1640:client3_1_setattr_cbk] 0-poolsave-client-0:
> >> remote operation failed: Transport endpoint is not connected
> >> [2011-06-09 17:01:35.784811] E [rpc-clnt.c:338:saved_frames_unwind]
> >> (-->/usr/local/lib/libgfrpc.so.0(rpc_clnt_notify+0xb9) [0x2ab58145f7f9]
> >> (-->/usr/local/lib/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x7e) 
> >> [0x2ab58145ef8e]
> >> (-->/usr/local/lib/libgfrpc.so.0(saved_frames_destroy+0xe) 
> >> [0x2ab58145eefe]))) 0-poolsave-client-0:
> >> forced unwinding frame type(GlusterFS 3.1) op(STAT(1)) called at 
> >> 2011-06-09 17:01:35.752414
> >> [2011-06-09 17:01:35.784828] I 
> >> [client3_1-fops.c:411:client3_1_stat_cbk] 0-poolsave-client-0: remote
> >> operation failed: Transport endpoint is not connected
> >> [2011-06-09 17:01:35.784875] I [client.c:1883:client_rpc_notify] 
> >> 0-poolsave-client-0: disconnected
> >> [2011-06-09 17:01:35.785400] W [socket.c:204:__socket_rwv] 
> >> 0-poolsave-client-1: readv failed
> >> (Connection reset by peer)
> >> [2011-06-09 17:01:35.785435] W 
> >> [socket.c:1494:__socket_proto_state_machine] 0-poolsave-client-1:
> >> reading from socket failed. Error (Connection reset by peer), peer 
> >> (10.68.217.86:24011)
> >> [2011-06-09 17:01:35.785496] E [rpc-clnt.c:338:saved_frames_unwind]
> >> (-->/usr/local/lib/libgfrpc.so.0(rpc_clnt_notify+0xb9) [0x2ab58145f7f9]
> >> (-->/usr/local/lib/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x7e) 
> >> [0x2ab58145ef8e]
> >> (-->/usr/local/lib/libgfrpc.so.0(saved_frames_destroy+0xe) 
> >> [0x2ab58145eefe]))) 0-poolsave-client-1:
> >> forced unwinding frame type(GlusterFS 3.1) op(SETATTR(38)) called at 
> >> 2011-06-09 17:01:35.752089
> >> [2011-06-09 17:01:35.785516] I 
> >> [client3_1-fops.c:1640:client3_1_setattr_cbk] 0-poolsave-client-1:
> >> remote operation failed: Transport endpoint is not connected
> >> [2011-06-09 17:01:35.785542] W 
> >> [client3_1-fops.c:4379:client3_1_xattrop] 0-poolsave-client-0: failed
> >> to send the fop: Transport endpoint is not connected
> >> [2011-06-09 17:01:35.817662] I [socket.c:2272:socket_submit_request] 
> >> 0-poolsave-client-1: not
> >> connected (priv->connected = 0)
> >> [2011-06-09 17:01:35.817698] W [rpc-clnt.c:1411:rpc_clnt_submit] 
> >> 0-poolsave-client-1: failed to
> >> submit rpc-request (XID: 0x576x Program: GlusterFS 3.1, ProgVers: 310, 
> >> Proc: 33) to rpc-transport
> >> (poolsave-client-1)
> >> [2011-06-09 17:01:35.817721] W 
> >> [client3_1-fops.c:4735:client3_1_inodelk] 0-poolsave-client-0: failed
> >> to send the fop: Transport endpoint is not connected
> >> [2011-06-09 17:01:35.817744] W [rpc-clnt.c:1411:rpc_clnt_submit] 
> >> 0-poolsave-client-1: failed to
> >> submit rpc-request (XID: 0x577x Program: GlusterFS 3.1, ProgVers: 310, 
> >> Proc: 29) to rpc-transport
> >> (poolsave-client-1)
> >> [2011-06-09 17:01:35.817780] I 
> >> [client3_1-fops.c:1226:client3_1_inodelk_cbk] 0-poolsave-client-1:
> >> remote operation failed: Transport endpoint is not connected
> >> [2011-06-09 17:01:35.817897] E [rpc-clnt.c:338:saved_frames_unwind]
> >> (-->/usr/local/lib/libgfrpc.so.0(rpc_clnt_notify+0xb9) [0x2ab58145f7f9]
> >> (-->/usr/local/lib/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x7e) 
> >> [0x2ab58145ef8e]
> >> (-->/usr/local/lib/libgfrpc.so.0(saved_frames_destroy+0xe) 
> >> [0x2ab58145eefe]))) 0-poolsave-client-1:
> >> forced unwinding frame type(GlusterFS 3.1) op(STAT(1)) called at 
> >> 2011-06-09 17:01:35.784870
> >> [2011-06-09 17:01:35.817918] I 
> >> [client3_1-fops.c:411:client3_1_stat_cbk] 0-poolsave-client-1: remote
> >> operation failed: Transport endpoint is not connected
> >> [2011-06-09 17:01:35.817969] I [client.c:1883:client_rpc_notify] 
> >> 0-poolsave-client-1: disconnected
> >> [2011-06-09 17:01:35.817988] E [afr-common.c:2546:afr_notify] 
> >> 0-poolsave-replicate-0: All subvolumes
> >> are down. Going offline until atleast one of them comes back up.
> >> [2011-06-09 17:01:35.818007] E [socket.c:1685:socket_connect_finish] 
> >> 0-poolsave-client-1: connection
> >> to 10.68.217.86:24011 failed (Connection refused)
> >> [2011-06-09 17:01:35.818606] I [afr.h:838:AFR_LOCAL_INIT] 
> >> 0-poolsave-replicate-0: no subvolumes up
> >> [2011-06-09 17:01:35.819129] I [afr-inode-read.c:270:afr_stat] 
> >> 0-poolsave-replicate-0: /uvs00/log:
> >> no child is up
> >> [2011-06-09 17:01:35.819354] I [afr-inode-read.c:270:afr_stat] 
> >> 0-poolsave-replicate-0: /uvs00/log:
> >> no child is up
> >> [2011-06-09 17:01:35.820090] I [afr-inode-read.c:270:afr_stat] 
> >> 0-poolsave-replicate-0: /uvs00: no
> >> child is up
> >> [2011-06-09 17:01:35.820760] I [afr-inode-read.c:270:afr_stat] 
> >> 0-poolsave-replicate-0: /: no child is up
> >> [2011-06-09 17:01:35.821212] I [afr-inode-read.c:270:afr_stat] 
> >> 0-poolsave-replicate-0: /: no child is up
> >> [2011-06-09 17:01:35.821600] I [afr-inode-read.c:270:afr_stat] 
> >> 0-poolsave-replicate-0: /: no child is up
> >> [2011-06-09 17:01:35.822123] I [afr-inode-read.c:270:afr_stat] 
> >> 0-poolsave-replicate-0: /: no child is up
> >> [2011-06-09 17:01:35.822511] I [afr-inode-read.c:270:afr_stat] 
> >> 0-poolsave-replicate-0: /: no child is up
> >> [2011-06-09 17:01:35.822975] I [afr-inode-read.c:270:afr_stat] 
> >> 0-poolsave-replicate-0: /: no child is up
> >> [2011-06-09 17:01:35.823286] I [afr-inode-read.c:270:afr_stat] 
> >> 0-poolsave-replicate-0: /: no child is up
> >> [2011-06-09 17:01:35.823583] I [afr-inode-read.c:270:afr_stat] 
> >> 0-poolsave-replicate-0: /: no child is up
> >> [2011-06-09 17:01:35.823857] I [afr-inode-read.c:270:afr_stat] 
> >> 0-poolsave-replicate-0: /: no child is up
> >> [2011-06-09 17:01:47.518006] I [afr-inode-read.c:270:afr_stat] 
> >> 0-poolsave-replicate-0: /: no child is up
> >> [2011-06-09 17:01:49.39204] E [socket.c:1685:socket_connect_finish] 
> >> 0-poolsave-client-0: connection
> >> to 10.68.217.85:24014 failed (Connection refused)
> >> [2011-06-09 17:01:49.136932] I [afr-inode-read.c:270:afr_stat] 
> >> 0-poolsave-replicate-0: /: no child is up
> >>
> >>
> >>
> >>  > Message: 7
> >>  > Date: Thu, 9 Jun 2011 12:56:39 +0530
> >>  > From: Shehjar Tikoo <shehjart at gluster.com>
> >>  > Subject: Re: [Gluster-users] Glusterfs 3.2.0 NFS Problem
> >>  > To: J?rgen Winkler <juergen.winkler at xidras.com>
> >>  > Cc: gluster-users at gluster.org
> >>  > Message-ID: <4DF075AF.3040509 at gluster.com>
> >>  > Content-Type: text/plain; charset="us-ascii"; format=flowed
> >>  >
> >>  > This can happen if all your servers were unreachable for a few 
> >> seconds. The
> >>  > situation must have rectified during the restart. We could confirm 
> >> if you
> >>  > change the log level on nfs to DEBUG and send us the log.
> >>  >
> >>  > Thanks
> >>  > -Shehjar
> >>  >
> >>  > Ju"rgen Winkler wrote:
> >>  > > Hi,
> >>  > >
> >>  > > i noticed a strange behavior with NFS and Glusterfs 3.2.0 , 3 of our
> >>  > > Servers are loosing the Mount but when you restart the Volume on the
> >>  > > Server it works again without a remount.
> >>  > >
> >>  > > On the server i noticed this entries in the Glusterfs/Nfs 
> >> log-file when
> >>  > > the mount on the Client becomes unavailable :
> >>  > >
> >>  > > [2011-06-08 14:37:02.568693] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:02.569212] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:02.611910] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:02.624477] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:04.288272] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:04.296150] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:04.309247] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:04.320939] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:04.321786] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:04.333609] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:04.334089] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:04.344662] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:04.352666] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:04.354195] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:04.360446] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:04.369331] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:04.471556] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:04.480013] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:05.639700] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:05.652535] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:07.578469] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:07.588949] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:07.590395] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:07.591414] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:07.591932] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:07.592596] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:07.639317] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:07.652919] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:09.332435] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:09.340622] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:09.349360] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:09.349550] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:09.360445] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:09.369497] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:09.369752] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:09.382097] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > > [2011-06-08 14:37:09.382387] I [afr-inode-read.c:270:afr_stat]
> >>  > > 0-ksc-replicate-0: /: no child is up
> >>  > >
> >>  > >
> >>  > > Thx for the help
> >>  > >
> >>  > > _______________________________________________
> >>  > > Gluster-users mailing list
> >>  > > Gluster-users at gluster.org
> >>  > > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
> >>  >
> >>  >
> >>  >
> >>  > ------------------------------
> >>  >
> >>  > _______________________________________________
> >>  > Gluster-users mailing list
> >>  > Gluster-users at gluster.org
> >>  > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
> >>  >
> >>  >
> >>  > End of Gluster-users Digest, Vol 38, Issue 14
> >>  > *********************************************
> >>
> >>
> >>
> >> _______________________________________________
> >> Gluster-users mailing list
> >> Gluster-users at gluster.org
> >> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
> > 
> 
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20110610/1d6cd038/attachment.html>


More information about the Gluster-users mailing list