[Gluster-devel] brick stops responding

[Pranith Kumar Karampuri]

Could you provide statedumps of the bricks and mounts please.
Just curious what does the test do?

Pranith

----- Original Message -----
> From: "Emmanuel Dreyfus" <manu at netbsd.org>
> To: gluster-devel at nongnu.org
> Sent: Thursday, January 16, 2014 8:11:36 PM
> Subject: [Gluster-devel] brick stops responding
> 
> Hi
> 
> With 3.5beta1, at some point I have a brick that stops responding, and
> reconection never happens.
> 
> Client log:
> 
> [2014-01-16 12:57:47.896907] C
> [client-handshake.c:127:rpc_client_ping_timer_expired] 0-gfs35b1-client-0:
> server 192.0.2.99:49152 has not responded in the last 42 seconds,
> disconnecting.
> [2014-01-16 12:57:47.897315] E [rpc-clnt.c:369:saved_frames_unwind]
> 0-gfs35b1-client-0: forced unwinding frame type(GlusterFS 3.3)
> op(INODELK(29)) called at 2014-01-16 12:56:33.651927 (xid=0xded6c2)
> [2014-01-16 12:57:47.897466] W [client-rpc-fops.c:1529:client3_3_inodelk_cbk]
> 0-gfs35b1-client-0: remote operation failed: Socket is not connected
> [2014-01-16 12:57:47.897643] E [rpc-clnt.c:369:saved_frames_unwind]
> 0-gfs35b1-client-0: forced unwinding frame type(GlusterFS 3.3)
> op(INODELK(29)) called at 2014-01-16 12:56:33.651998 (xid=0xded6c3)
> [2014-01-16 12:57:47.897756] W [client-rpc-fops.c:1529:client3_3_inodelk_cbk]
> 0-gfs35b1-client-0: remote operation failed: Socket is not connected
> (many times, andit never reconects)
> 
> Brick log:
> 
> [2014-01-16 12:57:48.036133] I [client_t.c:166:gf_client_get]
> 0-gfs35b1-server:
> client_uid=bacasable.example.net-17263-2014/01/15-18:24:08:99795-gfs35b1-client-0-0
> [2014-01-16 12:57:48.041763] I [server-handshake.c:575:server_setvolume]
> 0-gfs35b1-server: accepted client from
> bacasable.example.net-17263-2014/01/15-18:24:08:99795-gfs35b1-client-0-0
> (version: 3.5beta1)
> [2014-01-16 12:58:52.609335] W [socket.c:522:__socket_rwv]
> 0-tcp.gfs35b1-server: writev on 192.0.2.6:1018 failed (Broken pipe)
> [2014-01-16 12:58:52.828718] I [server.c:521:server_rpc_notify]
> 0-gfs35b1-server: disconnecting connectionfrom
> bacasable.example.net-17263-2014/01/15-18:24:08:99795-gfs35b1-client-0-0
> [2014-01-16 13:00:17.465098] I [client_t.c:166:gf_client_get]
> 0-gfs35b1-server:
> client_uid=bacasable.example.net-17263-2014/01/15-18:24:08:99795-gfs35b1-client-0-0
> [2014-01-16 13:00:17.465240] I [server-handshake.c:575:server_setvolume]
> 0-gfs35b1-server: accepted client from
> bacasable.example.net-17263-2014/01/15-18:24:08:99795-gfs35b1-client-0-0
> (version: 3.5beta1)
> [2014-01-16 13:03:07.220384] I [client_t.c:166:gf_client_get]
> 0-gfs35b1-server:
> client_uid=bacasable.example.net-17263-2014/01/15-18:24:08:99795-gfs35b1-client-0-0
> [2014-01-16 13:03:07.220499] I [server-handshake.c:575:server_setvolume]
> 0-gfs35b1-server: accepted client from
> bacasable.example.net-17263-2014/01/15-18:24:08:99795-gfs35b1-client-0-0
> (version: 3.5beta1)
> [2014-01-16 13:05:56.938213] I [client_t.c:166:gf_client_get]
> 0-gfs35b1-server:
> client_uid=bacasable.example.net-17263-2014/01/15-18:24:08:99795-gfs35b1-client-0-0
> [2014-01-16 13:05:56.938357] I [server-handshake.c:575:server_setvolume]
> 0-gfs35b1-server: accepted client from
> bacasable.example.net-17263-2014/01/15-18:24:08:99795-gfs35b1-client-0-0
> (version: 3.5beta1)
> [2014-01-16 13:07:21.750081] I [client_t.c:166:gf_client_get]
> 0-gfs35b1-server:
> client_uid=bacasable.example.net-17263-2014/01/15-18:24:08:99795-gfs35b1-client-0-0
> [2014-01-16 13:07:21.768110] I [server-handshake.c:575:server_setvolume]
> 0-gfs35b1-server: accepted client from
> bacasable.example.net-17263-2014/01/15-18:24:08:99795-gfs35b1-client-0-0
> (version: 3.5beta1)
> [2014-01-16 13:10:11.457756] I [client_t.c:166:gf_client_get]
> 0-gfs35b1-server:
> client_uid=bacasable.example.net-17263-2014/01/15-18:24:08:99795-gfs35b1-client-0-0
> [2014-01-16 13:10:11.457897] I [server-handshake.c:575:server_setvolume]
> 0-gfs35b1-server: accepted client from
> bacasable.example.net-17263-2014/01/15-18:24:08:99795-gfs35b1-client-0-0
> (version: 3.5beta1)
> [2014-01-16 13:13:01.196338] I [client_t.c:166:gf_client_get]
> 0-gfs35b1-server:
> client_uid=bacasable.example.net-17263-2014/01/15-18:24:08:99795-gfs35b1-client-0-0
> [2014-01-16 13:13:01.196482] I [server-handshake.c:575:server_setvolume]
> 0-gfs35b1-server: accepted client from
> bacasable.example.net-17263-2014/01/15-18:24:08:99795-gfs35b1-client-0-0
> (version: 3.5beta1)
> 
> Any idea?
> 
> 
> --
> Emmanuel Dreyfus
> manu at netbsd.org
> 
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> https://lists.nongnu.org/mailman/listinfo/gluster-devel
>