[Gluster-devel] test failure - tar on gluster

[Harris Landgarten]

Using mainline-2.4-patch-181

I tried the following

cd /mnt/glusterfs
sudo tar -cvjf test.tbz secondary/0/9

secondary/0/9 contains 62104 files ranging from 100b to 30mb in size

The idea of the test was to force reading and writing from the same gluster client for an extented period.

The test failed with the following from client and brick logs.

client:

[Jun 15 08:31:42] [DEBUG/tcp-client.c:54/do_handshake()] transport/tcp-client:dictionary length = 50
[Jun 15 08:42:39] [CRITICAL/client-protocol.c:218/call_bail()] client/protocol:bailing transport
[Jun 15 08:42:39] [DEBUG/tcp.c:123/cont_hand()] tcp:forcing poll/read/write to break on blocked socket (if any)
[Jun 15 08:42:39] [CRITICAL/client-protocol.c:218/call_bail()] client/protocol:bailing transport
[Jun 15 08:42:39] [DEBUG/tcp.c:123/cont_hand()] tcp:forcing poll/read/write to break on blocked socket (if any)
[Jun 15 08:42:39] [CRITICAL/client-protocol.c:218/call_bail()] client/protocol:bailing transport
[Jun 15 08:42:39] [DEBUG/tcp.c:123/cont_hand()] tcp:forcing poll/read/write to break on blocked socket (if any)
[Jun 15 08:42:39] [CRITICAL/client-protocol.c:218/call_bail()] client/protocol:bailing transport
[Jun 15 08:42:39] [DEBUG/tcp.c:123/cont_hand()] tcp:forcing poll/read/write to break on blocked socket (if any)
[Jun 15 08:42:39] [CRITICAL/client-protocol.c:218/call_bail()] client/protocol:bailing transport
[Jun 15 08:42:39] [DEBUG/tcp.c:123/cont_hand()] tcp:forcing poll/read/write to break on blocked socket (if any)
[Jun 15 08:42:39] [CRITICAL/client-protocol.c:218/call_bail()] client/protocol:bailing transport
[Jun 15 08:42:39] [DEBUG/tcp.c:123/cont_hand()] tcp:forcing poll/read/write to break on blocked socket (if any)
[Jun 15 08:42:39] [CRITICAL/client-protocol.c:218/call_bail()] client/protocol:bailing transport
[Jun 15 08:42:39] [DEBUG/tcp.c:123/cont_hand()] tcp:forcing poll/read/write to break on blocked socket (if any)
[Jun 15 08:42:39] [CRITICAL/client-protocol.c:218/call_bail()] client/protocol:bailing transport
[Jun 15 08:42:39] [DEBUG/tcp.c:123/cont_hand()] tcp:forcing poll/read/write to break on blocked socket (if any)
[Jun 15 08:42:39] [CRITICAL/client-protocol.c:218/call_bail()] client/protocol:bailing transport
[Jun 15 08:42:39] [DEBUG/tcp.c:123/cont_hand()] tcp:forcing poll/read/write to break on blocked socket (if any)
[Jun 15 08:42:39] [CRITICAL/client-protocol.c:218/call_bail()] client/protocol:bailing transport
[Jun 15 08:42:39] [DEBUG/tcp.c:123/cont_hand()] tcp:forcing poll/read/write to break on blocked socket (if any)
[Jun 15 08:42:39] [CRITICAL/client-protocol.c:218/call_bail()] client/protocol:bailing transport
[Jun 15 08:42:39] [DEBUG/tcp.c:123/cont_hand()] tcp:forcing poll/read/write to break on blocked socket (if any)
[Jun 15 08:42:39] [CRITICAL/client-protocol.c:218/call_bail()] client/protocol:bailing transport
[Jun 15 08:42:39] [DEBUG/tcp.c:123/cont_hand()] tcp:forcing poll/read/write to break on blocked socket (if any)
[Jun 15 08:42:39] [CRITICAL/client-protocol.c:218/call_bail()] client/protocol:bailing transport
[Jun 15 08:42:39] [DEBUG/tcp.c:123/cont_hand()] tcp:forcing poll/read/write to break on blocked socket (if any)
[Jun 15 08:42:39] [CRITICAL/client-protocol.c:218/call_bail()] client/protocol:bailing transport
[Jun 15 08:42:39] [DEBUG/tcp.c:123/cont_hand()] tcp:forcing poll/read/write to break on blocked socket (if any)
[Jun 15 08:42:39] [CRITICAL/client-protocol.c:218/call_bail()] client/protocol:bailing transport
[Jun 15 08:42:39] [DEBUG/tcp.c:123/cont_hand()] tcp:forcing poll/read/write to break on blocked socket (if any)
[Jun 15 08:42:39] [CRITICAL/client-protocol.c:218/call_bail()] client/protocol:bailing transport
[Jun 15 08:42:39] [DEBUG/tcp.c:123/cont_hand()] tcp:forcing poll/read/write to break on blocked socket (if any)
[Jun 15 08:42:39] [CRITICAL/client-protocol.c:218/call_bail()] client/protocol:bailing transport
[Jun 15 08:42:39] [DEBUG/tcp.c:123/cont_hand()] tcp:forcing poll/read/write to break on blocked socket (if any)
[Jun 15 08:42:39] [CRITICAL/client-protocol.c:218/call_bail()] client/protocol:bailing transport
[Jun 15 08:42:39] [DEBUG/tcp.c:123/cont_hand()] tcp:forcing poll/read/write to break on blocked socket (if any)
[Jun 15 08:42:39] [ERROR/common-utils.c:110/full_rwv()] libglusterfs:full_rwv: 91574 bytes r/w instead of 131418 (Broken pipe)
[Jun 15 08:42:39] [ERROR/client-protocol.c:204/client_protocol_xfer()] protocol/client:transport_submit failed
[Jun 15 08:42:39] [DEBUG/client-protocol.c:2609/client_protocol_cleanup()] protocol/client:cleaning up state in transport object 0x80773e8
[Jun 15 08:42:39] [DEBUG/client-protocol.c:2712/client_protocol_interpret()] protocol/client:frame not found for blk with callid: 62211
[Jun 15 08:42:39] [DEBUG/client-protocol.c:2609/client_protocol_cleanup()] protocol/client:cleaning up state in transport object 0x80773e8
[Jun 15 08:42:39] [CRITICAL/tcp.c:81/tcp_disconnect()] transport/tcp:client1: connection to server disconnected


Brick1:
[Jun 15 08:42:39] [ERROR/common-utils.c:110/full_rwv()] libglusterfs:full_rwv: 28656 bytes r/w instead of 65744 (Connection reset by peer)
[Jun 15 08:42:39] [ERROR/proto-srv.c:117/generic_reply()] protocol/server:transport_writev failed
[Jun 15 08:42:39] [ERROR/tcp.c:110/tcp_except()] transport/tcp:shutdown () - error: Transport endpoint is not connected
[Jun 15 08:42:39] [ERROR/common-utils.c:110/full_rwv()] libglusterfs:full_rwv: 0 bytes r/w instead of 65746 (Broken pipe)
[Jun 15 08:42:39] [ERROR/proto-srv.c:117/generic_reply()] protocol/server:transport_writev failed
[Jun 15 08:42:39] [ERROR/tcp.c:110/tcp_except()] transport/tcp:shutdown () - error: Transport endpoint is not connected
[Jun 15 08:42:39] [DEBUG/proto-srv.c:2826/open_file_cleanup_fn()] protocol/server:force releaseing file 0x8053e90
[Jun 15 08:42:39] [DEBUG/proto-srv.c:2826/open_file_cleanup_fn()] protocol/server:force releaseing file 0x8053068
[Jun 15 08:42:39] [DEBUG/proto-srv.c:2867/proto_srv_cleanup()] protocol/server:cleaned up xl_private of 0x8052f98
[Jun 15 08:42:39] [CRITICAL/tcp.c:81/tcp_disconnect()] transport/tcp:server: connection to server disconnected
[Jun 15 08:42:39] [DEBUG/tcp-server.c:229/gf_transport_fini()] tcp/server:destroying transport object for 10.255.62.81:1022 (fd=7)

Brick2:

no errors