[Bugs] [Bug 1655901] glusterfsd 5.1 crashes in socket.so

[bugzilla at redhat.com]

https://bugzilla.redhat.com/show_bug.cgi?id=1655901



--- Comment #3 from joao.bauto at neuro.fchampalimaud.org ---
I'm also getting a somewhat similar error in gluster 5.0 with multiple crashes
on different clients. Sometimes it takes a couple of days to crash or it can be
within hours. The mount error message is transport endpoint not connected and
it's fixed by unmount and mount again.

Here is the information on one of the clients with a volume mounted using
glusterfuse.

gluster setup:

Volume Name: tank
Type: Distribute
Volume ID: 9582685f-07fa-41fd-b9fc-ebab3a6989cf
Status: Started
Snapshot Count: 0
Number of Bricks: 8
Transport-type: tcp
Bricks:
Brick1: node-01:/tank/volume1/brick
Brick2: node-02:/tank/volume1/brick
Brick3: node-03:/tank/volume1/brick
Brick4: node-04:/tank/volume1/brick
Brick5: node-01:/tank/volume2/brick
Brick6: node-02:/tank/volume2/brick
Brick7: node-03:/tank/volume2/brick
Brick8: node-04:/tank/volume2/brick

installed packages:

glusterfs.x86_64                     5.0-1.el7                 @centos-gluster5 
glusterfs-api.x86_64                 5.0-1.el7                 @centos-gluster5 
glusterfs-cli.x86_64                 5.0-1.el7                 @centos-gluster5 
glusterfs-client-xlators.x86_64      5.0-1.el7                 @centos-gluster5 
glusterfs-fuse.x86_64                5.0-1.el7                 @centos-gluster5 
glusterfs-libs.x86_64                5.0-1.el7                 @centos-gluster5 
glusterfs-server.x86_64              5.0-1.el7                 @centos-gluster5

gdb core file:

#0  0x00007ff2c18f0cd9 in wb_fulfill_cbk () from
/usr/lib64/glusterfs/5.0/xlator/performance/write-behind.so
Missing separate debuginfos, use: debuginfo-install
glusterfs-server-5.0-1.el7.x86_64
(gdb) bt
#0  0x00007ff2c18f0cd9 in wb_fulfill_cbk () from
/usr/lib64/glusterfs/5.0/xlator/performance/write-behind.so
#1  0x00007ff2c1b725f9 in dht_writev_cbk () from
/usr/lib64/glusterfs/5.0/xlator/cluster/distribute.so
#2  0x00007ff2c1e142e5 in client4_0_writev_cbk () from
/usr/lib64/glusterfs/5.0/xlator/protocol/client.so
#3  0x00007ff2cf71cc70 in rpc_clnt_handle_reply () from /lib64/libgfrpc.so.0
#4  0x00007ff2cf71d043 in rpc_clnt_notify () from /lib64/libgfrpc.so.0
#5  0x00007ff2cf718f23 in rpc_transport_notify () from /lib64/libgfrpc.so.0
#6  0x00007ff2c430737b in socket_event_handler () from
/usr/lib64/glusterfs/5.0/rpc-transport/socket.so
#7  0x00007ff2cf9b45a9 in event_dispatch_epoll_worker () from
/lib64/libglusterfs.so.0
#8  0x00007ff2ce7b3e25 in start_thread (arg=0x7ff2ab7fe700) at
pthread_create.c:308
#9  0x00007ff2ce07cbad in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:113


gluster log:

[2018-12-13 10:08:15.916548] E [MSGID: 101191]
[event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch
handler
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker]
0-epoll: Failed to dispatch handler" repeated 1597 times between [2018-12-13
10:08:15.916548] and [2018-12-13 10:08:30.786295]
[2018-12-13 10:17:56.635788] E [MSGID: 101191]
[event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch
handler
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker]
0-epoll: Failed to dispatch handler" repeated 2572 times between [2018-12-13
10:17:56.635788] and [2018-12-13 10:18:04.789341]
pending frames:
frame : type(0) op(0)
frame : type(0) op(0)
patchset: git://git.gluster.org/glusterfs.git
signal received: 11
time of crash: 
2018-12-13 10:18:09
configuration details:
argp 1
backtrace 1
dlfcn 1
libpthread 1
llistxattr 1
setfsid 1
spinlock 1
epoll.h 1
xattr.h 1
st_atim.tv_nsec 1
package-string: glusterfs 5.0
/lib64/libglusterfs.so.0(+0x26570)[0x7ff2cf950570]
/lib64/libglusterfs.so.0(gf_print_trace+0x334)[0x7ff2cf95aae4]
/lib64/libc.so.6(+0x362f0)[0x7ff2cdfb42f0]
/usr/lib64/glusterfs/5.0/xlator/performance/write-behind.so(+0x9cd9)[0x7ff2c18f0cd9]
/usr/lib64/glusterfs/5.0/xlator/cluster/distribute.so(+0x745f9)[0x7ff2c1b725f9]
/usr/lib64/glusterfs/5.0/xlator/protocol/client.so(+0x5e2e5)[0x7ff2c1e142e5]
/lib64/libgfrpc.so.0(+0xec70)[0x7ff2cf71cc70]
/lib64/libgfrpc.so.0(+0xf043)[0x7ff2cf71d043]
/lib64/libgfrpc.so.0(rpc_transport_notify+0x23)[0x7ff2cf718f23]
/usr/lib64/glusterfs/5.0/rpc-transport/socket.so(+0xa37b)[0x7ff2c430737b]
/lib64/libglusterfs.so.0(+0x8a5a9)[0x7ff2cf9b45a9]
/lib64/libpthread.so.0(+0x7e25)[0x7ff2ce7b3e25]
/lib64/libc.so.6(clone+0x6d)[0x7ff2ce07cbad]

-- 
You are receiving this mail because:
You are on the CC list for the bug.
You are the assignee for the bug.