[Bugs] [Bug 1214169] glusterfsd crashed while rebalance and self-heal were in progress

[bugzilla at redhat.com]

https://bugzilla.redhat.com/show_bug.cgi?id=1214169



--- Comment #5 from Shruti Sampat <ssampat at redhat.com> ---
On the same volume, another brick process crashed after some time.

>From the brick logs -

[2015-04-22 12:31:23.720702] E [posix.c:4433:posix_removexattr] 0-vol-posix:
null gfid for path /linux-3.19.4/include/net/netprio_cgroup.h
frame : type(0) op(0)
frame : type(0) op(20)
frame : type(0) op(0)
frame : type(0) op(0)
frame : type(0) op(0)
frame : type(0) op(40)
patchset: git://git.gluster.com/glusterfs.git
signal received: 11
time of crash: 
2015-04-22 12:31:23
configuration details:
[2015-04-22 12:31:23.731428] E [posix.c:4433:posix_removexattr] 0-vol-posix:
null gfid for path /linux-3.19.4/include/net/raw.h
argp 1
backtrace 1
dlfcn 1
libpthread 1
llistxattr 1
setfsid 1
spinlock 1
epoll.h 1
xattr.h 1
st_atim.tv_nsec 1
package-string: glusterfs 3.7dev
/usr/lib64/libglusterfs.so.0(_gf_msg_backtrace_nomem+0xb6)[0x3ad30221c6]
/usr/lib64/libglusterfs.so.0(gf_print_trace+0x33f)[0x3ad303de2f]
/lib64/libc.so.6[0x3ad14326a0]
/lib64/libpthread.so.0(pthread_spin_lock+0x0)[0x3ad180c380]
/usr/lib64/glusterfs/3.7dev/xlator/features/quota.so(+0x589b)[0x7f484d94689b]
/usr/lib64/glusterfs/3.7dev/xlator/features/quota.so(quota_fill_inodectx+0x1fa)[0x7f484d94f6aa]
/usr/lib64/glusterfs/3.7dev/xlator/features/quota.so(quota_readdirp_cbk+0x13e)[0x7f484d94fa9e]
/usr/lib64/glusterfs/3.7dev/xlator/features/marker.so(marker_readdirp_cbk+0x13e)[0x7f484db71bbe]
/usr/lib64/libglusterfs.so.0(default_readdirp_cbk+0xc2)[0x3ad302e622]
/usr/lib64/glusterfs/3.7dev/xlator/features/locks.so(pl_readdirp_cbk+0x18b)[0x7f484e5b6cfb]
/usr/lib64/glusterfs/3.7dev/xlator/features/access-control.so(posix_acl_readdirp_cbk+0x27a)[0x7f484e7d0b7a]
/usr/lib64/glusterfs/3.7dev/xlator/features/bitrot-stub.so(br_stub_readdirp_cbk+0x214)[0x7f484e9db304]
/usr/lib64/glusterfs/3.7dev/xlator/storage/posix.so(posix_do_readdir+0x1b8)[0x7f484f871498]
/usr/lib64/glusterfs/3.7dev/xlator/storage/posix.so(posix_readdirp+0x1ee)[0x7f484f872fde]
/usr/lib64/libglusterfs.so.0(default_readdirp+0x83)[0x3ad3027333]
/usr/lib64/libglusterfs.so.0(default_readdirp+0x83)[0x3ad3027333]
/usr/lib64/libglusterfs.so.0(default_readdirp+0x83)[0x3ad3027333]
/usr/lib64/glusterfs/3.7dev/xlator/features/bitrot-stub.so(br_stub_readdirp+0x259)[0x7f484e9d8e29]
/usr/lib64/glusterfs/3.7dev/xlator/features/access-control.so(posix_acl_readdirp+0x19d)[0x7f484e7cd4bd]
/usr/lib64/glusterfs/3.7dev/xlator/features/locks.so(pl_readdirp+0x204)[0x7f484e5b5d94]
/usr/lib64/libglusterfs.so.0(default_readdirp+0x83)[0x3ad3027333]
/usr/lib64/libglusterfs.so.0(default_readdirp_resume+0x142)[0x3ad3029db2]
/usr/lib64/libglusterfs.so.0(call_resume+0x80)[0x3ad3046470]
/usr/lib64/glusterfs/3.7dev/xlator/performance/io-threads.so(iot_worker+0x158)[0x7f484e1a1388]
/lib64/libpthread.so.0[0x3ad18079d1]
/lib64/libc.so.6(clone+0x6d)[0x3ad14e88fd]
---------


While self-heal was running, I attempted to remove a directory on the mount
point.

# rm -fr linux-3.19.4
rm: cannot remove `linux-3.19.4/include/crypto': Directory not empty
rm: cannot remove `linux-3.19.4/include/drm': Directory not empty
rm: cannot remove `linux-3.19.4/include/media': Directory not empty
rm: cannot remove `linux-3.19.4/include/net/netfilter': Directory not empty
rm: cannot remove `linux-3.19.4/include/net/bluetooth': Directory not empty
rm: cannot remove `linux-3.19.4/include/net/caif': Directory not empty
rm: cannot remove `linux-3.19.4/include/net/tc_act': Directory not empty

I also see a lot of the following messages in brick logs -

<snip>

[2015-04-22 12:31:14.461836] E [posix.c:4433:posix_removexattr] 0-vol-posix:
null gfid for path /linux-3.19.4/include/net/netns
[2015-04-22 12:31:18.132176] E [posix.c:4433:posix_removexattr] 0-vol-posix:
null gfid for path /linux-3.19.4/include/net/caif/caif_layer.h
[2015-04-22 12:31:23.675448] E [posix.c:4433:posix_removexattr] 0-vol-posix:
null gfid for path /linux-3.19.4/include/net/ip6_fib.h
[2015-04-22 12:31:23.691089] E [posix.c:4433:posix_removexattr] 0-vol-posix:
null gfid for path /linux-3.19.4/include/net/ip_fib.h
[2015-04-22 12:31:23.699589] E [posix.c:4433:posix_removexattr] 0-vol-posix:
null gfid for path /linux-3.19.4/include/net/lib80211.h
[2015-04-22 12:31:23.711344] E [posix.c:4433:posix_removexattr] 0-vol-posix:
null gfid for path /linux-3.19.4/include/net/neighbour.h

</snip>

-- 
You are receiving this mail because:
You are on the CC list for the bug.
You are the assignee for the bug.