[Gluster-devel] stale list (3.4.0qa5)

Mon Dec 17 16:15:11 UTC 2012

Hi

Here is a crash with 3.4.0qa5, ir may ring a bell:

Core was generated by `glusterfsd'.
Program terminated with signal 11, Segmentation fault.
#0  0xb9e6be99 in list_del (old=0xa26e3854)
    at ../../../../libglusterfs/src/list.h:51
51              old->prev->next = old->next;
(gdb) print old
$1 = (struct list_head *) 0xa26e3854
(gdb) print *old
$2 = {next = 0xbabebabe, prev = 0xcafecafe}
(gdb) bt
#0  0xb9e6be99 in list_del (old=0xa26e3854)
    at ../../../../libglusterfs/src/list.h:51
#1  ioc_inode_destroy (ioc_inode=0xa26e3820) at ioc-inode.c:228
#2  0xb9e64265 in ioc_forget (this=0xba0e7000, inode=0xa26c4fa0)
    at io-cache.c:311
#3  0xbb7a420e in __inode_destroy (inode=0xa26c4fa0) at inode.c:319
#4  inode_table_prune (table=<optimized out>) at inode.c:1279
#5  0xbb7a433f in inode_unref (inode=0xa26c4fa0) at inode.c:444
#6  0xbb7b0ba0 in fd_destroy (fd=0xb7a1d3b8) at fd.c:516
#7  fd_unref (fd=0xb7a1d3b8) at fd.c:543
#8  0xb9e7fa23 in dht_local_wipe (this=0xba0e4000, local=0xb9490460)
    at dht-helper.c:190
#9  0xb9ea8a7d in dht_open (frame=0xbb3acca0, this=0xba0e4000, loc=0xa29f0010,
    flags=0, fd=0xb7a1d3b8, xdata=0x0) at dht-inode-read.c:124
#10 0xbb79d930 in default_open (frame=0xbb3acca0, this=0xba0e5000,
    loc=0xa29f0010, flags=0, fd=0xb7a1d3b8, xdata=0x0) at defaults.c:970
#11 0xb9e71063 in ra_open (frame=0xbb3ab5e0, this=0xba0e6000, loc=0xa29f0010,
    flags=0, fd=0xb7a1d3b8, xdata=0x0) at read-ahead.c:187
#12 0xb9e63306 in ioc_open (frame=0xbb3aae00, this=0xba0e7000, loc=0xa29f0010,
    flags=0, fd=0xb7a1d3b8, xdata=0x0) at io-cache.c:816
#13 0xb9e54510 in qr_open (frame=0xbb3ac060, this=0xba0e8000, loc=0xa29f0010,
    flags=0, fd=0xb7a1d3b8, xdata=0x0) at quick-read.c:805
#14 0xbb79d930 in default_open (frame=0xbb3ac060, this=0xba0e9000,
---Type <return> to continue, or q <return> to quit---
    loc=0xa29f0010, flags=0, fd=0xb7a1d3b8, xdata=0x0) at defaults.c:970
#15 0xb9e36565 in io_stats_open (frame=0xbb3ad640, this=0xba0ea000,
    loc=0xa29f0010, flags=0, fd=0xb7a1d3b8, xdata=0x0) at io-stats.c:2035
#16 0xb9fec9ac in fuse_open_resume (state=0xa29f0000) at fuse-bridge.c:1993
#17 0xb9fde905 in fuse_resolve_done (state=<optimized out>)
    at fuse-resolve.c:647
#18 fuse_resolve_all (state=<optimized out>) at fuse-resolve.c:676
#19 0xb9fde498 in fuse_resolve (state=0xa29f0000) at fuse-resolve.c:633
#20 0xb9fde93d in fuse_resolve_all (state=<optimized out>)
    at fuse-resolve.c:672
#21 0xb9fde1b7 in fuse_resolve_continue (state=0xa29f0000)
    at fuse-resolve.c:692
#22 0xb9fde2e2 in fuse_resolve_inode (state=0xa29f0000) at fuse-resolve.c:345
#23 0xb9fde735 in fuse_resolve (state=0xa29f0000) at fuse-resolve.c:630
#24 0xb9fde921 in fuse_resolve_all (state=<optimized out>)
    at fuse-resolve.c:665
#25 0xb9fdede7 in fuse_resolve_and_resume (state=0xa29f0000,
    fn=0xb9fec702 <fuse_open_resume>) at fuse-resolve.c:705
#26 0xb9fe2487 in fuse_open (this=0xba0b1000, finh=0xa2613110, msg=0xa2613138)
    at fuse-bridge.c:2009
#27 0xb9ff6f6e in fuse_thread_proc (data=0xba0b1000) at fuse-bridge.c:4417
#28 0xbb75a4ea in ?? () from /usr/lib/libpthread.so.1
#29 0xbb507ea0 in ___lwp_park50 () from /usr/lib/libc.so.12
---Type <return> to continue, or q <return> to quit---
#30 0xb7800000 in ?? ()
#31 0xbf7fd6f4 in ?? ()

client log before the crash:
[2012-12-17 07:33:45.657637] W [client-rpc-fops.c:1981:client3_3_setattr_cbk] 0-gfs33-client-2: remote operation failed: No such file or directory
[2012-12-17 07:33:45.658720] W [client-rpc-fops.c:1981:client3_3_setattr_cbk] 0-gfs33-client-3: remote operation failed: No such file or directory
[2012-12-17 07:33:45.660143] W [client-rpc-fops.c:1745:client3_3_xattrop_cbk] 0-gfs33-client-2: remote operation failed: No such file or directory. Path: (null) (--)
[2012-12-17 07:33:45.660344] W [client-rpc-fops.c:1745:client3_3_xattrop_cbk] 0-gfs33-client-3: remote operation failed: No such file or directory. Path: (null) (--)
[2012-12-17 07:33:45.672642] W [client-rpc-fops.c:1532:client3_3_inodelk_cbk] 0-gfs33-client-2: remote operation failed: No such file or directory
[2012-12-17 07:33:45.689535] W [client-rpc-fops.c:1532:client3_3_inodelk_cbk] 0-gfs33-client-3: remote operation failed: No such file or directory
[2012-12-17 07:33:45.694390] W [client-rpc-fops.c:1532:client3_3_inodelk_cbk] 0-gfs33-client-2: remote operation failed: No such file or directory
[2012-12-17 07:33:45.695337] W [client-rpc-fops.c:1532:client3_3_inodelk_cbk] 0-gfs33-client-3: remote operation failed: No such file or directory
[2012-12-17 07:33:45.695361] I [afr-lk-common.c:996:afr_lock_blocking] 0-gfs33-replicate-1: unable to lock on even one child
[2012-12-17 07:33:45.695380] I [afr-transaction.c:1043:afr_post_blocking_inodelk_cbk] 0-gfs33-replicate-1: Blocking inodelks failed.
[2012-12-17 07:33:45.695528] W [fuse-bridge.c:953:fuse_setattr_cbk] 0-glusterfs-fuse: 7750639: SETATTR() /manu/netbsd/usr/src/tools/mpfr/build/conf15916subs.sh =>-1 (No such file or directory)
[2012-12-17 07:35:15.115845] W [fuse-bridge.c:875:fuse_fd_cbk] 0-glusterfs-fuse:7762131: OPEN() <gfid:8088553f-3d8e-408a-8f37-e210db15b80a> => -1 (Invalid argument)
[2012-12-17 07:35:15.116648] W [defaults.c:1291:default_release]  0-fuse: xlatordoes not implement release_cbk
pending frames:
frame : type(0) op(0)
frame : type(0) op(0)
frame : type(1) op(SETATTR)
frame : type(1) op(SETATTR)
frame : type(1) op(SETATTR)
[dozens of them]
frame : type(1) op(SETATTR)
frame : type(1) op(SETATTR)
frame : type(1) op(FSETATTR)
frame : type(1) op(FSETATTR)
frame : type(1) op(FSETATTR)
frame : type(1) op(FSETATTR)
frame : type(0) op(0)

patchset: git://git.gluster.com/glusterfs.git
signal received: 11
time of crash: 2012-12-17 07:35:15configuration details:
dlfcn 1
fdatasync 1
libpthread 1
llistxattr 1
spinlock 1
extattr.h 1
xattr.h 1
st_atimespec.tv_nsec 1
package-string: glusterfs 3.4.0qa5

-- 
Emmanuel Dreyfus
manu at netbsd.org