[Gluster-users] 1.4.0RC6 AFR problems (backtrace info attached)

Tue Dec 23 12:43:01 UTC 2008

here's the backtrace info from 2 of my crashes:  a logfile excerpt is 
at the end
(gdb) bt
#0  0x0000000000e6dbf2 in afr_truncate_wind (frame=0x7fada8ad9330,
     this=0xe6e770) at afr-inode-write.c:1145
#1  0x0000000000e72c7d in afr_write_pending_pre_op_cbk (frame=0x7fada8ad9330,
     cookie=0x8, this=0x185e740, op_ret=<value optimized out>,
     op_errno=<value optimized out>, xattr=<value optimized out>)
     at afr-transaction.c:431
#2  0x00000000001212e0 in default_xattrop_cbk (frame=<value optimized out>,
     cookie=<value optimized out>, this=<value optimized out>, op_ret=0,
     op_errno=-1465023696, dict=0xe72b30) at defaults.c:1015
#3  0x000000000060edb0 in posix_xattrop (frame=0x7fada8ad8f10, this=0x1857920,
     loc=0x7fada8ad96d0, optype=GF_XATTROP_ADD_ARRAY, xattr=0x7fada8ada440)
     at posix.c:2474
#4  0x0000000000122090 in default_xattrop (frame=0x7fada8ad79a0,
     this=0x185c9d0, loc=0x7fada8ad96d0, flags=GF_XATTROP_ADD_ARRAY,
     dict=0x7fada8ada440) at defaults.c:1026
#5  0x0000000000e7374b in afr_write_pending_pre_op (frame=0x7fada8ad9330,
     this=0x185e740) at afr-transaction.c:494
#6  0x0000000000e73985 in afr_lock_rec (frame=0x7fada8ad9330, this=0x185e740,
     child_index=2) at afr-transaction.c:690
#7  0x0000000000e74044 in afr_lock_cbk (frame=0x7fada8ad9330,
     cookie=<value optimized out>, this=0x185e740,
     op_ret=<value optimized out>, op_errno=0) at afr-transaction.c:617
#8  0x000000000081ed2c in pl_inodelk (frame=0x7fada8ad88d0, this=0x185c9d0,
     loc=<value optimized out>, cmd=7, flock=0x412e6e70) at internal.c:157
#9  0x0000000000e73d4b in afr_lock_rec (frame=0x7fada8ad9330,
     this=<value optimized out>, child_index=0) at afr-transaction.c:709
#10 0x0000000000e73f40 in afr_transaction (frame=0x7fada8ad9330,
     this=0x185e740, type=AFR_DATA_TRANSACTION) at afr-transaction.c:856
#11 0x0000000000e6f062 in afr_truncate (frame=0x7fada8ada480, this=0x185e740,
     loc=<value optimized out>, offset=0) at afr-inode-write.c:1229
#12 0x00000000018d6bc0 in fuse_setattr (req=<value optimized out>,
     ino=<value optimized out>, attr=0x412e7000, valid=<value optimized out>,
     fi=<value optimized out>) at fuse-bridge.c:810
#13 0x0000000001099173 in do_setattr (req=0x7fada8ad91f0, nodeid=214525249896,
     inarg=<value optimized out>) at fuse_lowlevel.c:486
#14 0x00000000018d7d35 in fuse_thread_proc (data=0x185f070)
     at fuse-bridge.c:2506
#15 0x00000031f360729a in start_thread () from /lib64/libpthread.so.0
#16 0x00000031f2ae439d in clone () from /lib64/libc.so.6


(gdb) bt
#0  0x0000000000e6dbf2 in afr_truncate_wind (frame=0x1917520, this=0xe6e770)
     at afr-inode-write.c:1145
#1  0x0000000000e72c7d in afr_write_pending_pre_op_cbk (frame=0x1917520,
     cookie=0x8, this=0x1718740, op_ret=<value optimized out>,
     op_errno=<value optimized out>, xattr=<value optimized out>)
     at afr-transaction.c:431
#2  0x00000000001212e0 in default_xattrop_cbk (frame=<value optimized out>,
     cookie=<value optimized out>, this=<value optimized out>, op_ret=0,
     op_errno=26304544, dict=0x0) at defaults.c:1015
#3  0x000000000060edb0 in posix_xattrop (frame=0x19176e0, this=0x1711920,
     loc=0x1918520, optype=GF_XATTROP_ADD_ARRAY, xattr=0x1917610)
     at posix.c:2474
#4  0x0000000000122090 in default_xattrop (frame=0x1915f60, this=0x17169d0,
     loc=0x1918520, flags=GF_XATTROP_ADD_ARRAY, dict=0x1917610)
     at defaults.c:1026
#5  0x0000000000e7374b in afr_write_pending_pre_op (frame=0x1917520,
     this=0x1718740) at afr-transaction.c:494
#6  0x0000000000e73985 in afr_lock_rec (frame=0x1917520, this=0x1718740,
     child_index=2) at afr-transaction.c:690
#7  0x0000000000e74044 in afr_lock_cbk (frame=0x1917520,
     cookie=<value optimized out>, this=0x1718740,
     op_ret=<value optimized out>, op_errno=0) at afr-transaction.c:617
#8  0x000000000081ed2c in pl_inodelk (frame=0x1915b00, this=0x17169d0,
     loc=<value optimized out>, cmd=7, flock=0x42f4be70) at internal.c:157
#9  0x0000000000e73d4b in afr_lock_rec (frame=0x1917520,
     this=<value optimized out>, child_index=0) at afr-transaction.c:709
#10 0x0000000000e73f40 in afr_transaction (frame=0x1917520, this=0x1718740,
     type=AFR_DATA_TRANSACTION) at afr-transaction.c:856
#11 0x0000000000e6f062 in afr_truncate (frame=0x19183c0, this=0x1718740,
     loc=<value optimized out>, offset=0) at afr-inode-write.c:1229
#12 0x0000000005f00bc0 in fuse_setattr (req=<value optimized out>,
     ino=<value optimized out>, attr=0x42f4c000, valid=<value optimized out>,
     fi=<value optimized out>) at fuse-bridge.c:810
#13 0x00000000039d7173 in do_setattr (req=0x19179f0, nodeid=214525249896,
     inarg=<value optimized out>) at fuse_lowlevel.c:486
#14 0x0000000005f01d35 in fuse_thread_proc (data=0x1719070)
     at fuse-bridge.c:2506
#15 0x00000031f360729a in start_thread () from /lib64/libpthread.so.0
#16 0x00000031f2ae439d in clone () from /lib64/libc.so.6


logfile excerpt:
  60: #end-volume
+-----
2008-12-23 00:28:38 E [socket.c:708:socket_connect_finish] home2: 
connection failed (Connection timed out)
pending frames:

Signal received: 11
configuration details:argp 1
backtrace 1
dlfcn 1
fdatasync 1
libpthread 1
llistxattr 1
setfsid 1
spinlock 1
epoll.h 1
xattr.h 1
tv_nsec 1
package-string: glusterfs 1.4.0rc6
/lib64/libc.so.6[0x31f2a322a0]
/usr/local/lib/glusterfs/1.4.0rc6/xlator/cluster/afr.so(afr_truncate_wind+0x72)[0xe6dbf2]
/usr/local/lib/glusterfs/1.4.0rc6/xlator/cluster/afr.so(afr_write_pending_pre_op_cbk+0xcd)[0xe72c7d]
/usr/local/lib/libglusterfs.so.0(default_xattrop_cbk+0x20)[0x1212e0]
/usr/local/lib/glusterfs/1.4.0rc6/xlator/storage/posix.so(posix_xattrop+0x1e0)[0x60edb0]
/usr/local/lib/libglusterfs.so.0(default_xattrop+0xc0)[0x122090]
/usr/local/lib/glusterfs/1.4.0rc6/xlator/cluster/afr.so(afr_write_pending_pre_op+0x4fb)[0xe7374b]
/usr/local/lib/glusterfs/1.4.0rc6/xlator/cluster/afr.so[0xe73985]
/usr/local/lib/glusterfs/1.4.0rc6/xlator/cluster/afr.so(afr_lock_cbk+0xa4)[0xe74044]
/usr/local/lib/glusterfs/1.4.0rc6/xlator/features/posix-locks.so(pl_inodelk+0x11c)[0x81ed2c]
/usr/local/lib/glusterfs/1.4.0rc6/xlator/cluster/afr.so[0xe73d4b]
/usr/local/lib/glusterfs/1.4.0rc6/xlator/cluster/afr.so(afr_transaction+0x110)[0xe73f40]
/usr/local/lib/glusterfs/1.4.0rc6/xlator/cluster/afr.so(afr_truncate+0x1f2)[0xe6f062]
/usr/local/lib/glusterfs/1.4.0rc6/xlator/mount/fuse.so[0x6753bc0]
/usr/local/lib/libfuse.so.2[0x1099173]
/usr/local/lib/glusterfs/1.4.0rc6/xlator/mount/fuse.so[0x6754d35]
/lib64/libpthread.so.0[0x31f360729a]
/lib64/libc.so.6(clone+0x6d)[0x31f2ae439d]
---------

Version      : glusterfs 1.4.0rc6 built on Dec 23 2008 00:22:39
TLA Revision : glusterfs--mainline--3.0--patch-792
Starting Time: 2008-12-23 00:41:14
Command line : /usr/local/sbin/glusterfs --log-level=WARNING 
--volfile=/etc/glusterfs/glusterfs-home.vol /home
given volfile
+-----