[Gluster-devel] Buffer overrun in 3.4.0qa6

[Emmanuel Dreyfus]

Here is a crash server side with 3.4.0qa6. Note that frame is filled by
an ASCII string that obviously comes from a file on the glusterfs
volume: there has been buffer overrun somewhere.

I have tried to run glusterfs with electric fence in the past, without
any success. Is there a preferred way to debug that kind of thing?

The backtrace may be insightful to someone knowledgable.

crash at server.c:
136                     frame->next->prev = frame->prev;

#0  0xb98d7022 in FRAME_DESTROY (frame=0xbb786f20)
    at ../../../../libglusterfs/src/stack.h:136
#1  STACK_DESTROY (stack=0xba90757c)
    at ../../../../libglusterfs/src/stack.h:174
#2  server_submit_reply (frame=0xba907cd8, req=0xb94060c0,
arg=0xb13ff284, 
    payload=0x0, payloadcount=0, iobref=0xb020ab50, 
    xdrproc=0xbbb655c0 <xdr_gfs3_lookup_rsp>) at server.c:197
#3  0xb98eb329 in server_lookup_cbk (frame=0xba907cd8,
cookie=0xbb786eb0, 
    this=0xb9c28000, op_ret=0, op_errno=<optimized out>,
inode=0xb8b05bf4, 
    stbuf=0xb13ff910, xdata=0xba509258, postparent=0xb13ff8a8)
    at server-rpc-fops.c:164
#4  0xb9d2428e in io_stats_lookup_cbk (frame=0xbb786eb0,
cookie=0xbb7859b0, 
    this=0xb9c27000, op_ret=0, op_errno=0, inode=0xb8b05bf4,
buf=0xb13ff910, 
    xdata=0xba509258, postparent=0xb13ff8a8) at io-stats.c:1479
#5  0xb9d3126f in marker_lookup_cbk (frame=0xbb7859b0,
cookie=0xbb786d60, 
    this=0xb9c26000, op_ret=0, op_errno=0, inode=0xb8b05bf4,
buf=0xb13ff910, 
    dict=0xba509258, postparent=0xb13ff8a8) at marker.c:2211
#6  0xbbb99a13 in default_lookup_cbk (frame=0xbb786d60,
cookie=0xbb785550, 
    this=0xb9c25000, op_ret=0, op_errno=0, inode=0xb8b05bf4,
buf=0xb13ff910, 
    xdata=0xba509258, postparent=0xb13ff8a8) at defaults.c:37
#7  0xb9d509ec in iot_lookup_cbk (frame=0xbb785550, cookie=0xbb786270, 
    this=0xb9c24000, op_ret=0, op_errno=0, inode=0xb8b05bf4,
buf=0xb13ff910, 
    xdata=0xba509258, postparent=0xb13ff8a8) at io-threads.c:336
#8  0xb9d69d8d in pl_lookup_cbk (frame=0xbb786270, cookie=0xbb786040, 
    this=0xb9c23000, op_ret=0, op_errno=0, inode=0xb8b05bf4,
buf=0xb13ff910, 
    xdata=0xba509258, postparent=0xb13ff8a8) at posix.c:2020
#9  0xbb80b841 in posix_acl_lookup_cbk (frame=0xbb786040,
cookie=0xbb786f20, 
    this=0xb9c21000, op_ret=0, op_errno=0, inode=0xb8b05bf4,
buf=0xb13ff910, 
    xattr=0xba509258, postparent=0xb13ff8a8) at posix-acl.c:741
#10 0xb9d7e07b in posix_lookup (frame=0xbb786f20, this=0xb9c20000, 
    loc=0xba60c478, xdata=0xba509990) at posix.c:178
#11 0xbb809c27 in posix_acl_lookup (frame=0xbb786040, this=0xb9c21000, 
    loc=0xba60c478, xattr=0xba509990) at posix-acl.c:793
#12 0xb9d6452d in pl_lookup (frame=0xbb786270, this=0xb9c23000, 
    loc=0xba60c478, xdata=0xba509990) at posix.c:2062
#13 0xb9d541be in iot_lookup_wrapper (frame=0xbb785550, this=0xb9c24000,
    loc=0xba60c478, xdata=0xba509990) at io-threads.c:346
#14 0xbbbae3cb in call_resume_wind (stub=0xba60c458) at call-stub.c:2689
#15 call_resume (stub=0xba60c458) at call-stub.c:4142
#16 0xb9d545c5 in iot_worker (data=0xb9c42040) at io-threads.c:191

(gdb) print frame 
$1 = (call_frame_t *) 0xbb786f20
(gdb) print *frame
$2 = {root = 0x4e4f4c47, parent = 0x90a2c47, next = 0x45524353, 
  prev = 0x54092c47, local = 0x414f4c46, this = 0x44547c54, ret =
0x4c42554f, 
(...)

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu at netbsd.org