[Gluster-devel] Coredump generated by bug-808400-stripe.t (lock migration?)

[Jeff Darcy]

>     (gdb) p *xattr
>     Cannot access memory at address 0xadc0de00007f288c
> 
> 
> This indicates a use-after-free problem. The 0xadc0de from *xattr would
> be "0xdeadc0de", just a little chopped of. This pattern is written to
> the area after GF_FREE() is called on a pointer when compiles with
> --enable-debug.
> 
> The question now is, why got this free'd, and if that is correct, why is
> the stripe xlator trying to free the xattr again?
> 
> At least, the above is my guessing. I can be wrong of course :-)
> 
> Who wants to look into this?

I looked into this a bit after it blew up here:

http://build.gluster.org/job/rackspace-regression-2GB-triggered/9955/consoleFull

Same patch, same test.  Building on what you found, I looked at the next
stack frame up.

(gdb) up
#2  0x00007f08a9917e7c in client3_3_fgetxattr_cbk (req=0x7f0890005ebc, 
    iov=0x7f0890005efc, count=1, myframe=0x7f089000570c)
    at /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/xlators/protocol/client/src/client-rpc-fops.c:1162

It's not just the xattr value that's messed up.  It's the whole frame.

(gdb) p *frame
$6 = {root = 0x36600000000000, parent = 0xadc0de00007f0890, 
  next = 0x27b7000000000de, prev = 0x3000007f08a4, local = 0xac0000000000, 
  this = 0x36600000000000, ret = 0xadc0de00007f0890, ref_count = 222, 
  lock = 41644032, cookie = 0x3000007f08a4, complete = _gf_false, op = 44032, 
  begin = {tv_sec = 15305201858641920, tv_usec = -5926493018029881200}, end = {
    tv_sec = 178859755513577694, tv_usec = 52776566458532}, 

None of these fields make sense.  There are three '0xdeadc0de' markers within the frame, indicating multiple free/reuse cycles.  What's especially weird is that all of these (plus some pointers) are offset by one.  My guess would be that the frame itself, or perhaps the entire call stack, was freed earlier as part of a premature STACK_UNWIND.  Unfortunately, the "variably sized type" nonsense is getting in the way of further debugging.  Nonetheless, the fact that both instances have been related to this particular patch might offer some avenues for investigation.