[Gluster-devel] glusterd crashes when synctask is used in conjunction with inner functions.

[Krishnan Parthasarathi]

----- Original Message -----
> From: "Jeff Darcy" <jdarcy at redhat.com>
> To: "Krishnan Parthasarathi" <kparthas at redhat.com>
> Cc: gluster-devel at nongnu.org
> Sent: Monday, October 8, 2012 11:16:11 PM
> Subject: Re: [Gluster-devel] glusterd crashes when synctask is used in conjunction with inner functions.
> 
> On 10/08/2012 01:21 PM, Krishnan Parthasarathi wrote:
> > FWIW, inner functions also make it harder to debug with gdb (read
> > breakpoints).
> > I am yet to demonstrably establish that inner functions' would
> > crash when run in synctask. I hope some of you might be able to
> > shed
> > light on how I could resolve that.
> 
> I think to answer that, we'd need to examine how inner functions are
> (or at
> least might be) implemented.  The fundamental problem is how the
> inner function
> actually accesses variables within the outer function's local scope.
>  For a
> direct call from outer to inner, it's pretty easy - it's just another
> set of
> stack offsets.  Otherwise, the inner function needs a pointer to the
> outer
> function's stack frame.  How does it get that?  Functions in between
> might be
> totally obvlivious to this pointer (it's not in their argument lists)
> so you
> can't just pass it directly.  They can't be stored directly in
> function
> pointers either, for similar reasons.  The only seemingly workable
> option would
> be to generate a "thunk" dynamically, something like this:
> 
> 	int
> 	real_inner_function (void *outer_frame, int arg)
> 	{
> 		/* Same as user's function but with extra arg. */
> 	}
> 
> 	int
> 	inner_function (int arg)
> 	{
> 		real_inner_function(COMPILE_TIME_CONSTANT,arg);

Shouldn't the outer frame's address be RUNTIME_CONSTANT? 

> 	}
> 
> So now a pointer to inner_function is actually a pointer to a
> specific
> invocation within the context of an outer-function call.  What
> happens when
> that outer function returns?  Uh oh.  Is any of this safe in the

In my patch, the outer function wouldn't return before the inner
function returns. I guess we could rule that out here.

> presence of
> both pthreads and ucontext threads?  Uh oh again, and we do use both
> in our
> synctask implementation.  Hmmm.  In a way, we're doing almost

An earlier gluster-devel discussion on pthread and ucontext threads
and long jmps if I am not wrong, lead to me hypothesise that inner functions
may not work well with synctask.

> everything we can
> to trip up gcc's inner-function implementation, so it's kind of no
> surprise
> that we've succeeded.
> 
> I haven't actually examined the gcc code to see if that's how inner
> functions
> are implemented.  There are other possibilities, but most are equally
> susceptible to our threading shenanigans.  The real proof point is
> whether the
> problems go away if we stop using inner functions.  There aren't too
> many uses

I do observe in my patch that the crash I observed goes away when I stop
using inner functions in the 'affected' code path. Which is why I was
convinced myself that I could submit my patch for review and root cause
why inner functions won't work with synctask asynchronously :-)

> currently, so it shouldn't be a prohibitively difficult experiment,
> and if
> you're right that inner functions to blame then we don't need to
> figure out
> why.  We should just stop using them.

Thanks for the explanation!
> 
> 		
> 
> 

~krish