[Gluster-devel] Need help diagnosing regression-test crashes

[Atin Mukherjee]

On 04/11/2016 05:39 PM, Atin Mukherjee wrote:
> 
> 
> On 04/11/2016 05:27 PM, Atin Mukherjee wrote:
>>
>>
>> On 04/09/2016 05:47 PM, Jeff Darcy wrote:
>>>>> atinm: Can you please look into the crash in the following test case?
>>>>> bugs/snapshot/bug-1322772-real-path-fix-for-snapshot.t
>>>> Do we have the link to the crash?
>>>
>>> The last two I looked at were these:
>>>
>>> https://build.gluster.org/job/rackspace-regression-2GB-triggered/19604/consoleFull
>>> https://build.gluster.org/job/rackspace-regression-2GB-triggered/19607/consoleFull
>>>
>>> We should probably move this part to bugzilla.  It's just some garden
>>> variety bug, I'm sure, not a flaw in our test infrastructure.
>> I tried to start debugging the crash. The issue here is the log tar for
>> this particular bug is empty for both the links, not sure how we ended
>> up in this case (rtalur, any idea?). From the backtrace it looks like
>> the glusterfsd process has crashed during cleanup_and_exit from
>> pmap_signout_cbk which means 'pkill gluster' was the trigger point for
>> this crash. However having logs would ease the debugging process.
> Ok, so Rtalur figured it out, since this test doesn't have a cleanup at
> the end the log continued in with bug-1002207.t. I'll continue to look
> into it and update once I have some significant data.
In bug-1322772-real-path-fix-for-snapshot.t I tried to simulate a reboot
of the node by removing all the snap bricks and post restart of glusterd
it should recreate the snap bricks which the test does with out any
issues. The mistake what I did here was I also unmounted the brick and
hence we lost all the xattrs set on this path and post restart of
glusterd the brick doesn't come up saying that it fails to retrieve the
volume id. Then it goes for a cleanup and from pmap_signout_cbk it
crashes. The reason of the crash is still under investigation however
[1] should ensure that brick process comes up post restarting glusterd
and we hit the happy path always.

[1] http://review.gluster.org/13974

~Atin
>>
>> Are you also unable to extract?
>>>
>>>
>>>
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>