[Gluster-devel] Need help diagnosing regression-test crashes

[Atin Mukherjee]

-Atin
Sent from one plus one
On 09-Apr-2016 9:57 am, "Vijay Bellur" <vbellur at redhat.com> wrote:
>
> On 04/09/2016 12:17 AM, Atin Mukherjee wrote:
>>
>> -Atin
>> Sent from one plus one
>> On 09-Apr-2016 9:32 am, "Rajesh Joseph" <rjoseph at redhat.com
>> <mailto:rjoseph at redhat.com>> wrote:
>>  >
>>  >
>>  >
>>  > On Sat, Apr 9, 2016 at 2:05 AM, Jeff Darcy <jdarcy at redhat.com
>> <mailto:jdarcy at redhat.com>> wrote:
>>  >>
>>  >> Upon further investigation, I've been able to determine that the
problem
>>  >> lies in this line of our generic cleanup routine.
>>  >>
>>  >>         type cleanup_lvm &>/dev/null && cleanup_lvm || true;
>>  >>
>>  >> This works great if snapshot.rc we're at the end of a test that
included
>>  >> snapshot.rc (which defines cleanup_lvm), but we've generally been
moving
>>  >> away from that in favor of calling it only at the beginning.  Thus,
when
>>  >> we go from a snapshot test to a non-snapshot test, the cleanup at the
>>  >> beginning of the latter does *not* clean up any LVM stuff that's left
>>  >> over.  What might have been a simple and correctly attributed
failure in
>>  >> the snapshot test can instead show up later.  In this case, the
sequence
>>  >> of events is as follows:
>>  >>
>>  >>  1) bug-1322772 (snapshot) test starts glusterd
>>  >>
>>  >>  2) bug-1322772 exits while the new glusterd is still initializing
>>  >>
>>  >>  3) run-tests.sh looks for new core files and finds none
>>  >>
>>  >>  4) run-tests.sh starts bug-1002207 (stripe) test
>>  >>
>>  >>  5) glusterd from bug-1322772 dumps core
>>  >>
>>  >>  6) bug-1002207 test completes
>>  >>
>>  >>  7) run-tests.sh sees new core and misattributes it to bug-1002207
>>  >>
>>  >> The question is what to do about this.  Unconditionally calling
>>  >> lvm_cleanup from generic cleanup is simple, but might make regression
>>  >> tests noticeably slower.  Another possibility would be to change all
>>  >> snapshot tests to call cleanup (or at least cleanup_lvm) at the end,
or
>>  >> use bash's "trap" mechanism to ensure the same.  I'm not wild about
any
>>  >> of those, but lean toward the "trap" approach.  Anyone else have any
>>  >> opinions?
>>  >
>>  >
>>  > I think each snapshot test script should call cleanup_lvm and trap is
a
>>  > great suggestion.
>>  >
>>  > atinm: Can you please look into the crash in the following test case?
>>  > bugs/snapshot/bug-1322772-real-path-fix-for-snapshot.t
>>
>> Do we have the link to the crash?
>
>
> OT - Possibly unrelated glusterd crash in mainline [1]. This needs some
attention too.
Sure, will take a look.
>
> -Vijay
>
> [1] http://www.gluster.org/pipermail/maintainers/2016-April/000619.html
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20160409/169a9604/attachment.html>