[Gluster-devel] Growing concern about regular testcase failures
justin at gluster.org
Wed Apr 30 13:53:43 UTC 2014
On 02/04/2014, at 7:52 PM, Harshavardhana wrote:
>> I've been running the regression tests in several different
>> environments, and some of the tests are very sensitive.
> Regression tests should be more stricter in their behavior, otherwise
> we have no way of knowing what is a stable patch. In future when we
> have 1000's of test cases - it could so happen that we will end up
> with a merged unstable patch.
Btw, some follow up here:
Weirdly, I didn't see the rest of your email below (or at least I
don't remember it. Maybe was too tired :<).
>> Test 65 of quota.t is a known bug (3.5 blocker). Varun is
>> aware of it:
> This is one bug which hits at random for pretty much any unrelated
> patch, if test case failure is the issue are we even sure quota is
> working as expected?
From memory, it was just an incorrect assumption in the test, and
it needed to wait for the storage to settle (there was a signal or
something it needed to wait for).
>> The rpm.t one is likely something to do with the environment,
>> (eg no tty), as when I run it manually it's fine. Haven't
>> gotten up to investigating it properly yet, but will soon.
> I agree since these are using pretty much a lot of RHEL/Fedora specific tools.
This one was due to a default setting (requiretty) in /etc/sudoers.
Changing that let it work. :)
>> The most difficult thing so far has been the lack of logging
>> for stdout and stderr. Everything useful seems to be redirected
>> to /dev/null instead of $TESTNUM.log (or $TESTNUM.stdout &
>> $TESTNUM.stderr). With the exception of rpm.t that is, which
>> has an optional DEBUG=1 flag.
> This can be indeed modified by redirection, let me see what i can do.
Did you have time to look into this?
>> Also non-great is not being able to effectively debug bash
>> scripts. eg set breakpoint, step, step, check variable, aha!
>> problem found, (etc). But, that's not as critical as the
>> lack of logging.
> This would be hard to do, but we can implement a way to using
> 'abrt-cli' to print a backtrace back to the "original patch" as
> comment. Since i don't have access to these systems, i wouldn't know
> how to do.
I can create an account for you on Rackspace (as a member of our
Gluster Community organisation account), and also show you how
to kick off the regression testing stuff remotely. The Python
code for it is here:
It's grown messy and needs refactoring at some point, but it works. :)
Open Source and Standards @ Red Hat
More information about the Gluster-devel