[Gluster-devel] Reducing regression runs (hopefully)

[Raghavendra Talur]

On Mon, Jul 25, 2016 at 6:08 PM, Jeff Darcy <jdarcy at redhat.com> wrote:

> > I have a few proposals to reduce this turn around time:
> >
> > 1. We do not clear the Verified tag. This means if you want to re-run
> >    regressions you have to manually trigger it. If your patch is rebased
> on
> >    top
> >    of another patch, you may have to retrigger failing regressions
> manually.
> > 2. We do give automatic +1 for regressions if the change is *only* in
> >    `extras/`, to MAINTAINERS file and other no-op changes. Please
> correct me
> >    here. I think the changes here do not affect regressions. If I'm
> wrong and
> >    they do, I'd love to know which files do affect regressions. I've
> taken
> >    the
> >    MAINTAINERS file as an example, I'm also curious to know what other
> no-op
> >    changes can be made.
>
> I think you're on the right track here, Nigel.  :)  I'd also add that
> changes
> to one .t file shouldn't require that any others be run (the same is not
> true
> of .rc files though).
>

+1


>
> On the more ambitious side, I think we could also optimize around the idea
> that some parts of the system aren't even present for some tests.  For
> example, most of our tests don't even use GFAPI and won't be affected by a
> GFAPI-only change.  Conversely, GFAPI tests won't be affected by a
> FUSE-only
> change.  AFR, EC, and JBR code and tests are mutually exclusive in much the
> same way.  We have the burn-in test to catch "stragglers" and git to revert
> any patch with surprising effects, so IMO (at least on master) we should be
> pretty aggressive about pruning out tests that provide little value.
>

 Yes. We introduced #G_TESTDEF_* keys in our .t files with the same idea.
Currently, we have two keys
a.  G_TESTDEF_TEST_STATUS_CENTOS6
b.  G_TESTDEF_TEST_STATUS_NETBSD7

Value for these keys is used by our framework to determine[1] whether to
run the test or not. It is fairly easy to add more keys. To take the same
example of a GFAPI change,
G_TESTDEF_TESTS_COMPONENTS="gfapi,dht,afr,"
G_TESTDEF_NO_IMPACT_COMPONENTS="fuse"

Combination of values from these two keys would decide whether to trigger
the test for a change in a component or not.

If you wish to provide such feature and implement in any different way,
ensure that such feature should be usable by the developer on his/her
laptop. Hence, it is better if it is developed in the framework than in
jenkins/gerrit configuration.


[1] http://review.gluster.org/#/c/13393/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20160725/6c551a5e/attachment.html>