[Gluster-devel] Reducing regression runs (hopefully)

[Raghavendra Talur]

>
> >
> > On the more ambitious side, I think we could also optimize around the
> idea
> > that some parts of the system aren't even present for some tests.  For
> > example, most of our tests don't even use GFAPI and won't be affected by
> a
> > GFAPI-only change.  Conversely, GFAPI tests won't be affected by a
> FUSE-only
> > change.  AFR, EC, and JBR code and tests are mutually exclusive in much
> the
> > same way.  We have the burn-in test to catch "stragglers" and git to
> revert
> > any patch with surprising effects, so IMO (at least on master) we should
> be
> > pretty aggressive about pruning out tests that provide little value.
>
> Raghavendra has been working on a patchset that does a better job of
> segregating tests by module. I'll let him explain the specifics.
>

 tests directory uses two different taxonomies today. One is by type of
tests where you find bugs/ performance/ basic/ etc. The other of type of
feature where you have features/ experimental/ bitrot/ georep/ etc. I am
working on moving all into first category where we have

1. functional/ basic
2. regression
3. performance (example: measure time for kernel untar with and without
patch)
4. integration (tests with samba/qemu/nfs-ganesha)
5. stress (ambitious for now, mentioned for completeness)
6. upgrade/update (we should not be breaking updates)


> My vision in this regard is something like this:
> * A patchset gets Verified +1.
> * A meta job is kicked off which determines regression jobs to run.
>   If the patch only touches GFAPI, we kick off the GFAPI regression tests.
> If
>   it touches multiple modules, we kick off the tests for these specific
>   modules.
>
As explained in the other reply this is not that trivial. Most of the
components do interact with each other and may break the functionality. It
is only modules which provide alternate functionality like AFR, EC and JBR
that can be tested exclusive manner.



> * The results for each platform are aggregated under our current labels
> similar
>   to how we aggregate results of multiple tests into one label for smoke.
>

This is possible.


>
> Am I being too ambitious here? Is there something I'm missing?
>
> --
> nigelb
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20160725/8094856c/attachment.html>