[Gluster-devel] Regression Voting Changes

[Nigel Babu]

On Wed, Jun 21, 2017 at 06:05:32PM +0530, Atin Mukherjee wrote:
> On Tue, Jun 20, 2017 at 9:17 AM, Nigel Babu <nigelb at redhat.com> wrote:
>
> > Hello folks,
> >
> > Amar has proposed[1] these changes in the past and I'd like to announce us
> > going
> > live with them as we've not received any strong feedback against it.
> >
> > ## Centos Regression
> > * On master, we only run tests/basic as pre-merge testing.
> >
>
> I should have read this and Amar's email thread more carefully but
> unfortunately I missed to read the above point in both the cases, so
> apologies. I think running only tests/basic on master is not *sufficient*
> given our goal is to have more test coverages coming from each patch and I
> expect most of the patches if not all will have tests added and if we don't
> run the full regression suite this eventually means we don't test the patch
> on master. Also I'm not sure how reactive we are against the regression
> test burn failure reports, so if things go bad and if we don't react to it
> immediately it'd be difficult to get the master branch back to stable
> state. I'd suggest (and request) that we should run the full regression
> test suite on Centos.

These are valid concerns. I don't have an easy solution to any of them. So I've
reverted the Centos regression changes entirely.

> > * On release branches, we will run the entire suite of tests.
> > * Our regular regression-test-burn-in and regression-test-with-multiplex
> > will
> >   continue to run the full suite of tests as they currently do.
> >
> > ## NetBSD Regression
> > * We will not run a netbsd7-regression as required pre-merge test anymore.
> >   However, you should be able to trigger it with "recheck netbsd".
> > * A green NetBSD will no longer be required for merging patches, however
> > if you
> >   have a -1 vote, it will remain a blocker. This is so that reviewers can
> >   request a full NetBSD run, especially on release branches.
> > * We will do a periodic NetBSD regression run on all currently maintained
> >   branches (3.8, 3.10, and 3.11 at the moment) and master.
> >
> > ## Additional Changes
> > * As full regression runs per patch is run on release branches only (other
> > than
> >   the nightly on master), any failures need proper attention and possible
> > RCA.
> >   A re-trigger in the hopes of getting a green is no longer acceptable for
> >   release branches.
> > * fstat will soon track the regression-test-burn-in and
> >   regression-test-with-multiplex.
> > * As soon as we have the new jobs up, we'll add them to fstat so we can
> > track
> >   failure patterns.
> >
> > The CentOS changes are already in production. The NetBSD changes will land
> > in
> > production today.
> >
> > [1]: http://lists.gluster.org/pipermail/gluster-devel/2017-May/052868.html

--
nigelb