[Gluster-devel] Reconfigured Jenkins (smoke) jobs to not overwrite/reset regression-test results

[Atin Mukherjee]

On 30 May 2015 20:38, "Niels de Vos" <ndevos at redhat.com> wrote:
>
> On Sat, May 30, 2015 at 01:19:02PM +0200, Niels de Vos wrote:
> > Hi all,
> >
> > many of the Jenkins regression-tests have finished before all the smoke
> > jobs were done (we have fewer NetBSD slaves and the smoke tests were
> > waiting long in the queue). This caused the Verified +1 from the
> > regression-tests to be overwritten with Verified=0 once the smoke
> > results were available.
> >
> > The configuration change I have made in the several Jenkins jobs that
> > are doing smoke testing, is like this:
> >
> >   - open a Jenkins job configuration
> >   - scroll down to "Gerrit trigger"
> >   - click the [Advanced...] button
> >   - scroll down to "Gerrit Reporting Values"
> >   - enable "Skip Vote" for "Successful"
> >   - click the [Save] button on the bottom of the page
> >
> > This means that any successful result (Verified=0) will not get passed
> > on to Gerrit. If the regression test finished before, it will not be
> > overwritten anymore. This not-overwriting will be marked in a comment in
> > Gerrit with "SUCCESS (skipped)" next to the job. If a smoke test fails,
> > the failure will be reported and Verified=-1 gets set.
> >
> > Well, that is the/my theory at least. Let me know if you notice any
> > issues.
>
> Unfortunately, this does not seem to work :-/ When all smoke tests
> succeed and get marked as "skipped", the voting is still setting
> Verified=0 and with that overwriting a possible Verified=+/-1 from the
> regressions tests.
>
> So, the other solutions would be:
>
> a. chaining of Jenkins jobs (1. smoked -> 2. regressions)
> b. post results of the smoke test as different users
>
> The problem with "b" is that there is no requirement to wait for all
> responses from Jenkins. Once there is one Verified=+1, patches can get
> merged. Maintainers are supposed to check the passing of all required
> regression tests, but they do get missed on occasion.
Makes sense to me, if we can achieve 'a' then definitely its a better
option over 'b'.
>
> Ideas and suggestions welcome.
>
> Thanks,
> Niels
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20150530/66618d3d/attachment.html>