[Gluster-devel] Problem with smoke, regression ordering

[Pranith Kumar Karampuri]

On 07/06/2014 12:33 PM, Justin Clift wrote:
> On 05/07/2014, at 5:23 PM, Pranith Kumar Karampuri wrote:
>> hi Justin,
>>      If the regression results complete before the smoke test then 'green-tick-mark' is over-written and people don't realize that the regression succeeded by a simple glance at the list of patches. Can we do anything about it?
>
> Yeah.  At the moment, it's caused by people manually
> starting the old "regression" job on the build.gluster.org
> server.
>
> The build.gluster.org server can only run one thing at
> a time.  Everything else queues up.
>
> When the old regression test job runs, everything else is
> blocked until it finishes.  If there are a few regression
> tests lined up (or it hangs), then it can take hours until
> the smoke and rpm building jobs run.
>
> There are a few ways we could address this:
>
>   * Adjust the smoke test job so it runs on the Rackspace
>     slaves
>
>     Hopefully not hard.  But not sure.  We can try it out.
>
>   * Change the triggered regression test, so it doesn't
>     start automatically like this.
>
>     * We may be able to get a successful smoke test to
>       automatically trigger the regression run.  Ben
>       Turner would probably know how to make that work.
>
>     * Niels has suggested we might want to have the
>       regression test run when a +1 or +2 vote is given
>       instead.
Like Avati said a while back it depends on what you want to optimize on. 
Human review time (or) number of automatic regression job runs. I would 
like Human review time to be optimized by automatically triggering the 
regression runs and let the regressions catch some bugs. As a rule I 
don't review patches that are yet to pass regressions.

Pranith
>
>       I'm not really sure about this, because I wonder if
>       it's more useful to automatically test everything.
>
>         eg catching breakage early, before reviews are
>         done
>
>       I'm not strongly against it either though. ;)
>
>     Personally, I reckon we should have a discussion
>     on gluster-devel about this.  There might be really
>     good + / - for each, so a clear decision can be made.
>
>     And there may be other better ideas too.
>
> What're your thoughts on this stuff?
I like the present model. The only thing I feel needs a change is smoke 
test resetting the 'regression status'

Pranith
>
> Regards and best wishes,
>
> Justin Clift
>
> --
> GlusterFS - http://www.gluster.org
>
> An open source, distributed file system scaling to several
> petabytes, and handling thousands of clients.
>
> My personal twitter: twitter.com/realjustinclift
>