[Gluster-devel] WARNING: patches marked as Verified might not be

[Vijay Bellur]

On 04/17/2015 08:49 PM, Justin Clift wrote:
> On 17 Apr 2015, at 16:02, Jeff Darcy <jdarcy at redhat.com> wrote:
>> As mentioned in a previous email, recent changes to speed up regression
>> testing have uncovered a problem with how we track the verified status
>> of patches.  Specifically, the fault sequence is as follows:
>>
>> 1) Regression fails, "Gluster Build System" adds V-1
>>
>> 2) NetBSD regression passes, "NetBSD Build System" adds V+1
>>
>> 3) Smoke passes, "Gluster Build System" erases the V-1
>>
>> The order of the first two might be reversed.  What's new is that
>> regression never used to finish before smoke, and now it usually does
>> (for the failure case).  So, reviewers/committers, please be sure to
>> check the last *regression* result before assuming that green check mark
>> is valid.  I'm periodically going through the list of recently reviewed
>> patches and manually fixing up the status, but I can't catch all of them
>> and TBH don't know how much longer I'll keep trying.
>>
>> Gerrit/Jenkins maintainers: we have three options for a long term
>> solution.
>>
>> a) Run smoke and regression as separate Gerrit users, require
>>     concurrence for a patch to be considered V+1.  This is what we
>>     already do for NetBSD.  It's simple, and seems to work well.
>>
>> b) Stop triggering regression directly from the Gerrit event, trigger it
>>     from a successful smoke completion instead.  This is a bit more
>>     complicated, but has the additional benefit that regression *won't
>>     even run* if smoke fails (saving resources).
>>
>> c) Write a script to seek out and destroy improperly marked patches
>>     (basically automate what I've been doing the last coupld of days).
>>     It'll work, but it still leaves a window when patches are improperly
>>     marked.  We should only consider it if we run into significant
>>     problems with the other two approaches.
>>
>> Anyone else have any preference for (a) vs. (b)?  I can't implement (a)
>> myself.  I could implement (b), but I don't want to go messing with the
>> Jenkins configuration unless/until we have a consensus that it's the
>> right thing to do.
>
> (a) sounds good to me.  Vijay can setup the Gerrit account if he's
> agreeable. :)
>

A new user Gluster Smoke Test has been added to gerrit to run smoke 
tests. I will post an update after successfully integrating voting 
through this user.

Thanks,
Vijay