[Cinder.glusterfs.ci] [Third-party-announce] Cisco ml2 CI is disabled

Fri Jun 12 22:20:20 UTC 2015

Hello Anita

Thank you for pointing me out to the cacti and IRC log.

I did not change comment text of the "Cisco ml2 CI". If tests failed the CI should reply :
" Build failed. For help on isolating this failure, please contact cisco-openstack-neutron-ci at cisco.com. To re-run, post a 'recheck cisco-ml2' comment. "

I see there are guesses that looping might be caused by wrong " comment_filter " (regexp for check text) value. The CI uses:
        - event: comment-added
          comment_filter: (?i)^(Patch Set [0-9]+:\n\n)?\s*recheck(( (?:bug|lp)[\s#:]*(\d+))|( no bug))\s*$
        - event: comment-added
          comment_filter: (?i)^(Patch Set [0-9]+:\n\n)?\s*recheck cisco-ml2\s*$

My guess is: Jenkins died at " Jun 11, 2015 8:15:45 PM"  due to error='Cannot allocate memory'. And, maybe, it caused zuul to start looping. But I am not sure.

Could you advice further steps that help me to get the CI account enabled?

Thanks.

-----Original Message-----
From: Anita Kuno [mailto:anteaya at anteaya.info] 
Sent: Friday, June 12, 2015 11:13 PM
To: third-party-announce at lists.openstack.org
Subject: Re: [Third-party-announce] Cisco ml2 CI is disabled

On 06/12/2015 04:06 PM, Anita Kuno wrote:
> On 06/12/2015 03:47 PM, Nikolay Fedotov (nfedotov) wrote:
>> Hello All!
>>
>> I was wondering what kind of looping are you talking about? 
>>
>> I see the change https://review.openstack.org/#/c/188221/ has 5  patchset and each Cisco CI replied :
>> - Cisco Tail-f CI - 4 times
>> - Cisco N1kv CI - 4 times
>> - Cisco APIC CI - 4 times
>> - Cisco ml2 CI - 0 times
> 
> We had to perform database surgery to remove the hundreds of comments 
> left by Cisco ml2 CI. Trust us it was looping.
> 
> We also had to restart gerrit to close the connection as our gerrit 
> input spiked because of it. See the inbound spike from yesterday?
> http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=
> 34&rra_id=all
> that was you.
> 
> Your comment text changed yesterday sometime. No other patches were 
> affected by this.
> 
> Please confirm your comment text changed yesterday.
> 
> Thank you,
> Anita.

Also here is the log of the incident:

http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2015-06-11.log.html#t2015-06-11T20:34:08

> 
>>
>> So each CI systems tested every (except the 5th) patchset only once.
>>
>> Thanks.
>>
>> -----Original Message-----
>> From: Asselin, Ramy [mailto:ramy.asselin at hp.com]
>> Sent: Thursday, June 11, 2015 11:55 PM
>> To: Announcements for third party CI operators.
>> Subject: Re: [Third-party-announce] Cisco ml2 CI is disabled
>>
>> Wow, good call. We should document that and find a way to protect that from happening.
>>
>> -----Original Message-----
>> From: Doug Wiegley [mailto:dougw at a10networks.com]
>> Sent: Thursday, June 11, 2015 1:49 PM
>> To: Announcements for third party CI operators.
>> Subject: Re: [Third-party-announce] Cisco ml2 CI is disabled
>>
>> Not sure if this is your issue, but this looks similar to a looping issue that I inflicted on myself, by including my recheck text in my CI comment.
>>
>>
>> "Cisco ml2 CI2:22 PMPatch Set 4:
>> Build failed. For help on isolating this failure, please contact cisco-openstack-neutron-ci at cisco.com. To re-run, post a 'recheck cisco-ml2' comment.
>>
>> * ml2-nexus <http://128.107.233.10:8080/job/ml2-nexus/None> EXCEPTION
>> * python27 <http://128.107.233.10:8080/job/python27/225> SUCCESS in 6m 06s (non-voting) "
>>
>> Thanks,
>> Doug
>>
>>
>>
>>
>>
>> On 6/11/15, 2:40 PM, "Anita Kuno" <anteaya at anteaya.info> wrote:
>>
>>> https://review.openstack.org/#/c/188221/
>>>
>>> The patch speaks for itself.
>>>
>>> Thank you, Russell Bryant for the alert.
>>>
>>> Before this system is enabled we will need to hear the results of 
>>> your investigation as to why this happened as well as you assurance 
>>> that it won't happen again.
>>>
>>> Thank you,
>>> Anita.
>>>
>>> _______________________________________________
>>> Third-party-announce mailing list
>>> Third-party-announce at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/third-party-anno
>>> unc
>>> e
>> _______________________________________________
>> Third-party-announce mailing list
>> Third-party-announce at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/third-party-annou
>> nce
>>
>> _______________________________________________
>> Third-party-announce mailing list
>> Third-party-announce at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/third-party-annou
>> nce
>>
>> _______________________________________________
>> Third-party-announce mailing list
>> Third-party-announce at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/third-party-annou
>> nce
>>
> 

_______________________________________________
Third-party-announce mailing list
Third-party-announce at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/third-party-announce

_______________________________________________
Third-party-announce mailing list
Third-party-announce at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/third-party-announce