[Cinder.glusterfs.ci] [Third-party-announce] OpenStackCI2 at hds.com is disabled

Fri Jul 17 16:52:30 UTC 2015

Thanks!

On Fri, Jul 17, 2015 at 1:32 PM, Anita Kuno <anteaya at anteaya.info> wrote:

> On 07/14/2015 02:16 PM, Erlon Cruz wrote:
> > On Tue, Jul 14, 2015 at 1:06 PM, Anita Kuno <anteaya at anteaya.info>
> wrote:
> >
> >> On 07/14/2015 11:39 AM, Erlon Cruz wrote:
> >>> Hi Anita,
> >>>
> >>> I think I knew what happened, we had a power failure this weekend, and
> >> the
> >>> VM that was running the Jenkins master for this CI was automatically
> >>> booted, if you see on the CI reports all of then where sent on 13/07.
> >>
> >> Great are you able to take any steps so that if you have a power failure
> >> in future that a human needs to evaluate the system prior to it
> >> reporting on patches.
> >>
> >>
> > Sure, we have done it!
> >
> >
> >>>
> >>> Whats is there any criteria for removing CIs?
> >>
> >> This gerrit account has been disabled. The CI has not be removed. There
> >> is a difference.
> >>
> >> That was what I meant. Disable.
> >
> >
> >> The criteria for disabling CIs is meeting infra requirements as outlined
> >> here:
> >>
> http://docs.openstack.org/infra/system-config/third_party.html#requirements
> >> as well as infra having discretion at disabling a system at any time:
> >> "The OpenStack Infrastructure team disables mis-behaving third-party ci
> >> accounts at its discretion." from
> >>
> >>
> http://docs.openstack.org/infra/system-config/third_party.html#permissions-on-your-third-party-system
> >>
> >>> I fell this removal was very
> >>> unfair.
> >>
> >> That's fine and I appreciate you sharing your thoughts with me.
> >>
> >>
> >>> I have seen CIs reporting errors for weeks
> >>
> >> Have you communicated those errors to anyone? The core reviewers of the
> >> project? The operators of the CI in question? Shared them at a third
> >> party meeting?
> >>
> >>
> > Yes, we talked[1] to cinder guys and informed about the problem when it
> > happened, also informed about the patch that was ready to fix the
> > problem[2], that was only merged after I asked again.
> >
> >
> >> Part of the reason why my actions may seem so heavy handed is that my
> >> vision of CI operators working collaboratively with each other to ensure
> >> high quality of all systems has not come to pass. Quality as a whole is
> >> low and folks tend to ignore CI reports as noise, even the useful ones.
> >> As a survival technique I disable systems when I find out about them.
> >> This system came to my attention and I diabled it.
> >>
> >
> > We worked in a collaboration scheme since the problem came up. Since we
> > detected the problem, that was not our fault BTW, we are working with
> > Cinder team to get it fixed. Collaboration is opposed to the action that
> > was taken here. You see a problem and before informing the people that
> > could solve the problem or trying to understand the status of the problem
> > just disable the CI.
> >
> >
> >>
> >>
> >>
> http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2015-07-13.log.html#t2015-07-13T21:36:48
> >>
> >>
> >>
> >>> and then, only after
> >>> warnings to operators in this mailing list, they are removed.
> >>
> >> Nothing has been removed. Infra takes care of the gerrit accounts,
> >> nothing more.
> >>
> >>> In our case,
> >>> it was less than 24h, we got no warnings and got this account disabled.
> >>> Today, we can't test the CI against the patch that entered (yesterday)
> >> and
> >>> will have to do all the process to get the account again. Is it
> possible
> >> to
> >>> re-enable the account so we can continue the process to test the patch
> >> that
> >>> should fix the problem?
> >>
> >> Let's get some confirmation a power outage in future will have a
> >> different outcome.
> >>
> >
> > I have done this. If you re-enable it, it will run in silent mode until
> we
> > assure that it is stable enough to post to gerrit.
> >
> >
> >>
> >> Thanks Erlon,
> >> Anita.
> >>
> >>>
> >>> Thanks,
> >>> Erlon
> >>>
> >>> On Tue, Jul 14, 2015 at 12:06 PM, Anita Kuno <anteaya at anteaya.info>
> >> wrote:
> >>>
> >>>> On 07/14/2015 10:18 AM, Erlon Cruz wrote:
> >>>>> Hi Anita,
> >>>>>
> >>>>> There was a change in an external library (lockutils) that was
> causing
> >>>> the
> >>>>> patches run in our CI to fail. We have reported the problem[1],
> >>>> submitted a
> >>>>> fix[2], and shut down the CI while the fix wasn't merged.
> >>>>
> >>>> Okay well somehow you shut down your system but it was still leaving
> >>>> comments on gerrit patches. How did that happen?
> >>>>
> >>>> Thanks Erlon,
> >>>> Anita.
> >>>>
> >>>>
> >>>>> That was about 10
> >>>>> days ago, and the patch only landed yesterday. We will re-run the
> above
> >>>>> tests and submit the successful results.
> >>>>>
> >>>>> [1] https://bugs.launchpad.net/cinder/+bug/1470112
> >>>>> [2] https://review.openstack.org/#/c/197686/
> >>>>>
> >>>>> On Mon, Jul 13, 2015 at 7:06 PM, Anita Kuno <anteaya at anteaya.info>
> >>>> wrote:
> >>>>>
> >>>>>> https://review.openstack.org/#/c/199537/
> >>>>>>
> >>>>>> https://review.openstack.org/#/c/198853/
> >>>>>>
> >>>>>> https://review.openstack.org/#/c/193937/
> >>>>>>
> >>>>>> https://review.openstack.org/#/c/201241/
> >>>>>>
> >>>>>> This account is posting NOT_REGISTERED on patches.
> >>>>>>
> >>>>>> This is an indication that zuul (if you are using zuul) is unable to
> >>>>>> clearly communicate with Jenkins nodes. This is an internal error
> and
> >>>>>> has nothing to do with the patch.
> >>>>>>
> >>>>>> Please fix your system and post your incident report with details
> as a
> >>>>>> response.
> >>>>>>
> >>>>>> Please test your system on ci-sandbox after it is working again.
> >>>>>>
> >>>>>> Thank you,
> >>>>>> Anita.
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> Third-party-announce mailing list
> >>>>>> Third-party-announce at lists.openstack.org
> >>>>>>
> >>>>
> >>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/third-party-announce
> >>>>>> Please attend the third party meetings:
> >>>>>> http://eavesdrop.openstack.org/#Third_Party_Meeting
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> Third-party-announce mailing list
> >>>>> Third-party-announce at lists.openstack.org
> >>>>>
> >>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/third-party-announce
> >>>>> Please attend the third party meetings:
> >>>> http://eavesdrop.openstack.org/#Third_Party_Meeting
> >>>>>
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> Third-party-announce mailing list
> >>>> Third-party-announce at lists.openstack.org
> >>>>
> >>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/third-party-announce
> >>>> Please attend the third party meetings:
> >>>> http://eavesdrop.openstack.org/#Third_Party_Meeting
> >>>>
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> Third-party-announce mailing list
> >>> Third-party-announce at lists.openstack.org
> >>>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/third-party-announce
> >>> Please attend the third party meetings:
> >> http://eavesdrop.openstack.org/#Third_Party_Meeting
> >>>
> >>
> >>
> >> _______________________________________________
> >> Third-party-announce mailing list
> >> Third-party-announce at lists.openstack.org
> >>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/third-party-announce
> >> Please attend the third party meetings:
> >> http://eavesdrop.openstack.org/#Third_Party_Meeting
> >>
> >
> >
> > [1]
> >
> http://eavesdrop.openstack.org/irclogs/%23openstack-cinder/%23openstack-cinder.2015-07-02.log.html
> > [2
> >
> http://eavesdrop.openstack.org/irclogs/%23openstack-cinder/%23openstack-cinder.2015-07-08.log.html
> >
> >
> >
> > _______________________________________________
> > Third-party-announce mailing list
> > Third-party-announce at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/third-party-announce
> > Please attend the third party meetings:
> http://eavesdrop.openstack.org/#Third_Party_Meeting
> >
>
> This account is now re-enabled.
>
> In future please keep a conversation in the same thread, starting a new
> thread which is a part of the conversation is confusing when I go to
> look for information.
>
> Reference:
> lists.openstack.org/pipermail/third-party-announce/2015-July/000245.htmls
>
> Thank you Erlon,
> Anita.
>
> _______________________________________________
> Third-party-announce mailing list
> Third-party-announce at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/third-party-announce
> Please attend the third party meetings:
> http://eavesdrop.openstack.org/#Third_Party_Meeting
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/cinder.glusterfs.ci/attachments/20150717/cbb39f0a/attachment-0001.html>
-------------- next part --------------
_______________________________________________
Third-party-announce mailing list
Third-party-announce at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/third-party-announce
Please attend the third party meetings: http://eavesdrop.openstack.org/#Third_Party_Meeting