[Gluster-infra] [Gluster-devel] Unexpected jenkins restart

Michael Scherer mscherer at redhat.com
Tue May 26 21:00:13 UTC 2020


Le mardi 26 mai 2020 à 23:27 +0400, Smit Thakkar a écrit :
> Adding new builders can be done via the UI right?

That wouldn't be automated.

>  Can’t Jenkins API help
> with that too? As reloading jenkins everytime you add builder is a
> bit too
> much IMO

That's what the code is doing now (and working, minus 1 ansible issue):
https://github.com/gluster/gluster.org_ansible_configuration/commit/cfe6e823f7fe2e72c3f5a8f9983b241acfc27c9b

But still, if we do want to move to manage more than just jobs/nodes
with Ansible, I do think we need to find  a solution for that, or do it
another way (like, do everything using groovy, which seems to be how
people do config as code with jenkins).


> On Tue, May 26, 2020 at 15:04 Michael Scherer <mscherer at redhat.com>
> wrote:
> 
> > Le mardi 26 mai 2020 à 12:50 +0200, Michael Scherer a écrit :
> > > Le mardi 26 mai 2020 à 12:33 +0200, Michael Scherer a écrit :
> > > > Hi,
> > > > 
> > > > while working on the jenkins automation (mostly adding new host
> > > > with
> > > > CLI), I did a jenkins-cli reload-configuration. Turn out that
> > > > this
> > > > is,
> > > > contrary to what I expected, also seems to have blocked jenkins
> > > > somehow.
> > > > 
> > > > I am trying to figure exactly what is happening, cause I do not
> > > > expect
> > > > that to be so long.
> > > 
> > > My current hypothesis is that jenkins is still running, but
> > > currently
> > > reloading the config seems to imply "read all xml files in
> > > /var/lib/jenkins" or something. I do not know where this do come
> > > from,
> > > but this also mean that the server is busy parsing all builds
> > > results
> > > in /var/lib/jenkins/jobs, a 72G directory with around 148652
> > > files if
> > > I
> > > can trust find and wc.
> > > 
> > > I will dig a bit more before interrupting the server brutally
> > 
> > Seems jenkins did manage to come back by itself, after 20 minutes.
> > 
> > The good news is that it seems that after 3 days of work, we found
> > a
> > way (minus some ansible hack to be fixed) to dynamically add
> > builders.
> > The bad new is that jenkins reload is busted, cause 20 minutes of
> > downtime do not seems optimal at all, and we would need to fix that
> > before going on.
> > 
> > For folks who are eager on reading java, that's somewhere here:
> > 
> > 
> > https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/jenkins/model/Jenkins.java#L3234
> > 
> > ( from
> > ./core/src/main/java/hudson/cli/ReloadConfigurationCommand.java
> > to doReload, to reload to loadTasks).
> > 
> > --
> > Michael Scherer / He/Il/Er/Él
> > Sysadmin, Community Infrastructure
> > 
> > 
> > 
> > _______________________________________________
> > 
> > Community Meeting Calendar:
> > 
> > Schedule -
> > Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
> > Bridge: https://bluejeans.com/441850968
> > 
> > 
> > 
> > 
> > Gluster-devel mailing list
> > Gluster-devel at gluster.org
> > https://lists.gluster.org/mailman/listinfo/gluster-devel
> > 
> > 
-- 
Michael Scherer / He/Il/Er/Él
Sysadmin, Community Infrastructure



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part
URL: <http://lists.gluster.org/pipermail/gluster-infra/attachments/20200526/9debc1db/attachment.sig>


More information about the Gluster-infra mailing list