[Gluster-infra] Updating /opt/qa and installing dependencies

[Michael Scherer]

Le lundi 18 juillet 2016 à 09:51 +0530, Nigel Babu a écrit :
> On Fri, Jul 15, 2016 at 04:45:24PM +0200, Michael Scherer wrote:
> > Le vendredi 15 juillet 2016 à 11:43 +0530, Nigel Babu a écrit :
> > > Hello,
> > >
> > > I'm toying with the idea that each job will have commands to install
> > > dependencies and update the /opt/qa folder. This will make the whole process of
> > > adding a new node in the testing pool trivial requiring not a lot of effort
> > > from the infrastructure team.
> >
> > I do think this will make things more complex.
> >
> > Now, we have 1 ansible playbook to make a working jenkins builder [1],
> > and later, we would have a half playbook, half jenkins jobs.
> >
> > We would still need to have stuff in ansible, because we need to
> > configure the VM before leting jenkins run (installing java, setting the
> > user and ssh keys, for a start, removing extra network card and ipv6 for
> > more complicated stuff).
> >
> > What advantages would it bring to convert the existing work to jenkins
> > jobs that we do not already have ?
> >
> > > However, if we break the /opt/qa repo, we'll spread the breakage everywhere.
> > > The fallback for errors would be to push a revert. Thoughts on this idea?
> >
> > [1]
> > https://github.com/gluster/gluster.org_ansible_configuration/tree/master/roles/jenkins_builder
> > --
> > Michael Scherer
> > Sysadmin, Community Infrastructure and Platform, OSAS
> >
> >
> 
> Here's what I'm thinking. I'm going to redefine our pool so we have a centos
> pool. All linux jobs run on this pool. Ansible will be used to bootstrap these
> machines. But in case we add a new job which needs a new package installed for
> that particular test, the changes need to be made in both places, but you are
> not blocked by the ansible run to deploy the new test. It can be pushed to
> Jenkins immediately after merging.

So that would be to remove friction for something that do not happen
that much in practice, if I am not wrong.

Because I still do not see how this is worth the extra complexity on
ansible playbooks side, nor why we can't ask to people to push a ansible
patch at the same time as a job. 

> Primarily, I want to remove any friction in deploying new tests.
> 
> This means two things:
> 1. The /opt/qa folder needs to be updated at the start of every run.
> 2. Install any dependencies before the test. Most likely not needed.

The 1 is ok. We will see right away when stuff broke (like on freebsd)

But I am not ok for 2 to be done by the job itself. 
People should put a PR on the ansible repo, and it will be deployed once
merged as well. 

This will make sure that we have one specific place to look for all
package installation, and will avoid hidden missing deps (like "job A
require rpm B, job C requires rpm B too, but since job A is everywhere,
no one ever add rpm B to job C, and so the day we remove job A, job C is
suddenly incomplete").

This will also be slightly faster to not check the package db on every
run, even if I recognize this is not gonna have a huge impact on perf.

And if we ever start to need to add more repo than EPEL (since EPEL do
not seems to provides the right go packages for some tests that were
proposed), I rather have a good overview of that to avoid conflict in
the long term.

> I'm open to other solutions that will reduce this friction without needing
> manual intervention.

Making a PR to ansible is not more a manual intervention than making a
PR for a job.

-- 
Michael Scherer
Sysadmin, Community Infrastructure and Platform, OSAS


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part
URL: <http://www.gluster.org/pipermail/gluster-infra/attachments/20160718/0de4d3e2/attachment.sig>