[Gluster-devel] Nightly Pipeline Plans

[Niels de Vos]

On Thu, Dec 15, 2016 at 05:20:26PM +0530, Nigel Babu wrote:
> Hello folks,
> 
> This is a request for feedback for this plan to build a pipeline that will go
> from a commit to builds that are tested and signed. While the
> nightly.gluster.org site will be up by the time 3.10 is released, we will not
> yet be automating releases with this pipeline by then.

Could you explain what distributions, versions and architectures you can
cover with this? I suspect that the different distributions that provide
packages for the glusterfs packages have a broader range than what we
can test in our environment?

Also, builds for distributions normally need to be done in the build
system of those distributions. They will not accept builds from external
environments... I am not sure if/how this can become a replacement for
packages that are part of the distributions.

To me this looks like a lot of duplication of the plans with the builds
done through the CentOS CI and have the different CentOS SIGs consume
those builds for their testing. Has there been any thought about running
tests against other projects that use Gluster?

We've already discussed that the CentOS Build System can be used for
doing nightly builds by bots, and have these progress from testing to
release repositories. At least for the RPMs that are used by CentOS (and
possibly RHEL) users, I would much prefer this approach.

Thanks,
Niels


> # Why
> * Create a trusted way to build and distribute Gluster.
> * Get our distribution packaging to be less of a mess.
> * Automate the hell out of our packaging system so we're not caught out of the
>   blue with packaging bugs.
> * Fix security bugs discovered via automation so that distributions do not red
>   flag us.
> * In long-term, this pipeline will replace our current release process.
> 
> # How
> * Do a nightly build.
> * Test the nightly build. To begin with, use Glusto tests (add Coverty,
>   rpmlint, and other automated tests in the future).
> * If the test passes, build packages from that commit, sign it, and make them
>   available on nightly.gluster.org as a repo.
> 
> # Potential Issues
> * We want to build fresh packages from the known good commit rather than
>   copying the packages for security reasons.
> * Password-less gpg and security. The solution needs to work for .deb and .rpm
> 
> # Detailed Steps
> * Trigger the nightly build from build.gluster.org
> * Once that passes, trigger a glusto build.
> * Once that passes, trigger a new RPM build off separate build nodes.
> * The signing machine will pick up the new packages, sign them, and push them
>   to nightly.gluster.org
> * The signing machine needs to be disconnected from the internet entirely.
> 
> # Quantum of Work
> * New Jenkins jobs to trigger the process
> * A key signing machine isolated from the internet.
> * The code to take built packages, sign them, and push them out.
> * A machine to deliver the packages running chacra[1].
> 
> [1]: https://github.com/ceph/chacra
> 
> --
> nigelb



> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: not available
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20161215/ad4f2e0f/attachment.sig>