[Gluster-infra] Automatic nightly builds for glusterfs RPMs?
Niels de Vos
ndevos at redhat.com
Wed Jan 15 17:01:47 UTC 2014
On Tue, Jan 14, 2014 at 10:54:05PM +0530, Vijay Bellur wrote:
> On 01/14/2014 05:36 PM, Niels de Vos wrote:
> >On Mon, Jan 13, 2014 at 11:38:54PM +0530, Vijay Bellur wrote:
> >>On 01/13/2014 12:30 AM, Niels de Vos wrote:
> >>>Hi all,
> >>>during the last weekly meeting we touched upon creating nightly builds
> >>>from the glusterfs.git repository. It seems that Fedora offers a service
> >>>called COPR  that makes this really easy. Of course, this builds only
> >>>RPMs for Fedora and EPEL, but I think it is a good start.
> >>>IMHO we should aim for building packages from the main glusterfs.git
> >>>repository, and not pull in anything from the Fedora packaging
> >>>infrastructure (dist-git). There are some differences between the Fedora
> >>>packages and the community ones. In future, these should get minimized,
> >>>but we are a little more flexible with the community packages (less
> >>>strict packaging guidelines, making glusterfs-regression-test possible).
> >>>I've put a script together  that can be used to create the needed
> >>>SRPM, scp's it to a public download location, and triggers a COPR build.
> >>>This script has been used to create COPR repositories for the master
> >>>branch , release-3.5  and release-3.4 . Fedora 18-20 and EPEL-6
> >>>are currently available (x86_64 only). For an unknown reason, EPEL-5
> >>>fails to build :-/
> >>>It should be pretty easy to use the COPR repositories for automated
> >>>testing. The RPMs are not intended for general consumption. There are
> >>>some things that aren't too nice (like, master branch packages have
> >>>a version of 'date +%Y%m%d'). I have not tried to install any of the
> >>>generated RPMs yet. The main purpose of this exercise is to show whats
> >>>possible with minimal effort.
> >>>Any ideas, thoughts or other feedback is much appreciated. Things I'd
> >>>like to see answered are:
> >>>- Can something like this be used?
> >>I think we could give this a try.
> >>>- How/when should these packages get build?
> >>Nightly if commits have been merged in the last 24 hours.
> >Updated the script  with a "-H <HOURS>" switch. It defaults to 24
> >hours and will not trigger a build if 'git log --since=24hours' is
> >>>- Should this be run on build.gluster.org or download.gluster.org?
> >>We can possibly run the script as a jenkins job or through cron in
> >I can setup a cron-job on build.gluster.org that runs as my user. Any
> >particular time that would be most convenient?
> Don't seem to have a preference. Maybe 00:00 UTC ? :)
> >>>- Sync the repositories to download.gluster.org?
> >>+1. download.gluster.org is something that we can publish as the
> >>source for all nightly builds.
> >I dont know how (if) automatic pushes to download.gluster.org are done.
> >Maybe just setup an rsync or similar job that copies the COPR
> >repositories over?
Things that need to be done:
1. make sure that the following works on the server
- git clone
- make dist
- rpmbuild -ts *.tar.gz
-> build.gluster.org is fine for this
2. install copr-cli on the system
- depends on python-argparse and python-requests (from EPEL-6?)
- el6 package from http://people.fedoraproject.org/~devos/copr/
-> no el6 builds in Fedora EPEL (requested in Bug 1053142)
3. copy the API-key from http://copr.fedoraproject.org/api
-> done on my user account on build.gluster.org
4. setup a cronjob to trigger the builds
- start time 00:00 UTC
- use a fixed working directory for git caching (and verification?)
- use a shell script for sequential builds
(optionally the script can wait until a build has finished)
5. pick a system where the packages should be located
- pull the packages on download.gluster.org
- or, push them to download.gluster.org
- what target directory?
- add a README.WARNING to make clear that the packages are for
6. trigger a sync of the repos
- use lftp or something similar, this works for me:
$ lftp -e 'mirror --only-newer' \
Current blocking points:
- I need help/permission/agreement to do step 2
- Any input, recommendation for step 5 + 6
More information about the Gluster-infra