[Gluster-Maintainers] proposal on few changes in regression voting to keep upstream 'master' fast moving
atumball at redhat.com
Mon May 22 09:23:39 UTC 2017
NOTE: Currently sent to only maintainers list, planning to send it to
'gluster-devel@' by Wednesday (24th) if there are no serious comments.
Below is mainly a proposal, and I would like to hear people's thought on
- Over the years, we have added many test cases to our regression test
suite, and currently the testing time stands at ~5hrs for 1 patch. But the
current way of '*.t' based regression can't be called as a 'complete' test
for the filesystem.
- Correct me if I am wrong, but even when we have to make a release,
other than the same set of tests, we don't have much to validate the build.
Now considering above points and taking the proposal of 'Good Build' from
, I am thinking of making below changes to how we look at testing and
'What to test on nightly build':
- Build verification
- Run all the regression as it runs now.
- Run CentOS regression
- Run NetBSD regression
- Run coverity
- Run gcov/lcov (for coverage)
- Run more tests with currently optional options made as default (like
- Open up the infra to community contribution, so anyone can write test
cases to make sure GlusterFS passes their usecases, everynight.
- Should be possible to run a python script, ruby script, or a bash
script, need not be in a 'prove' like setup.
- <Add more here>
- make the overall regression lightweight.
- Run what netbsd tests run now (ie, basic and features in tests).
- Don't run netbsd builds, instead add a compilation test on centos
32bit machine to keep reminding ourself how many warnings we get.
- Make sure 'master' branch is well tested in 'Nightly'.
- Let the approach of maintainers and over all project is to promote new
changes, instead of being very sceptical about new patches, ideas.
- Provide option to run the whole nightly build suit with a given
patchset to maintainers, so when in doubt, they can ask for the build to
complete before merging. Mostly applies to new feature or some changes
which change the way things behave fundamentally.
- During the release-plan come out with target number of 'coverity'
issues, and line coverage % to meet. Also consider number of 'glusto-tests'
- Agree to branch out early (at least 45days, compared to current
30days), so we can iron-out the issues caused by the making the 'master'
branch process lean.
- Change the voting logic, add more tests now (Example: fall back to
current regression suit).
- On the first build, run agreed performance tests and compare the
numbers with previous versions.
- Run NetBSD regression now.
- Btw, noticed the latest NetBSD package is for 3.8.9 (done in Jan).
- Work with Emmanuel <manu at netbsd.org> for better options here.
- Run nightly on the branch.
- Block the release till we meet the initially agreed metrics during the
release-plan. (like coverity/glusto-tests, line coverage etc).
- On the final build, run agree performance tests and publish the
numbers. Make sure it is not regressed.
We all agreeing to this is critical, as I saw that (refer mail on 90days
there were more than 600 patches which were old, and many of these are good
patches which would make the project better.
Please take time to review the points here and comment if any. Planning to
raise a ticket to Infra team to validate these changes by June 1st. Lets
move towards these changes by June 15th if there are not any serious
 - http://lists.gluster.org/pipermail/gluster-devel/2017-May/052844.html
Amar Tumballi (amarts)
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the maintainers