[Gluster-Maintainers] [Proposal]: New branch (earlier: Changes to how we test and vote each patch)

Wed May 31 01:25:12 UTC 2017

All,

Here is another proposal, which I would like to start immediately to enable
many features possible for Gluster 4.0 release.

We will create a 'new' branch called '*experimental*', where anyone can
post a patch, for which only jenkins job which runs is 'smoke' (build +
fops validation).

   - This is where one can push any conceptual changes for review and get
   it merged quickly.
      - To make sure we don't delay, after every 1 week if there are no
      merges, a patch would be merged. (If there are no -1).
      - Manually, it can be merged even before.
   - Review comments on this branch is absolutely about concept and design,
   and no nitpicks.
   - even ./rfc.sh would be changed to allow (with a question) even in case
   of Errors.
   - All fundamental changes (like file type in GFID, DHT layout changes,
   etc) gets merged here first, and once baked can be taken as a single patch
   and get merged in master, which would make it to next release branch cut.
   - This branch would have life for 6 months, and after 6 months a new
   branch would be cut-off from master, so any features which can't be
   stabilized in 6 months, needs to be rebasing at least once in 6 months to
   new branch. This way, it would be much easier for everyone to experiment
   their ideas.
   - As this branch doesn't need a bugzilla ID to work, having github
   issues (for an RFE) would be important. This can fetch -1.
   - I am willing to maintain this branch to make sure it is in all its
   sanity, and keep track of patches which gets in.

Let me know what each of you would think.

On Wed, May 24, 2017 at 2:13 PM, Amar Tumballi <atumball at redhat.com> wrote:

> All,
>
> Below is mainly a proposal, and I would like to hear people's thought on
> these.
>
>
>    - Over the years, we have added many test cases to our regression test
>    suite, and currently the testing time stands at ~5hrs for 1 patch. But the
>    current way of '*.t' based regression can't be called as a 'complete' test
>    for the filesystem.
>
>
>    - Correct me if I am wrong, but even when we have to make a release,
>    other than the same set of tests, we don't have much to validate the build.
>       - Pranith / Aravinda, I heard there was some effort on these lines,
>       and you guys prepared a list during 3.9 cycle, share them if relevant.
>
> Now considering above points and taking the proposal of 'Good Build' from
> Nigel
> <http://lists.gluster.org/pipermail/gluster-devel/2017-March/052245.html>
> [1],  I am thinking of making below changes to how we look at testing and
> stability.
>
> *'What to test on nightly build':*
>
>    - Build verification
>    - Run all the regression as it runs now.
>       - Run CentOS regression
>       - Run NetBSD regression
>       - Run coverity
>    - Run gcov/lcov (for coverage)
>    - Run more tests with currently optional options made as default (like
>    brick-multiplexing etc).
>    - Open up the infra to community contribution, so anyone can write
>    test cases to make sure GlusterFS passes their usecases, everynight.
>       - Should be possible to run a python script, ruby script, or a bash
>       script, need not be in a 'prove' like setup.
>
>
> *'master' branch:*
>
>    - make the overall regression lightweight.
>       - Run what netbsd tests run now in CentOS regression (ie, basic and
>       features in tests).
>       - Don't run netbsd builds, instead add a compilation test on centos
>       32bit machine to keep reminding ourself how many warnings we get.
>    - Make sure 'master' branch is well tested in 'Nightly'.
>    - Let the approach of maintainers and over all project is to promote
>    new changes, instead of being very sceptical about new patches, ideas.
>    - Provide option to run the whole nightly build suit with a given
>    patchset to maintainers, so when in doubt, they can ask for the build to
>    complete before merging. Mostly applies to new feature or some changes
>    which change the way things behave fundamentally.
>
> *'release-x.y' branch:*
>
>    - During the release-plan come out with target number of 'coverity'
>    issues, and line coverage % to meet. Also consider number of 'glusto-tests'
>    to pass.
>    - Agree to branch out early (at least 45days, compared to current
>    30days), so we can iron-out the issues caused by the making the 'master'
>    branch process lean.
>
>
>    - Change the voting logic, add more tests now (Example: fall back to
>    current regression suit).
>    - On the first build, run agreed performance tests and compare the
>    numbers with previous versions.
>    - Run NetBSD regression now.
>       - Btw, noticed the latest NetBSD package is for 3.8.9 (done in Jan).
>       - Work with Emmanuel  <manu at netbsd.org> for better options here.
>    - Run nightly on the branch.
>    - Block the release till we meet the initially agreed metrics during
>    the release-plan. (like coverity/glusto-tests, line coverage etc).
>       - For 3.12 release itself we can fix the coverity completely, if
>       all 30+ developers we have spend a day fixing these issues. Most of it is
>       in general 1-2 line fixes.
>       - Improving line-coverage in tests is an on-going effort IMO.
>       - One of the suggestion was to 'Not' block release because of
>       these, I am fine this as a best effort based approach.
>       - *On the final release build:*
>       - Run agreed performance tests and publish the numbers. Make sure
>       it is not regressed.
>       - Add install and upgrade tests to this mix (install should be
>       covered when running above tests, but not upgrades)
>       - Upgrade tests to test out all(?) options
>       - Add, version compatibility tests
>       - Add testing documented procedures into the mix, IOW, if we state
>       "this is how you setup(/recover from/address) XYZ" then have a test case
>       for that. This ensures documented procedures are right, or are improved
>       over time.
>
>
> IMHO, We all proceeding on this is critical, as I saw that (refer mail on
> 90days old patches)
> <http://lists.gluster.org/pipermail/gluster-devel/2017-May/052844.html>[2],
> there were more than 600 patches which were old, and many of these are good
> patches which would make the project better.
>
> Please take time to review the points here and comment if any. Planning to
> raise a ticket to Infra team by June 1st. Lets move towards these changes
> by June 15th if there are not any serious concerns.
>
> [1] - http://lists.gluster.org/pipermail/gluster-devel/2017-March/
> 052245.html
> [2] - http://lists.gluster.org/pipermail/gluster-devel/2017-May/
> 052844.html
>
>
> Regards,
> Amar
> --
> Amar Tumballi (amarts)
>

-- 
Amar Tumballi (amarts)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/maintainers/attachments/20170531/ecca9007/attachment-0001.html>