[Gluster-devel] Gluster testing matrix

[M S Vishwanath Bhat]

More subcategories to add as suggested:

1. RDMA transport ptotocol
2. Openstatck-Manila integration
3. IPv6 support

New test categories which was missed in the last mail

1. Static Code analysis (Coverity/SonarQube)
2. Doc checks (Most likely be manual - All new feature or updates should
have corresponding documentation)


Exit criterion for

a) each patch -
   1. Smoke tests upon submission
   2. Functional/regression after review and before merging

b) Nightly builds
   1. Regression + Functional
   2. Static Code Analysis
   3. performance regression
   4. Basic distro packaging

c) Minor releases (.x releases)
   1. Regression + Functional
   2. performance regression
   3. Full distro packaging sanity tests
   4. Longevity*
   5. Integration**
   6. Doc checks
   7. Upgrade testing

d) Major releases
   1. Regression + Functional
   2. performance regression
   3. Full distro packaging sanity tests
   4. Longevity*
   5. Integration**
   6. Doc checks
   7. Upgrade testing


* We are not sure about longevity in minor releases considering the one
month release schedule, and longevity tests has to run for at least 1-2
weeks.
** Currently integration tests are owned and maintained by the other
projects (like swift/openstack-manila). Should we setup our own integration
tests? Not sure how to qualify integration tests before releasing.

If we have missed anything or If you have any suggestions, please let us
know.


Best Regards,
msvbhat & rtalur



On 25 November 2015 at 17:39, Raghavendra Talur <rtalur at redhat.com> wrote:

> Hi All,
>
> Here is a table representing the current state of Gluster testing.
>
> * Things in green are categories for which we have some kind of testing in
> place.
> * Things in red are the ones which don't have any tests.
> * Things in yellow are the ones which have no known tests or are not
> managed by Gluster community.
>
>
>
> Test Category/Sub-Category
>
>
>
>
>
> smoke source build + Posix complaince + Dbench
>
>
>
>
> functional tests/basic
>
>
>
>
> regression tests/bugs
>
>
>
>
> performance regression N/A
>
>
>
>
> integration Backends/FS Protocols Consumers/Cloud Environments Tools libgfapi
> bindings OS environment xfs smb qemu gdeploy java firewalld ext4 nfs
> openstack/cinder heketi python ufw btrfs swift openshift/docker/containers
> ruby selinux zfs
> aws
> go apparmor
>
> azure
>
>
>
>
> hadoop
>
>
> update major version upgrades minor version upgrades
>
>
>
> longevity a. memory leaks
> b. log accumulation
>
>
>
>
> distro packaging a. pkg build + smoke
> b. sysvinit/systemd
>
>
>
>
>
>
> I will send separate mails for each of the categories above highlighting
> plans for them.
>
> Use this thread to indicate addition/deletion/changes to this matrix.
> We will put this at gluster.org website once it is final.
>
> Thanks,
> Raghavendra Talur & MSV Bhat
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20151216/5aaff455/attachment-0001.html>