[Bugs] [Bug 1316533] New: RFE: Provide a mechanism to disable some tests in regression

[bugzilla at redhat.com]

https://bugzilla.redhat.com/show_bug.cgi?id=1316533

            Bug ID: 1316533
           Summary: RFE: Provide a mechanism to disable some tests in
                    regression
           Product: GlusterFS
           Version: 3.7.9
         Component: tests
          Assignee: bugs at gluster.org
          Reporter: rtalur at redhat.com
                CC: bugs at gluster.org
        Depends On: 1295704



+++ This bug was initially created as a clone of Bug #1295704 +++

Description of problem:

Sometimes it is useful to have a test in test repo but not execute in
regression runs.

Reasons could be:
1. Test is not deterministic enough or corrupts environment etc.
2. We don't have a fix yet but the test is already there.


tests/bad_tests serves as the directory where we can send type 1 tests
mentioned above. You can use "git mv" to move the .t file from its current
place to tests/bad_tests. Make sure you file a bug on the test file and use
that for mv patch.

tests/known_bugs serves as the directory where we can send type 2 tests
mentioned above. You can use "git mv" to move the .t file from its current
place to tests/known_bugs or add a new .t file directly under tests/known_bugs.
Make sure you file a bug for the bug you have discovered and use that for
mv/add patch.

--- Additional comment from Vijay Bellur on 2016-01-05 16:10:12 IST ---

REVIEW: http://review.gluster.org/13173 (tests: Introduce mechanism to disable
tests) posted (#1) for review on master by Raghavendra Talur
(rtalur at redhat.com)

--- Additional comment from Vijay Bellur on 2016-01-12 15:30:27 IST ---

REVIEW: http://review.gluster.org/13173 (tests: Introduce mechanism to disable
tests) posted (#2) for review on master by Raghavendra Talur
(rtalur at redhat.com)

--- Additional comment from Vijay Bellur on 2016-01-13 10:01:11 IST ---

REVIEW: http://review.gluster.org/13173 (tests: Introduce mechanism to disable
tests) posted (#3) for review on master by Atin Mukherjee (amukherj at redhat.com)

--- Additional comment from Vijay Bellur on 2016-02-08 12:44:13 IST ---

REVIEW: http://review.gluster.org/13393 (tests: Add mechanism for disabled
tests) posted (#1) for review on master by Raghavendra Talur
(rtalur at redhat.com)

--- Additional comment from Vijay Bellur on 2016-03-03 01:49:22 IST ---

REVIEW: http://review.gluster.org/13393 (tests: Add mechanism for disabled
tests) posted (#2) for review on master by Raghavendra Talur
(rtalur at redhat.com)

--- Additional comment from Vijay Bellur on 2016-03-03 02:45:42 IST ---

REVIEW: http://review.gluster.org/13393 (tests: Add mechanism for disabled
tests) posted (#3) for review on master by Raghavendra Talur
(rtalur at redhat.com)

--- Additional comment from Vijay Bellur on 2016-03-09 00:25:52 IST ---

REVIEW: http://review.gluster.org/13393 (tests: Add mechanism for disabled
tests) posted (#4) for review on master by Raghavendra Talur
(rtalur at redhat.com)

--- Additional comment from Vijay Bellur on 2016-03-09 00:29:05 IST ---

REVIEW: http://review.gluster.org/13393 (tests: Add mechanism for disabled
tests) posted (#5) for review on master by Raghavendra Talur
(rtalur at redhat.com)

--- Additional comment from Vijay Bellur on 2016-03-09 00:30:21 IST ---

REVIEW: http://review.gluster.org/13393 (tests: Add mechanism for disabled
tests) posted (#6) for review on master by Raghavendra Talur
(rtalur at redhat.com)

--- Additional comment from Vijay Bellur on 2016-03-09 12:08:22 IST ---

REVIEW: http://review.gluster.org/13393 (tests: Add mechanism for disabled
tests) posted (#7) for review on master by Raghavendra Talur
(rtalur at redhat.com)

--- Additional comment from Vijay Bellur on 2016-03-10 12:49:46 IST ---

COMMIT: http://review.gluster.org/13393 committed in master by Raghavendra
Talur (rtalur at redhat.com) 
------
commit f6e5bc68d294cc67f1e7f4db043d2531d4cfec80
Author: Raghavendra Talur <rtalur at redhat.com>
Date:   Mon Feb 8 12:13:54 2016 +0530

    tests: Add mechanism for disabled tests

    Requirements:
    Should be able to skip tests from run-tests.sh run.
    Should be granular enough to disable on subset of OSes.

    Solution:
    Tests can have special comment lines with some comma separated values
    within them.

    Key names used to determine test status are
    G_TESTDEF_TEST_STATUS_CENTOS6
    G_TESTDEF_TEST_STATUS_NETBSD7
    Some examples:
    G_TESTDEF_TEST_STATUS_CENTOS6=BAD_TEST,BUG=123456
    G_TESTDEF_TEST_STATUS_NETBSD7=KNOWN_ISSUE,BUG=4444444
    G_TESTDEF_TEST_STATUS_CENTOS6=BAD_TEST,BUG=123456;555555
    You can change status of test to enabled or delete the line only if all the
    bugs are closed or modified or if the patch fixes it.

    Change-Id: Idee21fecaa5837fd4bd06e613f5c07a024f7b0c2
    BUG: 1295704
    Signed-off-by: Raghavendra Talur <rtalur at redhat.com>
    Reviewed-on: http://review.gluster.org/13393
    Smoke: Gluster Build System <jenkins at build.gluster.com>
    NetBSD-regression: NetBSD Build System <jenkins at build.gluster.org>
    CentOS-regression: Gluster Build System <jenkins at build.gluster.com>
    Reviewed-by: Rajesh Joseph <rjoseph at redhat.com>
    Reviewed-by: Kaushal M <kaushal at redhat.com>


Referenced Bugs:

https://bugzilla.redhat.com/show_bug.cgi?id=1295704
[Bug 1295704] RFE: Provide a mechanism to disable some tests in regression
-- 
You are receiving this mail because:
You are on the CC list for the bug.
You are the assignee for the bug.