[Bugs] [Bug 1577672] New: Brick-mux regressions failing for over 8+ weeks on master

[bugzilla at redhat.com]

https://bugzilla.redhat.com/show_bug.cgi?id=1577672

            Bug ID: 1577672
           Summary: Brick-mux regressions failing for over 8+ weeks on
                    master
           Product: GlusterFS
           Version: mainline
         Component: unclassified
          Keywords: Triaged
          Severity: urgent
          Priority: urgent
          Assignee: bugs at gluster.org
          Reporter: srangana at redhat.com
                CC: bugs at gluster.org



Brick-mux nightly regression is breaking on master for over 11+ weeks (since
around Feb-22nd, 2018):
https://build.gluster.org/job/regression-test-with-multiplex/

NOTE: This bug is opened to track the dependent failures, or newer ones.

Over time the problems seem to have morphed, or moved, due to other fixes in
between and an analysis of the last 13 builds (from Jenkins job number 725-738,
IOW https://build.gluster.org/job/regression-test-with-multiplex/[725..738])
show the following problems that need to be addressed, each are further filed
as separate bugs (or will be as the case progresses).

1) Test timeouts without anything really having run in the test (as far as I
can tell at present) (timeout in 200/300 seconds) (number if brackets is #
times this has failed in the analyzed runs)
./tests/basic/afr/entry-self-heal.t (4)
./tests/bitrot/br-state-check.t (8)
./tests/bugs/core/bug-1432542-mpx-restart-crash.t (1)
./tests/bugs/index/bug-1559004-EMLINK-handling.t (2)

2) ./tests/bugs/replicate/bug-1363721.t (4)
   ./tests/basic/afr/lk-quorum.t (5)

lk-quorum failures are easily observable in tests run on local machines as
well. The issue seems to be due to brick process not listening on or not coming
up for  one of the bricks, causing the checks for up counts to fail.

bug-1363721.t failures seem to be related to the same.

3) ./tests/bugs/glusterd/add-brick-and-validate-replicated-volume-options.t (7)

The above failure seems to constantly be when adding an previously added brick
(but using force), and failing in glusterd with transport end point not
connected.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
You are the assignee for the bug.