<div dir="ltr"><br><br><div class="gmail_quote"><div dir="ltr">On Wed, Aug 8, 2018 at 5:08 AM Shyam Ranganathan <<a href="mailto:srangana@redhat.com">srangana@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Deserves a new beginning, threads on the other mail have gone deep enough.<br>
<br>
NOTE: (5) below needs your attention, rest is just process and data on<br>
how to find failures.<br>
<br>
1) We are running the tests using the patch [2].<br>
<br>
2) Run details are extracted into a separate sheet in [3] named "Run<br>
Failures" use a search to find a failing test and the corresponding run<br>
that it failed in.<br>
<br>
3) Patches that are fixing issues can be found here [1], if you think<br>
you have a patch out there, that is not in this list, shout out.<br>
<br>
4) If you own up a test case failure, update the spreadsheet [3] with<br>
your name against the test, and also update other details as needed (as<br>
comments, as edit rights to the sheet are restricted).<br>
<br>
5) Current test failures<br>
We still have the following tests failing and some without any RCA or<br>
attention, (If something is incorrect, write back).<br>
<br>
./tests/bugs/replicate/bug-1290965-detect-bitrotten-objects.t (needs<br>
attention)<br>
./tests/00-geo-rep/georep-basic-dr-tarssh.t (Kotresh)<br>
./tests/bugs/glusterd/add-brick-and-validate-replicated-volume-options.t<br>
(Atin)<br></blockquote><div><br></div><div>This one is fixed through <a href="https://review.gluster.org/20651">https://review.gluster.org/20651</a> as I see no failures from this patch in the latest report from patch set 6. <br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
./tests/bugs/ec/bug-1236065.t (Ashish)<br>
./tests/00-geo-rep/georep-basic-dr-rsync.t (Kotresh)<br>
./tests/basic/ec/ec-1468261.t (needs attention)<br>
./tests/basic/afr/add-brick-self-heal.t (needs attention)<br>
./tests/basic/afr/granular-esh/replace-brick.t (needs attention)<br>
./tests/bugs/core/multiplex-limit-issue-151.t (needs attention)<br>
./tests/bugs/glusterd/validating-server-quorum.t (Atin)<br>
./tests/bugs/replicate/bug-1363721.t (Ravi)<br>
<br>
Here are some newer failures, but mostly one-off failures except cores<br>
in ec-5-2.t. All of the following need attention as these are new.<br>
<br>
./tests/00-geo-rep/00-georep-verify-setup.t<br>
./tests/basic/afr/gfid-mismatch-resolution-with-fav-child-policy.t<br>
./tests/basic/stats-dump.t<br>
./tests/bugs/bug-1110262.t<br>
./tests/bugs/glusterd/mgmt-handshake-and-volume-sync-post-glusterd-restart.t<br></blockquote><div><br></div><div>This failed because of <a href="https://review.gluster.org/20584">https://review.gluster.org/20584</a>. I believe there's some timing issue introduced from this patch. As I highlighted in <a href="https://review.gluster.org/#/c/20637">https://review.gluster.org/#/c/20637</a> as a comment I'd request you to revert this change and include <a href="https://review.gluster.org/20658">https://review.gluster.org/20658</a></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
./tests/basic/ec/ec-data-heal.t<br>
./tests/bugs/replicate/bug-1448804-check-quorum-type-values.t<br>
./tests/bugs/snapshot/bug-1482023-snpashot-issue-with-other-processes-accessing-mounted-path.t<br>
./tests/basic/ec/ec-5-2.t<br>
<br>
6) Tests that are addressed or are not occurring anymore are,<br>
<br>
./tests/bugs/glusterd/rebalance-operations-in-single-node.t<br>
./tests/bugs/index/bug-1559004-EMLINK-handling.t<br>
./tests/bugs/replicate/bug-1386188-sbrain-fav-child.t<br>
./tests/bugs/replicate/bug-1433571-undo-pending-only-on-up-bricks.t<br>
./tests/bitrot/bug-1373520.t<br>
./tests/bugs/distribute/bug-1117851.t<br>
./tests/bugs/glusterd/quorum-validation.t<br>
./tests/bugs/distribute/bug-1042725.t<br>
./tests/bugs/replicate/bug-1586020-mark-dirty-for-entry-txn-on-quorum-failure.t<br>
./tests/bugs/quota/bug-1293601.t<br>
./tests/bugs/bug-1368312.t<br>
./tests/bugs/distribute/bug-1122443.t<br>
./tests/bugs/core/bug-1432542-mpx-restart-crash.t<br>
<br>
Shyam (and Atin)<br>
<br>
On 08/05/2018 06:24 PM, Shyam Ranganathan wrote:<br>
> Health on master as of the last nightly run [4] is still the same.<br>
> <br>
> Potential patches that rectify the situation (as in [1]) are bunched in<br>
> a patch [2] that Atin and myself have put through several regressions<br>
> (mux, normal and line coverage) and these have also not passed.<br>
> <br>
> Till we rectify the situation we are locking down master branch commit<br>
> rights to the following people, Amar, Atin, Shyam, Vijay.<br>
> <br>
> The intention is to stabilize master and not add more patches that my<br>
> destabilize it.<br>
> <br>
> Test cases that are tracked as failures and need action are present here<br>
> [3].<br>
> <br>
> @Nigel, request you to apply the commit rights change as you see this<br>
> mail and let the list know regarding the same as well.<br>
> <br>
> Thanks,<br>
> Shyam<br>
> <br>
> [1] Patches that address regression failures:<br>
> <a href="https://review.gluster.org/#/q/starredby:srangana%2540redhat.com" rel="noreferrer" target="_blank">https://review.gluster.org/#/q/starredby:srangana%2540redhat.com</a><br>
> <br>
> [2] Bunched up patch against which regressions were run:<br>
> <a href="https://review.gluster.org/#/c/20637" rel="noreferrer" target="_blank">https://review.gluster.org/#/c/20637</a><br>
> <br>
> [3] Failing tests list:<br>
> <a href="https://docs.google.com/spreadsheets/d/1IF9GhpKah4bto19RQLr0y_Kkw26E_-crKALHSaSjZMQ/edit?usp=sharing" rel="noreferrer" target="_blank">https://docs.google.com/spreadsheets/d/1IF9GhpKah4bto19RQLr0y_Kkw26E_-crKALHSaSjZMQ/edit?usp=sharing</a><br>
> <br>
> [4] Nightly run dashboard: <a href="https://build.gluster.org/job/nightly-master/" rel="noreferrer" target="_blank">https://build.gluster.org/job/nightly-master/</a><br>
_______________________________________________<br>
maintainers mailing list<br>
<a href="mailto:maintainers@gluster.org" target="_blank">maintainers@gluster.org</a><br>
<a href="https://lists.gluster.org/mailman/listinfo/maintainers" rel="noreferrer" target="_blank">https://lists.gluster.org/mailman/listinfo/maintainers</a><br>
</blockquote></div></div>