[Gluster-devel] tests are timing out in master branch

[Amar Tumballi Suryanarayan]

Looks like after reverting a patch on RPC layer reconnection logic (
https://review.gluster.org/22750) things are back to normal.

For those who submitted a patch in last 1 week, please resubmit. (which
should take care of rebasing on top of this patch).

This event proves that there are very delicate races in our RPC layer,
which can trigger random failures. While it was discussed in brief earlier.
We need to debug this further, and come up with possible next actions.
Volunteers welcome.

I recommend to use https://github.com/gluster/glusterfs/issues/391 to
capture our observations, and continue on github from here.

-Amar


On Wed, May 15, 2019 at 11:46 AM Sankarshan Mukhopadhyay <
sankarshan.mukhopadhyay at gmail.com> wrote:

> On Wed, May 15, 2019 at 11:24 AM Atin Mukherjee <amukherj at redhat.com>
> wrote:
> >
> > There're random tests which are timing out after 200 secs. My belief is
> this is a major regression introduced by some commit recently or the
> builders have become extremely slow which I highly doubt. I'd request that
> we first figure out the cause, get master back to it's proper health and
> then get back to the review/merge queue.
> >
>
> For such dire situations, we also need to consider a proposal to back
> out patches in order to keep the master healthy. The outcome we seek
> is a healthy master - the isolation of the cause allows us to not
> repeat the same offense.
>
> > Sanju has already started looking into
> /tests/bugs/glusterd/optimized-basic-testcases-in-cluster.t to understand
> what test is specifically hanging and consuming more time.
> _______________________________________________
> Atin Mukherjee <amukherj at redhat.com>, Sankarshan Mukhopadhyay <
> sankarshan.mukhopadhyay at gmail.com>
> Community Meeting Calendar:
>
> APAC Schedule -https://review.gluster.org/22750
> Every 2nd and 4th Tuesday at 11:30 AM IST
> Bridge: https://bluejeans.com/836554017
>
> NA/EMEA Schedule -
> Every 1st and 3rd Tuesday at 01:00 PM EDT
> Bridge: https://bluejeans.com/486278655
>
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-devel
>
>
>

-- 
Amar Tumballi (amarts)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-devel/attachments/20190521/4d06bb3d/attachment.html>