[Gluster-devel] Release 3.12: Glusto run status
srangana at redhat.com
Mon Aug 28 22:43:03 UTC 2017
The latest Glusto run [a] that was started by Nigel, post fixing the
prior timeout issue, failed (much later though) again.
I took a look at the logs and my analysis is here [b]
@atin, @kaushal, @ppai can you take a look and see if the analysis is
In short glusterd has got an error when checking for rebalance stats
from one of the nodes as:
"Received commit RJT from uuid: 6f9524e6-9f9e-44aa-b2f4-393404adfd9d"
and the rebalance deamon on the node with that UUID is not really ready
to serve requests when this was called, hence I am assuming this is
causing the error. But need a once over by one of you folks.
@Shwetha, can we add a further timeout between rebalance start and
checking the status, just so that we avoid this timing issue on these nodes.
[a] glusto run: https://ci.centos.org/view/Gluster/job/gluster_glusto/377/
[b] analysis of the failure:
On 08/25/2017 04:29 PM, Shyam Ranganathan wrote:
> Nigel was kind enough to kick off a glusto run on 3.12 head a couple of
> days back. The status can be seen here .
> The run failed, but managed to get past what Glusto does on master (see
> ). Not that this is a consolation, but just stating the fact.
> The run  failed at,
> The test case failed due to,
> 17:10:28 E AssertionError: ('Volume %s : All process are not
> online', 'testvol_dispersed')
> The test case can be seen here , and the reason for failure is that
> Glusto did not wait long enough for the down brick to come up (it waited
> for 10 seconds, but the brick came up after 12 seconds or within the
> same second as the test for it being up. The log snippets pointing to
> this problem are here . In short there was no real bug or issue that
> caused the failure as yet.
> Glusto as a gating factor for this release was desirable, but having got
> this far on 3.12 does help.
> @nigel, we could try post increasing the timeout between bringing the
> brick up to checking if it is up, and try another run, let me know if
> that works, and what is needed from me to get this going.
>  Glusto 3.12 run:
>  Glusto on master:
>  Failed test case:
>  Log analysis pointing to the failed check:
> "Releases are made better together"
> Gluster-devel mailing list
> Gluster-devel at gluster.org
More information about the Gluster-devel