[Gluster-Maintainers] [Gluster-devel] Test: ./tests/bugs/ec/bug-1236065.t

Wed Aug 8 02:57:48 UTC 2018

I have posted a patch https://review.gluster.org/#/c/20657/ and start
brick-mux regression to validate the patch.

Thanks
Mohit Agrawal

On Wed, Aug 8, 2018 at 7:22 AM, Atin Mukherjee <amukherj at redhat.com> wrote:

> +Mohit
>
> Requesting Mohit for help.
>
> On Wed, 8 Aug 2018 at 06:53, Shyam Ranganathan <srangana at redhat.com>
> wrote:
>
>> On 08/07/2018 07:37 PM, Shyam Ranganathan wrote:
>> > 5) Current test failures
>> > We still have the following tests failing and some without any RCA or
>> > attention, (If something is incorrect, write back).
>> >
>> > ./tests/bugs/ec/bug-1236065.t (Ashish)
>>
>> Ashish/Atin, the above test failed in run:
>> https://build.gluster.org/job/regression-on-demand-
>> multiplex/172/consoleFull
>>
>> The above run is based on patchset 4 of
>> https://review.gluster.org/#/c/20637/4
>>
>> The logs look as below, and as Ashish is unable to reproduce this, and
>> all failures are on line 78 with a heal outstanding of 105, looks like
>> this run may provide some possibilities on narrowing it down.
>>
>> The problem seems to be glustershd not connecting to one of the bricks
>> that is restarted, and hence failing to heal that brick. This also looks
>> like what Ravi RCAd for the test: ./tests/bugs/replicate/bug-1363721.t
>>
>> ==================================================
>> Test times from: cat ./glusterd.log | grep TEST
>> [2018-08-06 20:56:28.177386]:++++++++++
>> G_LOG:./tests/bugs/ec/bug-1236065.t: TEST: 77 gluster --mode=script
>> --wignore volume heal patchy full ++++++++++
>> [2018-08-06 20:56:28.767209]:++++++++++
>> G_LOG:./tests/bugs/ec/bug-1236065.t: TEST: 78 ^0$ get_pending_heal_count
>> patchy ++++++++++
>> [2018-08-06 20:57:48.957136]:++++++++++
>> G_LOG:./tests/bugs/ec/bug-1236065.t: TEST: 80 rm -f 0.o 10.o 11.o 12.o
>> 13.o 14.o 15.o 16.o 17.o 18.o 19.o 1.o 2.o 3.o 4.o 5.o 6.o 7.o 8.o 9.o
>> ++++++++++
>> ==================================================
>> Repeated connection failure to client-3 in glustershd.log:
>> [2018-08-06 20:56:30.218482] I [rpc-clnt.c:2087:rpc_clnt_reconfig]
>> 0-patchy-client-3: changing port to 49152 (from 0)
>> [2018-08-06 20:56:30.222738] W [MSGID: 114043]
>> [client-handshake.c:1061:client_setvolume_cbk] 0-patchy-client-3: failed
>> to set the volume [Resource temporarily unavailable]
>> [2018-08-06 20:56:30.222788] W [MSGID: 114007]
>> [client-handshake.c:1090:client_setvolume_cbk] 0-patchy-client-3: failed
>> to get 'process-uuid' from reply dict [Invalid argument]
>> [2018-08-06 20:56:30.222813] E [MSGID: 114044]
>> [client-handshake.c:1096:client_setvolume_cbk] 0-patchy-client-3:
>> SETVOLUME on remote-host failed: cleanup flag is set for xlator.  Try
>> again later [Resource tempor
>> arily unavailable]
>> [2018-08-06 20:56:30.222845] I [MSGID: 114051]
>> [client-handshake.c:1201:client_setvolume_cbk] 0-patchy-client-3:
>> sending CHILD_CONNECTING event
>> [2018-08-06 20:56:30.222919] I [MSGID: 114018]
>> [client.c:2255:client_rpc_notify] 0-patchy-client-3: disconnected from
>> patchy-client-3. Client process will keep trying to connect to glusterd
>> until brick's port is
>>  available
>> ==================================================
>> Repeated connection messages close to above retries in
>> d-backends-patchy0.log:
>> [2018-08-06 20:56:38.530009] I [addr.c:55:compare_addr_and_update]
>> 0-/d/backends/patchy0: allowed = "*", received addr = "127.0.0.1"
>> [2018-08-06 20:56:38.530044] I [login.c:111:gf_auth] 0-auth/login:
>> allowed user names: 756f302a-66eb-4cc0-8f91-797183312f05
>> The message "I [MSGID: 101016] [glusterfs3.h:739:dict_to_xdr] 0-dict:
>> key 'trusted.ec.version' is would not be sent on wire in future [Invalid
>> argument]" repeated 6 times between [2018-08-06 20:56:37.931040] and
>>  [2018-08-06 20:56:37.933084]
>> [2018-08-06 20:56:38.530067] I [MSGID: 115029]
>> [server-handshake.c:786:server_setvolume] 0-patchy-server: accepted
>> client from
>> CTX_ID:cb3b4fed-62a4-4ad5-8b92-97838c651b22-GRAPH_ID:0-
>> PID:10506-HOST:builder104.clo
>> ud.gluster.org-PC_NAME:patchy-client-0-RECON_NO:-0 (version: 4.2dev)
>> [2018-08-06 20:56:38.540499] I [addr.c:55:compare_addr_and_update]
>> 0-/d/backends/patchy1: allowed = "*", received addr = "127.0.0.1"
>> [2018-08-06 20:56:38.540533] I [login.c:111:gf_auth] 0-auth/login:
>> allowed user names: 756f302a-66eb-4cc0-8f91-797183312f05
>> [2018-08-06 20:56:38.540555] I [MSGID: 115029]
>> [server-handshake.c:786:server_setvolume] 0-patchy-server: accepted
>> client from
>> CTX_ID:cb3b4fed-62a4-4ad5-8b92-97838c651b22-GRAPH_ID:0-
>> PID:10506-HOST:builder104.clo
>> ud.gluster.org-PC_NAME:patchy-client-1-RECON_NO:-0 (version: 4.2dev)
>> [2018-08-06 20:56:38.552442] I [addr.c:55:compare_addr_and_update]
>> 0-/d/backends/patchy2: allowed = "*", received addr = "127.0.0.1"
>> [2018-08-06 20:56:38.552472] I [login.c:111:gf_auth] 0-auth/login:
>> allowed user names: 756f302a-66eb-4cc0-8f91-797183312f05
>> [2018-08-06 20:56:38.552494] I [MSGID: 115029]
>> [server-handshake.c:786:server_setvolume] 0-patchy-server: accepted
>> client from
>> CTX_ID:cb3b4fed-62a4-4ad5-8b92-97838c651b22-GRAPH_ID:0-
>> PID:10506-HOST:builder104.clo
>> ud.gluster.org-PC_NAME:patchy-client-2-RECON_NO:-0 (version: 4.2dev)
>> [2018-08-06 20:56:38.571671] I [addr.c:55:compare_addr_and_update]
>> 0-/d/backends/patchy4: allowed = "*", received addr = "127.0.0.1"
>> [2018-08-06 20:56:38.571701] I [login.c:111:gf_auth] 0-auth/login:
>> allowed user names: 756f302a-66eb-4cc0-8f91-797183312f05
>> [2018-08-06 20:56:38.571723] I [MSGID: 115029]
>> [server-handshake.c:786:server_setvolume] 0-patchy-server: accepted
>> client from
>> CTX_ID:cb3b4fed-62a4-4ad5-8b92-97838c651b22-GRAPH_ID:0-
>> PID:10506-HOST:builder104.clo
>> ud.gluster.org-PC_NAME:patchy-client-4-RECON_NO:-0 (version: 4.2dev)
>> [2018-08-06 20:56:38.580579] I [addr.c:55:compare_addr_and_update]
>> 0-/d/backends/patchy5: allowed = "*", received addr = "127.0.0.1"
>> [2018-08-06 20:56:38.580609] I [login.c:111:gf_auth] 0-auth/login:
>> allowed user names: 756f302a-66eb-4cc0-8f91-797183312f05
>> [2018-08-06 20:56:38.580630] I [MSGID: 115029]
>> [server-handshake.c:786:server_setvolume] 0-patchy-server: accepted
>> client from
>> CTX_ID:cb3b4fed-62a4-4ad5-8b92-97838c651b22-GRAPH_ID:0-
>> PID:10506-HOST:builder104.clo
>> ud.gluster.org-PC_NAME:patchy-client-5-RECON_NO:-0 (version: 4.2dev)
>> [2018-08-06 20:56:38.583444] I [addr.c:55:compare_addr_and_update]
>> 0-/d/backends/patchy6: allowed = "*", received addr = "127.0.0.1"
>> [2018-08-06 20:56:38.583472] I [login.c:111:gf_auth] 0-auth/login:
>> allowed user names: 756f302a-66eb-4cc0-8f91-797183312f05
>> [2018-08-06 20:56:38.583493] I [MSGID: 115029]
>> [server-handshake.c:786:server_setvolume] 0-patchy-server: accepted
>> client from
>> CTX_ID:cb3b4fed-62a4-4ad5-8b92-97838c651b22-GRAPH_ID:0-
>> PID:10506-HOST:builder104.clo
>> ud.gluster.org-PC_NAME:patchy-client-6-RECON_NO:-0 (version: 4.2dev)
>>
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at gluster.org
>> https://lists.gluster.org/mailman/listinfo/gluster-devel
>>
> --
> - Atin (atinm)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/maintainers/attachments/20180808/eb19cfc4/attachment-0001.html>