[Gluster-devel] [FAILED] regression tests: tests/bugs/distribute/bug-1066798.t, tests/basic/volume-snapshot.t
Avra Sengupta
asengupt at redhat.com
Mon Jul 20 12:53:52 UTC 2015
Got access to the slave machine. Had a look at the BT. The test-case
that keeps failing spuriously is kill_glusterd in cluster.rc, and the bt
looks like the following. Looks like the volinfo is corrupted. Is looks
like it was trying to access peer data after cleanup_and _exit was called.
[2015-07-19 10:23:31.799681] W [glusterfsd.c:1214:cleanup_and_exit]
(-->/lib64/libpthread.so.0(+0x79d1) [0x7f3f744729d1]
-->glusterd(glusterfs_sigwaiter+0xe4) [0x409734]
-->glusterd(cleanup_and_exit+0x87) [0x407ba7] ) 0-: received signum
(15), shutting down
[2015-07-19 10:23:31.802507] W [socket.c:637:__socket_rwv] 0-management:
readv on 127.1.1.3:24007 failed (No data available)
[2015-07-19 10:23:31.802581] I [timer.c:43:gf_timer_call_after]
(-->/build/install/lib/libgfrpc.so.0(rpc_transport_notify+0x12f)
[0x7f3f74f57aeb]
-->/build/install/lib/libgfrpc.so.0(rpc_clnt_notify+0x1ca)
[0x7f3f74f5b644]
-->/build/install/lib/libglusterfs.so.0(gf_timer_call_after+0xfb)
[0x7f3f751b9517] ) 0-timer: ctx cleanup started
[2015-07-19 10:23:31.802608] I [MSGID: 106004]
[glusterd-handler.c:5047:__glusterd_peer_rpc_notify] 0-management: Peer
<127.1.1.3> (<377dd370-1401-452b-831e-cbaf4340a376>), in state <Peer in
Cluster>, has disconnected from glusterd.
[2015-07-19 10:23:31.804501] W [socket.c:637:__socket_rwv] 0-management:
readv on 127.1.1.1:24007 failed (No data available)
[2015-07-19 10:23:31.804567] I [timer.c:43:gf_timer_call_after]
(-->/build/install/lib/libgfrpc.so.0(rpc_transport_notify+0x12f)
[0x7f3f74f57aeb]
-->/build/install/lib/libgfrpc.so.0(rpc_clnt_notify+0x1ca)
[0x7f3f74f5b644]
-->/build/install/lib/libglusterfs.so.0(gf_timer_call_after+0xfb)
[0x7f3f751b9517] ) 0-timer: ctx cleanup started
[2015-07-19 10:23:31.804612] I [MSGID: 106004]
[glusterd-handler.c:5047:__glusterd_peer_rpc_notify] 0-management: Peer
<127.1.1.1> (<fd3a8e54-a8fc-47d4-93fa-ed1a9e2c78fc>), in state <Peer in
Cluster>, has disconnected from glusterd.
#0 0x00007f8f3c400e2c in vfprintf () from ./lib64/libc.so.6
#1 0x00007f8f3c428752 in vsnprintf () from ./lib64/libc.so.6
#2 0x00007f8f3c408223 in snprintf () from ./lib64/libc.so.6
#3 0x00007f8f32cded8d in glusterd_volume_stop_glusterfs
(volinfo=0x253ee30, brickinfo=0x2549f60, del_brick=_gf_false)
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/xlators/mgmt/glusterd/src/glusterd-utils.c:1754
#4 0x00007f8f32cebd18 in glusterd_brick_stop (volinfo=0x253ee30,
brickinfo=0x2549f60, del_brick=_gf_false)
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/xlators/mgmt/glusterd/src/glusterd-utils.c:5458
#5 0x00007f8f32d76804 in glusterd_snap_volume_remove
(rsp_dict=0x7f8f1800102c, snap_vol=0x253ee30, remove_lvm=_gf_false,
force=_gf_false)
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/xlators/mgmt/glusterd/src/glusterd-snapshot.c:2897
#6 0x00007f8f32d76daf in glusterd_snap_remove (rsp_dict=0x7f8f1800102c,
snap=0x25388a0, remove_lvm=_gf_false, force=_gf_false)
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/xlators/mgmt/glusterd/src/glusterd-snapshot.c:3005
#7 0x00007f8f32da2e58 in glusterd_compare_and_update_snap
(peer_data=0x7f8f180024fc, snap_count=1, peername=0x7f8f18002370
"127.1.1.1", peerid=0x7f8f180023e0
"\375:\216T\250\374Gԓ\372\355\032\236,x\374p#")
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/xlators/mgmt/glusterd/src/glusterd-snapshot-utils.c:1849
#8 0x00007f8f32da311f in glusterd_compare_friend_snapshots
(peer_data=0x7f8f180024fc, peername=0x7f8f18002370 "127.1.1.1",
peerid=0x7f8f180023e0 "\375:\216T\250\374Gԓ\372\355\032\236,x\374p#")
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/xlators/mgmt/glusterd/src/glusterd-snapshot-utils.c:1904
#9 0x00007f8f32cc49b3 in glusterd_ac_handle_friend_add_req
(event=0x7f8f180023d0, ctx=0x7f8f18002460)
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/xlators/mgmt/glusterd/src/glusterd-sm.c:831
#10 0x00007f8f32cc5250 in glusterd_friend_sm () at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/xlators/mgmt/glusterd/src/glusterd-sm.c:1253
#11 0x00007f8f32cbadd4 in __glusterd_handle_incoming_friend_req
(req=0x7f8f1800128c) at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/xlators/mgmt/glusterd/src/glusterd-handler.c:2541
#12 0x00007f8f32cb36aa in glusterd_big_locked_handler
(req=0x7f8f1800128c, actor_fn=0x7f8f32cbac38
<__glusterd_handle_incoming_friend_req>)
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/xlators/mgmt/glusterd/src/glusterd-handler.c:79
#13 0x00007f8f32cbae0a in glusterd_handle_incoming_friend_req
(req=0x7f8f1800128c) at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/xlators/mgmt/glusterd/src/glusterd-handler.c:2551
#14 0x00007f8f3d61706d in rpcsvc_handle_rpc_call (svc=0x24b8430,
trans=0x7f8f20008e20, msg=0x7f8f18000e20) at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/rpc/rpc-lib/src/rpcsvc.c:699
#15 0x00007f8f3d6173e0 in rpcsvc_notify (trans=0x7f8f20008e20,
mydata=0x24b8430, event=RPC_TRANSPORT_MSG_RECEIVED, data=0x7f8f18000e20)
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/rpc/rpc-lib/src/rpcsvc.c:793
#16 0x00007f8f3d61caeb in rpc_transport_notify (this=0x7f8f20008e20,
event=RPC_TRANSPORT_MSG_RECEIVED, data=0x7f8f18000e20)
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/rpc/rpc-lib/src/rpc-transport.c:538
#17 0x00007f8f3134187b in socket_event_poll_in (this=0x7f8f20008e20) at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/rpc/rpc-transport/socket/src/socket.c:2285
#18 0x00007f8f31341dd1 in socket_event_handler (fd=20, idx=10,
data=0x7f8f20008e20, poll_in=1, poll_out=0, poll_err=0)
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/rpc/rpc-transport/socket/src/socket.c:2398
#19 0x00007f8f3d8d09e4 in event_dispatch_epoll_handler
(event_pool=0x249ec90, event=0x7f8f2dfb5e70) at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/event-epoll.c:570
#20 0x00007f8f3d8d0dd2 in event_dispatch_epoll_worker (data=0x24a97f0)
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/event-epoll.c:673
#21 0x00007f8f3cb379d1 in start_thread () from ./lib64/libpthread.so.0
#22 0x00007f8f3c4a18fd in clone () from ./lib64/libc.so.6
Regards,
Avra
On 07/20/2015 04:38 PM, Avra Sengupta wrote:
> The particular slave (slave21) containing the cores is down. I however
> have access to slave0, so trying to recreate it on that slave and will
> analyze the core when I get it.
>
> Regards,
> Avra
>
> On 07/20/2015 03:19 PM, Ravishankar N wrote:
>> One more core for volume-snapshot.t:
>> https://build.gluster.org/job/rackspace-regression-2GB-triggered/12605/consoleFull
>>
>> On 07/20/2015 03:00 PM, Raghavendra Talur wrote:
>>> Adding Susant and Avra for dht and snapshot test cases respectively.
>>>
>>>
>>> On Mon, Jul 20, 2015 at 11:45 AM, Milind Changire
>>> <milindchangire at gmail.com> wrote:
>>>
>>> http://build.gluster.org/job/rackspace-regression-2GB-triggered/12541/consoleFull
>>>
>>> http://build.gluster.org/job/rackspace-regression-2GB-triggered/12499/consoleFull
>>>
>>>
>>> Please advise.
>>>
>>> --
>>> Milind
>>>
>>>
>>> _______________________________________________
>>> Gluster-devel mailing list
>>> Gluster-devel at gluster.org <mailto:Gluster-devel at gluster.org>
>>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>>
>>>
>>>
>>>
>>> --
>>> *Raghavendra Talur *
>>>
>>>
>>>
>>> _______________________________________________
>>> Gluster-devel mailing list
>>> Gluster-devel at gluster.org
>>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>
>>
>>
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20150720/dd535e9e/attachment.html>
More information about the Gluster-devel
mailing list