[Gluster-users] GlusterFS as virtual machine storage

[Pavel Szalbot]

This is the qemu log of instance:

[2017-09-08 09:31:48.381077] C
[rpc-clnt-ping.c:160:rpc_clnt_ping_timer_expired]
0-gv_openstack_1-client-1: server 10.0.1.202:49152 has not responded
in the last 1 seconds, disconnecting.
[2017-09-08 09:31:48.382411] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--
> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
 0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FINODELK(30)) called at 2017-09-08 09:31:45.626882 (xid=0x27ab5)
[2017-09-08 09:31:48.382451] E [MSGID: 114031]
[client-rpc-fops.c:1593:client3_3_finodelk_cbk]
0-gv_openstack_1-client-1: remote operation failed [Transport endpoint
is not connected]
[2017-09-08 09:31:48.382835] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--
> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
 0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FINODELK(30)) called at 2017-09-08 09:31:45.627341 (xid=0x27ab6)
[2017-09-08 09:31:48.383293] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--
> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
 0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FINODELK(30)) called at 2017-09-08 09:31:45.627403 (xid=0x27ab7)
[2017-09-08 09:31:48.383639] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--
> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
 0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FINODELK(30)) called at 2017-09-08 09:31:45.627444 (xid=0x27ab8)
[2017-09-08 09:31:48.384015] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--
> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
 0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(ENTRYLK(31)) called at 2017-09-08 09:31:45.627464 (xid=0x27ab9)
[2017-09-08 09:31:48.384056] E [MSGID: 114031]
[client-rpc-fops.c:1646:client3_3_entrylk_cbk]
0-gv_openstack_1-client-1: remote operation failed [Transport endpoint
is not connected]
[2017-09-08 09:31:48.384077] E [MSGID: 108007]
[afr-lk-common.c:825:afr_unlock_entrylk_cbk]
0-gv_openstack_1-replicate-0:
/.shard/fe56e79d-7800-4191-8cad-276545017ce7.4223: unlock failed on
gv_openstack_1-client
-1 [Transport endpoint is not connected]
[2017-09-08 09:31:48.384336] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--
> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
 0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(WRITE(13)) called at 2017-09-08 09:31:45.626871 (xid=0x27ab4)
[2017-09-08 09:31:48.388095] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--
> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
 0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(READ(12)) called at 2017-09-08 09:31:45.628760 (xid=0x27aba)
[2017-09-08 09:31:48.388450] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--
> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
 0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(READ(12)) called at 2017-09-08 09:31:45.629421 (xid=0x27abb)
[2017-09-08 09:31:48.388757] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--
> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
 0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(LOOKUP(27)) called at 2017-09-08 09:31:45.647966 (xid=0x27abc)
[2017-09-08 09:31:48.389158] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--
> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
 0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(LOOKUP(27)) called at 2017-09-08 09:31:45.648128 (xid=0x27abd)
[2017-09-08 09:31:48.389502] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--
> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] ))))) 0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3) op(LOOKUP(27)) called at 2017-09-08 09:31:45.648260 (xid=0x27abe)
[2017-09-08 09:31:48.389838] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(LOOKUP(27)) called at 2017-09-08 09:31:45.652420 (xid=0x27abf)
[2017-09-08 09:31:48.390217] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(LOOKUP(27)) called at 2017-09-08 09:31:45.652552 (xid=0x27ac0)
[2017-09-08 09:31:48.390558] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(LOOKUP(27)) called at 2017-09-08 09:31:45.652678 (xid=0x27ac1)
[2017-09-08 09:31:48.390886] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(LOOKUP(27)) called at 2017-09-08 09:31:45.652810 (xid=0x27ac2)
[2017-09-08 09:31:48.391263] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(LOOKUP(27)) called at 2017-09-08 09:31:45.652938 (xid=0x27ac3)
[2017-09-08 09:31:48.391650] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.371223 (xid=0x27ac4)
[2017-09-08 09:31:48.392007] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.371319 (xid=0x27ac5)
[2017-09-08 09:31:48.392428] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.371406 (xid=0x27ac6)
[2017-09-08 09:31:48.392727] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.371470 (xid=0x27ac7)
[2017-09-08 09:31:48.393029] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.371533 (xid=0x27ac8)
[2017-09-08 09:31:48.393397] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.371607 (xid=0x27ac9)
[2017-09-08 09:31:48.393720] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.371676 (xid=0x27aca)
[2017-09-08 09:31:48.394092] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.371739 (xid=0x27acb)
[2017-09-08 09:31:48.396857] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.371815 (xid=0x27acc)
[2017-09-08 09:31:48.397273] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.371913 (xid=0x27acd)
[2017-09-08 09:31:48.399192] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.372010 (xid=0x27ace)
[2017-09-08 09:31:48.402229] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.372147 (xid=0x27acf)
[2017-09-08 09:31:48.402716] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.372247 (xid=0x27ad0)
[2017-09-08 09:31:48.403171] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.372347 (xid=0x27ad1)
[2017-09-08 09:31:48.403786] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.372438 (xid=0x27ad2)
[2017-09-08 09:31:48.404281] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.372535 (xid=0x27ad3)
[2017-09-08 09:31:48.406718] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.372632 (xid=0x27ad4)
[2017-09-08 09:31:48.407210] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.372752 (xid=0x27ad5)
[2017-09-08 09:31:48.407771] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.372840 (xid=0x27ad6)
[2017-09-08 09:31:48.408253] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.372943 (xid=0x27ad7)
[2017-09-08 09:31:48.408710] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.373054 (xid=0x27ad8)
[2017-09-08 09:31:48.409190] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.373175 (xid=0x27ad9)
[2017-09-08 09:31:48.409671] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.373276 (xid=0x27ada)
[2017-09-08 09:31:48.410135] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.373375 (xid=0x27adb)
[2017-09-08 09:31:48.410638] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.373466 (xid=0x27adc)
[2017-09-08 09:31:48.411136] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.373552 (xid=0x27add)
[2017-09-08 09:31:48.412471] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.373645 (xid=0x27ade)
[2017-09-08 09:31:48.413783] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.373754 (xid=0x27adf)
[2017-09-08 09:31:48.414972] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.373852 (xid=0x27ae0)
[2017-09-08 09:31:48.415517] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.373936 (xid=0x27ae1)
[2017-09-08 09:31:48.416428] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.374046 (xid=0x27ae2)
[2017-09-08 09:31:48.417311] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.374148 (xid=0x27ae3)
[2017-09-08 09:31:48.418227] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.374235 (xid=0x27ae4)
[2017-09-08 09:31:48.419323] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.374326 (xid=0x27ae5)
[2017-09-08 09:31:48.419910] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.374409 (xid=0x27ae6)
[2017-09-08 09:31:48.420479] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.374525 (xid=0x27ae7)
[2017-09-08 09:31:48.422134] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.374607 (xid=0x27ae8)
[2017-09-08 09:31:48.424896] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.374704 (xid=0x27ae9)
[2017-09-08 09:31:48.427236] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.374817 (xid=0x27aea)
[2017-09-08 09:31:48.427802] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.374923 (xid=0x27aeb)
[2017-09-08 09:31:48.428202] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.375012 (xid=0x27aec)
[2017-09-08 09:31:48.428472] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.375130 (xid=0x27aed)
[2017-09-08 09:31:48.428728] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.375245 (xid=0x27aee)
[2017-09-08 09:31:48.429042] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.375333 (xid=0x27aef)
[2017-09-08 09:31:48.429308] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.375427 (xid=0x27af0)
[2017-09-08 09:31:48.429557] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.375531 (xid=0x27af1)
[2017-09-08 09:31:48.429800] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.375651 (xid=0x27af2)
[2017-09-08 09:31:48.430054] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.375759 (xid=0x27af3)
[2017-09-08 09:31:48.430351] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.375857 (xid=0x27af4)
[2017-09-08 09:31:48.430722] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.375956 (xid=0x27af5)
[2017-09-08 09:31:48.431245] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.376060 (xid=0x27af6)
[2017-09-08 09:31:48.431796] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.376157 (xid=0x27af7)
[2017-09-08 09:31:48.432181] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.376251 (xid=0x27af8)
[2017-09-08 09:31:48.432487] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.376346 (xid=0x27af9)
[2017-09-08 09:31:48.432785] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.376457 (xid=0x27afa)
[2017-09-08 09:31:48.433089] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.376540 (xid=0x27afb)
[2017-09-08 09:31:48.433383] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.376630 (xid=0x27afc)
[2017-09-08 09:31:48.433678] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.376734 (xid=0x27afd)
[2017-09-08 09:31:48.433967] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.376827 (xid=0x27afe)
[2017-09-08 09:31:48.434274] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.376950 (xid=0x27aff)
[2017-09-08 09:31:48.434496] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.377054 (xid=0x27b00)
[2017-09-08 09:31:48.434778] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.377153 (xid=0x27b01)
[2017-09-08 09:31:48.435069] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.377249 (xid=0x27b02)
[2017-09-08 09:31:48.435356] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.377352 (xid=0x27b03)
[2017-09-08 09:31:48.435637] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.377448 (xid=0x27b04)
[2017-09-08 09:31:48.435917] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.377554 (xid=0x27b05)
[2017-09-08 09:31:48.436210] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.377637 (xid=0x27b06)
[2017-09-08 09:31:48.436487] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.377739 (xid=0x27b07)
[2017-09-08 09:31:48.436765] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.377836 (xid=0x27b08)
[2017-09-08 09:31:48.437050] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.377938 (xid=0x27b09)
[2017-09-08 09:31:48.437332] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.378052 (xid=0x27b0a)
[2017-09-08 09:31:48.437610] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:46.378149 (xid=0x27b0b)
[2017-09-08 09:31:48.437888] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:47.378411 (xid=0x27b0c)
[2017-09-08 09:31:48.438176] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:47.378515 (xid=0x27b0d)
[2017-09-08 09:31:48.438451] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:47.378645 (xid=0x27b0e)
[2017-09-08 09:31:48.438727] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GF-DUMP)
op(NULL(2)) called at 2017-09-08 09:31:47.378696 (xid=0x27b0f)
[2017-09-08 09:31:48.438894] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:47.378881 (xid=0x27b10)
[2017-09-08 09:31:48.439196] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:47.379090 (xid=0x27b11)
[2017-09-08 09:31:48.439466] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:47.379200 (xid=0x27b12)
[2017-09-08 09:31:48.439740] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:47.379303 (xid=0x27b13)
[2017-09-08 09:31:48.440016] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:47.379403 (xid=0x27b14)
[2017-09-08 09:31:48.440302] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:47.379537 (xid=0x27b15)
[2017-09-08 09:31:48.440578] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:47.379679 (xid=0x27b16)
[2017-09-08 09:31:48.440853] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:47.379831 (xid=0x27b17)
[2017-09-08 09:31:48.441137] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:47.379921 (xid=0x27b18)
[2017-09-08 09:31:48.441416] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:47.380023 (xid=0x27b19)
[2017-09-08 09:31:48.441688] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:47.380156 (xid=0x27b1a)
[2017-09-08 09:31:48.441960] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:47.380270 (xid=0x27b1b)
[2017-09-08 09:31:48.442244] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:47.380399 (xid=0x27b1c)
[2017-09-08 09:31:48.442520] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:47.380523 (xid=0x27b1d)
[2017-09-08 09:31:48.442792] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:47.380635 (xid=0x27b1e)
[2017-09-08 09:31:48.443079] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:47.380741 (xid=0x27b1f)
[2017-09-08 09:31:48.443355] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:47.380859 (xid=0x27b20)
[2017-09-08 09:31:48.443627] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-1: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:31:47.380969 (xid=0x27b21)
The message "E [MSGID: 114031]
[client-rpc-fops.c:1593:client3_3_finodelk_cbk]
0-gv_openstack_1-client-1: remote operation failed [Transport endpoint
is not connected]" repeated 109 times between [2017-09-08
09:31:48.382451] and [2017-09-08 09:31:48.626176]
[2017-09-08 09:32:17.489261] C
[rpc-clnt-ping.c:160:rpc_clnt_ping_timer_expired]
0-gv_openstack_1-client-2: server 10.0.1.200:49152 has not responded
in the last 1 seconds, disconnecting.
[2017-09-08 09:32:17.490362] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FINODELK(30)) called at 2017-09-08 09:32:16.055593 (xid=0x3071f)
[2017-09-08 09:32:17.490401] E [MSGID: 114031]
[client-rpc-fops.c:1593:client3_3_finodelk_cbk]
0-gv_openstack_1-client-2: remote operation failed [Transport endpoint
is not connected]
[2017-09-08 09:32:17.490736] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FINODELK(30)) called at 2017-09-08 09:32:16.055623 (xid=0x30720)
[2017-09-08 09:32:17.491133] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FINODELK(30)) called at 2017-09-08 09:32:16.055654 (xid=0x30721)
[2017-09-08 09:32:17.491480] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FINODELK(30)) called at 2017-09-08 09:32:16.055684 (xid=0x30722)
[2017-09-08 09:32:17.491791] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FINODELK(30)) called at 2017-09-08 09:32:16.055712 (xid=0x30723)
[2017-09-08 09:32:17.492181] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FINODELK(30)) called at 2017-09-08 09:32:16.055741 (xid=0x30724)
[2017-09-08 09:32:17.492506] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FINODELK(30)) called at 2017-09-08 09:32:16.055770 (xid=0x30725)
[2017-09-08 09:32:17.492811] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FINODELK(30)) called at 2017-09-08 09:32:16.055799 (xid=0x30726)
[2017-09-08 09:32:17.493131] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FSYNC(16)) called at 2017-09-08 09:32:13.675475 (xid=0x2f186)
[2017-09-08 09:32:17.493461] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:16.036379 (xid=0x3071d)
[2017-09-08 09:32:17.494173] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:16.055558 (xid=0x3071e)
[2017-09-08 09:32:17.495483] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FSTAT(25)) called at 2017-09-08 09:32:16.077348 (xid=0x30727)
[2017-09-08 09:32:17.495787] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FSTAT(25)) called at 2017-09-08 09:32:16.078270 (xid=0x30728)
[2017-09-08 09:32:17.496077] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:16.484276 (xid=0x30729)
[2017-09-08 09:32:17.496378] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:16.484349 (xid=0x3072a)
[2017-09-08 09:32:17.496655] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:16.484435 (xid=0x3072b)
[2017-09-08 09:32:17.496923] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:16.484493 (xid=0x3072c)
[2017-09-08 09:32:17.497202] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:16.484553 (xid=0x3072d)
[2017-09-08 09:32:17.497509] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:16.484606 (xid=0x3072e)
[2017-09-08 09:32:17.497780] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:16.484669 (xid=0x3072f)
[2017-09-08 09:32:17.498072] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:16.484722 (xid=0x30730)
[2017-09-08 09:32:17.498397] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:16.484776 (xid=0x30731)
[2017-09-08 09:32:17.498666] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:16.484830 (xid=0x30732)
[2017-09-08 09:32:17.499121] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:16.484873 (xid=0x30733)
[2017-09-08 09:32:17.499711] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:16.484956 (xid=0x30734)
[2017-09-08 09:32:17.500893] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:16.485011 (xid=0x30735)
[2017-09-08 09:32:17.501880] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:16.485128 (xid=0x30736)
[2017-09-08 09:32:17.502892] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:16.485243 (xid=0x30737)
[2017-09-08 09:32:17.504237] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:16.485307 (xid=0x30738)
[2017-09-08 09:32:17.505447] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:16.485370 (xid=0x30739)
[2017-09-08 09:32:17.506503] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:16.485462 (xid=0x3073a)
[2017-09-08 09:32:17.507727] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:16.485521 (xid=0x3073b)
[2017-09-08 09:32:17.508173] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:16.485578 (xid=0x3073c)
[2017-09-08 09:32:17.508925] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:16.485630 (xid=0x3073d)
[2017-09-08 09:32:17.509262] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:16.485688 (xid=0x3073e)
[2017-09-08 09:32:17.509641] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:16.485764 (xid=0x3073f)
[2017-09-08 09:32:17.509957] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GF-DUMP)
op(NULL(2)) called at 2017-09-08 09:32:16.485786 (xid=0x30740)
[2017-09-08 09:32:17.510224] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.486075 (xid=0x30741)
[2017-09-08 09:32:17.511645] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.486217 (xid=0x30742)
[2017-09-08 09:32:17.511983] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.486340 (xid=0x30743)
[2017-09-08 09:32:17.512326] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.486399 (xid=0x30744)
[2017-09-08 09:32:17.512860] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.486543 (xid=0x30745)
[2017-09-08 09:32:17.514077] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.486637 (xid=0x30746)
[2017-09-08 09:32:17.514838] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.486734 (xid=0x30747)
[2017-09-08 09:32:17.515413] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.486842 (xid=0x30748)
[2017-09-08 09:32:17.516271] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.486964 (xid=0x30749)
[2017-09-08 09:32:17.516775] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.487082 (xid=0x3074a)
[2017-09-08 09:32:17.517195] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.487190 (xid=0x3074b)
[2017-09-08 09:32:17.517635] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.487289 (xid=0x3074c)
[2017-09-08 09:32:17.519469] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.487400 (xid=0x3074d)
[2017-09-08 09:32:17.520008] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.487485 (xid=0x3074e)
[2017-09-08 09:32:17.520411] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.487617 (xid=0x3074f)
[2017-09-08 09:32:17.520891] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.487699 (xid=0x30750)
[2017-09-08 09:32:17.521190] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.487836 (xid=0x30751)
[2017-09-08 09:32:17.521489] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.487914 (xid=0x30752)
[2017-09-08 09:32:17.521932] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.488019 (xid=0x30753)
[2017-09-08 09:32:17.522218] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.488125 (xid=0x30754)
[2017-09-08 09:32:17.522548] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.488201 (xid=0x30755)
[2017-09-08 09:32:17.524259] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.488302 (xid=0x30756)
[2017-09-08 09:32:17.524610] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.488375 (xid=0x30757)
[2017-09-08 09:32:17.524892] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.488463 (xid=0x30758)
[2017-09-08 09:32:17.525911] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.488555 (xid=0x30759)
[2017-09-08 09:32:17.526272] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.488635 (xid=0x3075a)
[2017-09-08 09:32:17.526555] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.488713 (xid=0x3075b)
[2017-09-08 09:32:17.528152] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.488793 (xid=0x3075c)
[2017-09-08 09:32:17.528524] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.488870 (xid=0x3075d)
[2017-09-08 09:32:17.529167] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.488948 (xid=0x3075e)
[2017-09-08 09:32:17.529571] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.489064 (xid=0x3075f)
[2017-09-08 09:32:17.529863] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.489164 (xid=0x30760)
[2017-09-08 09:32:17.530152] E [rpc-clnt.c:365:saved_frames_unwind]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fbcadb09e8b]
(--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fbcad8d08ee]
(--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fbcad8d09fe]
(--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fbcad8d2170]
(--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fbcad8d2c20] )))))
0-gv_openstack_1-client-2: forced unwinding frame type(GlusterFS 3.3)
op(FXATTROP(34)) called at 2017-09-08 09:32:17.489247 (xid=0x30761)
[2017-09-08 09:32:33.490188] E [socket.c:2316:socket_connect_finish]
0-gv_openstack_1-client-1: connection to 10.0.1.202:24007 failed (No
route to host); disconnecting socket
The message "E [MSGID: 114031]
[client-rpc-fops.c:1593:client3_3_finodelk_cbk]
0-gv_openstack_1-client-2: remote operation failed [Transport endpoint
is not connected]" repeated 72 times between [2017-09-08
09:32:17.490401] and [2017-09-08 09:32:17.530223]
2017-09-08T09:41:36.512856Z qemu-kvm: terminating on signal 15 from pid 1528
[2017-09-08 09:41:36.616581] E [MSGID: 108006]
[afr-common.c:4799:afr_notify] 0-gv_openstack_1-replicate-0: All
subvolumes are down. Going offline until atleast one of them comes
back up.
2017-09-08 09:41:37.714+0000: shutting down
-ps


On Fri, Sep 8, 2017 at 11:42 AM,  <lemonnierk at ulrar.net> wrote:
> Oh, you really don't want to go below 30s, I was told.
> I'm using 30 seconds for the timeout, and indeed when a node goes down
> the VM freez for 30 seconds, but I've never seen them go read only for
> that.
>
> I _only_ use virtio though, maybe it's that. What are you using ?
>
>
> On Fri, Sep 08, 2017 at 11:41:13AM +0200, Pavel Szalbot wrote:
>> Back to replica 3 w/o arbiter. Two fio jobs running (direct=1 and
>> direct=0), rebooting one node... and VM dmesg looks like:
>>
>> [  483.862664] blk_update_request: I/O error, dev vda, sector 23125016
>> [  483.898034] blk_update_request: I/O error, dev vda, sector 2161832
>> [  483.901103] blk_update_request: I/O error, dev vda, sector 2161832
>> [  483.904045] Aborting journal on device vda1-8.
>> [  483.906959] blk_update_request: I/O error, dev vda, sector 2099200
>> [  483.908306] blk_update_request: I/O error, dev vda, sector 2099200
>> [  483.909585] Buffer I/O error on dev vda1, logical block 262144,
>> lost sync page write
>> [  483.911121] blk_update_request: I/O error, dev vda, sector 2048
>> [  483.912192] blk_update_request: I/O error, dev vda, sector 2048
>> [  483.913221] Buffer I/O error on dev vda1, logical block 0, lost
>> sync page write
>> [  483.914546] EXT4-fs error (device vda1):
>> ext4_journal_check_start:56: Detected aborted journal
>> [  483.916230] EXT4-fs (vda1): Remounting filesystem read-only
>> [  483.917231] EXT4-fs (vda1): previous I/O error to superblock detected
>> [  483.917353] JBD2: Error -5 detected when updating journal
>> superblock for vda1-8.
>> [  483.921106] blk_update_request: I/O error, dev vda, sector 2048
>> [  483.922147] blk_update_request: I/O error, dev vda, sector 2048
>> [  483.923107] Buffer I/O error on dev vda1, logical block 0, lost
>> sync page write
>>
>> Root fs is read-only even with 1s ping-timeout...
>>
>> I really hope I have been idiot for almost a year now and someone
>> shows what am I doing completely wrong because I dream about joining
>> the hordes of fellow colleagues who store multiple VMs in gluster and
>> never had a problem with it. I also suspect the CentOS libvirt version
>> to be the cause.
>>
>> -ps
>>
>>
>> On Fri, Sep 8, 2017 at 10:50 AM, Pavel Szalbot <pavel.szalbot at gmail.com> wrote:
>> > FYI I set up replica 3 (no arbiter this time), did the same thing -
>> > rebooted one node during lots of file IO on VM and IO stopped.
>> >
>> > As I mentioned either here or in another thread, this behavior is
>> > caused by high default of network.ping-timeout. My main problem used
>> > to be that setting it to low values like 3s or even 2s did not prevent
>> > the FS to be mounted as read-only in the past (at least with arbiter)
>> > and docs describe reconnect as very costly. If I set ping-timeout to
>> > 1s disaster of read-only mount is now prevented.
>> >
>> > However I find it very strange because in the past I actually did end
>> > up with read-only filesystem despite of the low ping-timeout.
>> >
>> > With replica 3 after node reboot iftop shows data flowing only to the
>> > one of remaining two nodes and there is no entry in heal info for the
>> > volume. Explanation would be very much appreciated ;-)
>> >
>> > Few minutes later I reverted back to replica 3 with arbiter (group
>> > virt, ping-timeout 1). All nodes are up. During first fio run, VM
>> > disconnected my ssh session, so I reconnected and saw ext4 problems in
>> > dmesg. I deleted the VM and started a new one. Glustershd.log fills
>> > with metadata heal shortly after fio job starts, but this time system
>> > is stable.
>> > Rebooting one of the nodes does not cause any problem (watching heal
>> > log, i/o on vm).
>> >
>> > So I decided to put more stress on VMs disk - I added second job with
>> > direct=1 and started it (now both are running) while one gluster node
>> > is still booting. What happened? One fio job reports "Bus error" and
>> > VM segfaults when trying to run dmesg...
>> >
>> > Is this gfapi related? Is this bug in arbiter?
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> http://lists.gluster.org/mailman/listinfo/gluster-users
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users