[2016-03-15 14:35:06.465165] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:35:06.484830] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:35:06.485694] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:35:06.485707] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:35:06.486095] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:35:06.486106] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:35:06.486112] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:35:06.486150] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:35:06.486200] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:35:06.486208] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:35:09.921504] E [store.c:408:gf_store_handle_retrieve] 0-: Path corresponding to /var/lib/glusterd/glusterd.info, returned error: (No such file or directory) [2016-03-15 14:35:09.921532] E [store.c:408:gf_store_handle_retrieve] 0-: Path corresponding to /var/lib/glusterd/glusterd.info, returned error: (No such file or directory) [2016-03-15 14:35:09.921539] I [glusterd-store.c:1441:glusterd_restore_op_version] 0-management: Detected new install. Setting op-version to maximum : 30504 [2016-03-15 14:35:09.921621] E [store.c:408:gf_store_handle_retrieve] 0-: Path corresponding to /var/lib/glusterd/options, returned error: (No such file or directory) Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:36:45.782158] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:36:45.783293] I [glusterd-handler.c:918:__glusterd_handle_cli_probe] 0-glusterd: Received CLI probe req 172.31.30.64 24007 [2016-03-15 14:36:45.786102] I [glusterd-handler.c:2931:glusterd_probe_begin] 0-glusterd: Unable to find peerinfo for host: 172.31.30.64 (24007) [2016-03-15 14:36:53.007545] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:36:53.007632] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:36:53.007645] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:36:53.008052] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:36:53.008082] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:36:56.158216] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:00.705728] E [store.c:408:gf_store_handle_retrieve] 0-: Path corresponding to /var/lib/glusterd/glusterd.info, returned error: (No such file or directory) [2016-03-15 14:37:00.705782] I [glusterd.c:168:glusterd_uuid_generate_save] 0-management: generated UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:37:07.138906] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:09.277697] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:11.403758] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:13.459017] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:15.515262] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:17.571812] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:19.634481] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:21.734343] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:22.147463] I [glusterd-rpc-ops.c:234:__glusterd_probe_cbk] 0-glusterd: Received probe resp from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:37:22.168148] I [glusterd-rpc-ops.c:306:__glusterd_probe_cbk] 0-glusterd: Received resp to probe req [2016-03-15 14:37:22.168215] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:37:22.367457] I [glusterd-handler.c:918:__glusterd_handle_cli_probe] 0-glusterd: Received CLI probe req 172.31.27.27 24007 [2016-03-15 14:37:22.368612] I [glusterd-handler.c:2931:glusterd_probe_begin] 0-glusterd: Unable to find peerinfo for host: 172.31.27.27 (24007) [2016-03-15 14:37:22.395611] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:22.395686] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:22.395702] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:37:22.396054] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:23.823457] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:25.879010] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:27.933933] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:30.092223] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:32.148269] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:34.206242] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:36.261697] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:38.317163] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:40.418302] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:40.571044] I [glusterd-rpc-ops.c:234:__glusterd_probe_cbk] 0-glusterd: Received probe resp from uuid: acfc1794-9080-4eb0-8f69-3abe78bbee16, host: 172.31.27.27 [2016-03-15 14:37:40.693658] I [glusterd-rpc-ops.c:306:__glusterd_probe_cbk] 0-glusterd: Received resp to probe req [2016-03-15 14:37:40.693723] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: acfc1794-9080-4eb0-8f69-3abe78bbee16, host: 172.31.27.27, port: 0 [2016-03-15 14:37:40.996043] I [glusterd-handler.c:918:__glusterd_handle_cli_probe] 0-glusterd: Received CLI probe req 172.31.26.134 24007 [2016-03-15 14:37:40.996588] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:37:41.151224] I [glusterd-handler.c:918:__glusterd_handle_cli_probe] 0-glusterd: Received CLI probe req 172.31.19.46 24007 [2016-03-15 14:37:41.153210] I [glusterd-handler.c:2931:glusterd_probe_begin] 0-glusterd: Unable to find peerinfo for host: 172.31.19.46 (24007) [2016-03-15 14:37:41.290294] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:41.290360] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:41.290368] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:37:41.290620] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:41.295737] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:37:41.380003] I [glusterd-handler.c:2346:__glusterd_handle_probe_query] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:41.380059] I [glusterd-handler.c:2398:__glusterd_handle_probe_query] 0-glusterd: Responded to 172.31.30.64, op_ret: 0, op_errno: 0, ret: 0 [2016-03-15 14:37:41.395401] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:41.395459] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:37:41.417433] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:37:41.444354] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:41.444385] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:37:41.444392] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:37:41.444442] I [glusterd-rpc-ops.c:234:__glusterd_probe_cbk] 0-glusterd: Received probe resp from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:37:41.519256] I [glusterd-rpc-ops.c:306:__glusterd_probe_cbk] 0-glusterd: Received resp to probe req [2016-03-15 14:37:41.519352] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:41.519371] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:37:41.646776] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:37:41.647238] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:37:41.657372] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fb64664247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fb646915182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fb64740f265]))) 0-: received signum (15), shutting down [2016-03-15 14:37:41.681593] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:37:41.685045] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:37:41.685543] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:37:41.685555] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:37:41.685893] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:37:41.685905] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:37:41.685910] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:37:41.685946] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:37:41.685997] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:37:41.686011] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:37:43.493948] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:37:43.497804] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:43.501458] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:43.504595] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:43.504634] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:43.504681] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:43.504688] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:37:43.505180] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:43.505218] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:43.505226] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:37:43.505440] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:43.505475] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:43.505483] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:37:43.510927] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:43.511875] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:37:43.511930] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:37:43.585772] I [glusterd-rpc-ops.c:234:__glusterd_probe_cbk] 0-glusterd: Received probe resp from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:37:43.693226] I [glusterd-rpc-ops.c:306:__glusterd_probe_cbk] 0-glusterd: Received resp to probe req [2016-03-15 14:37:43.718605] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:43.718662] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:37:43.739251] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:37:43.801542] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:37:43.826785] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:43.826814] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:37:43.826822] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:37:43.826882] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:44.426414] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:37:44.426458] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:37:44.426468] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:37:44.462144] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:44.520713] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:37:44.520770] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:37:44.555108] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:37:44.555168] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:37:44.580118] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:37:44.580141] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:37:44.619547] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:44.645063] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:37:44.684319] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:44.684355] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:37:44.684362] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:37:44.684369] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:37:45.566344] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:46.748401] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:37:46.759601] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f53b48f947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f53b4bcc182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f53b56c6265]))) 0-: received signum (15), shutting down [2016-03-15 14:37:46.770859] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:37:46.774727] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:37:46.776901] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:37:46.776914] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:37:46.777249] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:37:46.777260] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:37:46.777266] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:37:46.777302] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:37:46.777360] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:37:46.777369] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:37:48.404612] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:37:48.407576] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:48.410372] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:48.412975] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:48.413014] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:48.413063] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:48.413070] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:37:48.413605] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:48.413641] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:48.413651] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:37:48.413866] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:48.413910] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:48.413923] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:37:48.417819] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:48.418814] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:37:48.418869] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:37:48.452631] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:37:48.487374] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:37:48.487431] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:37:48.506201] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:37:48.506226] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:37:48.545514] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:37:48.545558] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:37:48.545582] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:37:48.545589] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:37:48.545604] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:37:49.152841] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:37:49.173865] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:49.173894] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:37:49.173902] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:37:49.173909] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:37:49.231739] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:37:49.247929] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:49.247980] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:37:49.280244] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:37:49.280271] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:37:49.343739] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:37:49.343776] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:50.473985] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:51.834948] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:37:51.845350] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fadbed6747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fadbf03a182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fadbfb34265]))) 0-: received signum (15), shutting down [2016-03-15 14:37:51.857941] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:37:51.861033] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:37:51.861714] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:37:51.861731] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:37:51.862244] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:37:51.862261] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:37:51.862271] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:37:51.862321] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:37:51.862398] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:37:51.862413] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:37:54.195850] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:37:54.200126] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:54.202936] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:54.207738] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:54.207776] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:54.207824] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:54.207832] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:37:54.208329] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:54.208364] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:54.208371] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:37:54.208584] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:54.208620] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:54.208628] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:37:54.212797] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:54.213779] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:37:54.213832] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:37:54.229172] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:37:54.270884] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:54.270941] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:37:54.288831] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:37:54.288854] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:37:54.351587] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:37:54.351640] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:37:54.375388] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:37:54.375412] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:37:54.397364] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:37:54.444034] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:37:54.464330] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:37:54.464369] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:37:54.464377] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:37:54.464384] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:37:54.464441] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:54.464466] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:54.464480] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:37:54.464485] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:37:54.464492] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:37:54.464527] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:37:54.464546] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:56.269085] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:56.918566] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:37:56.928716] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ffb9e03247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ffb9e305182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ffb9edff265]))) 0-: received signum (15), shutting down [2016-03-15 14:37:56.940090] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:37:56.943955] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:37:56.944544] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:37:56.944557] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:37:56.944900] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:37:56.944912] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:37:56.944918] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:37:56.944953] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:37:56.945006] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:37:56.945014] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:37:58.550342] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:37:58.553316] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:58.556329] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:58.559793] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:58.559832] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:58.559879] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:58.559892] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:37:58.560453] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:58.560488] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:58.560495] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:37:58.560741] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:58.560778] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:58.560785] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:37:58.565217] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:58.566135] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:37:58.566256] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:37:58.664947] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:58.665010] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:37:58.683845] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:37:58.683862] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:37:58.717982] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:37:58.718007] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:37:58.748188] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:37:58.781820] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:58.781851] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:37:58.781858] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:37:58.781865] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:37:58.807765] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:59.429950] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:37:59.447868] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:37:59.447915] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:37:59.466384] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:37:59.466402] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:37:59.482969] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:37:59.483004] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:00.621705] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:02.002265] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:38:02.012409] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f36b554647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f36b5819182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f36b6313265]))) 0-: received signum (15), shutting down [2016-03-15 14:38:02.025130] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:38:02.029369] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:38:02.029971] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:02.029984] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:02.030327] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:38:02.030338] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:38:02.030344] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:38:02.030379] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:38:02.030430] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:02.030439] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:03.606543] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:38:03.610079] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:03.613588] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:03.618206] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:03.618267] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:03.618324] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:03.618333] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:03.618900] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:03.618939] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:03.618947] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:03.619165] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:03.619203] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:03.619212] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:38:03.625440] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:03.626328] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:03.626386] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:38:03.721979] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:03.722045] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:38:03.739056] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:03.739083] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:03.762899] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:38:03.783907] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:03.783941] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:03.783949] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:03.783956] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:38:03.784010] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:38:03.823032] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:04.119151] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:04.156454] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:04.191956] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:04.192012] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:38:04.214558] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:04.214585] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:04.233037] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:04.266202] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:05.680978] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:07.085803] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:38:07.096931] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fb55961447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fb5598e7182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fb55a3e1265]))) 0-: received signum (15), shutting down [2016-03-15 14:38:07.108768] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:38:07.112689] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:38:07.113385] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:07.113402] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:07.113902] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:38:07.113919] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:38:07.113929] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:38:07.113976] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:38:07.114047] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:07.114060] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:08.662228] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:38:08.666254] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:08.670098] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:08.672997] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:08.673036] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:08.673082] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:08.673090] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:08.673610] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:08.673648] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:08.673656] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:08.673871] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:08.673911] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:08.673923] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:38:08.677763] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:08.678511] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:08.678563] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:38:08.690995] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:08.723250] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:38:08.757064] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:08.757123] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:38:08.776197] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:08.776229] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:08.812565] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:08.812631] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:38:08.830733] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:08.830768] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:08.851954] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:08.851985] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:08.852003] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:08.852010] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:08.852016] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:38:08.852055] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:38:08.872958] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:08.872985] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:08.872992] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:08.872998] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:38:08.873042] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:08.873062] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:10.733590] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:12.168270] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:38:12.178685] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f7a3ca1347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f7a3cce6182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f7a3d7e0265]))) 0-: received signum (15), shutting down [2016-03-15 14:38:12.190153] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:38:12.193237] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:38:12.193797] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:12.193809] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:12.194143] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:38:12.194154] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:38:12.194160] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:38:12.194196] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:38:12.194245] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:12.194253] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:13.850608] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:38:13.853949] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:13.858680] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:13.877202] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:13.877251] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:13.877303] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:13.877310] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:13.877891] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:13.877933] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:13.877943] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:13.878207] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:13.878240] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:13.878247] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:38:13.883748] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:13.884818] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:13.884874] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:38:13.953662] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:13.953727] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:38:13.975131] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:13.975157] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:14.039604] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:38:14.063546] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:38:14.084798] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:14.084846] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:14.084879] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:14.084892] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:14.084904] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:38:14.432905] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:14.450166] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:14.450213] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:38:14.494613] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:14.494636] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:14.515220] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:14.515250] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:15.940030] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:17.252653] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:38:17.263117] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f0bf9b2947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f0bf9dfc182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f0bfa8f6265]))) 0-: received signum (15), shutting down [2016-03-15 14:38:17.274144] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:38:17.278038] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:38:17.278627] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:17.278640] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:17.279049] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:38:17.279062] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:38:17.279068] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:38:17.279103] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:38:17.279154] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:17.279162] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:18.967144] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:38:18.970954] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:18.975137] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:18.978281] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:18.978320] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:18.978368] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:18.978375] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:18.978988] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:18.979028] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:18.979036] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:18.979267] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:18.979305] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:18.979313] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:38:18.984277] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:18.985414] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:18.985465] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:38:19.055873] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:19.055936] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:38:19.075643] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:19.075677] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:19.096435] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:19.096460] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:38:19.127817] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:38:19.161352] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:19.161386] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:19.161394] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:19.161401] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:38:19.162058] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:19.198823] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:19.199117] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:19.199165] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:38:19.221156] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:19.221190] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:19.303297] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:19.303331] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:21.041123] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:22.336655] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:38:22.347245] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f5613f6547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f5614238182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f5614d32265]))) 0-: received signum (15), shutting down [2016-03-15 14:38:22.358923] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:38:22.362123] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:38:22.363022] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:22.363041] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:22.363538] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:38:22.363556] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:38:22.363565] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:38:22.363625] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:38:22.363702] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:22.363716] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:23.976877] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:38:23.980257] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:23.983238] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:23.987282] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:23.987321] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:23.987366] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:23.987373] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:23.987941] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:23.987976] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:23.987983] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:23.988250] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:23.988281] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:23.988288] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:38:23.993906] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:23.995020] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:23.995072] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:38:24.071951] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:24.072018] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:38:24.091866] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:24.091889] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:24.173320] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:24.173374] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:24.173388] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:24.173401] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:38:24.173455] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:38:24.194978] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:24.449346] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:38:25.160351] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:25.206009] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:25.206067] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:38:25.244212] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:25.244234] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:25.274421] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:25.274503] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:26.049733] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:27.428739] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:38:27.440023] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fc9f132047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fc9f15f3182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fc9f20ed265]))) 0-: received signum (15), shutting down [2016-03-15 14:38:27.452618] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:38:27.454887] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:38:27.455395] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:27.455407] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:27.455752] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:38:27.455764] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:38:27.455770] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:38:27.455806] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:38:27.455858] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:27.455866] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:29.037004] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:38:29.040794] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:29.043763] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:29.046043] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:29.046083] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:29.046133] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:29.046141] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:29.046686] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:29.046721] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:29.046728] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:29.046983] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:29.047026] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:29.047036] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:38:29.051201] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:29.052284] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:29.052345] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:38:29.097479] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:29.097542] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:38:29.144214] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:29.144242] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:29.183894] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:38:29.205860] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:38:29.228831] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:29.228866] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:29.228887] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:29.228894] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:29.228901] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:38:29.435891] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:29.505044] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:29.505095] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:38:29.535844] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:29.535880] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:29.557864] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:29.557902] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:31.106572] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:32.516674] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:38:32.527961] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f56f9e4a47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f56fa11d182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f56fac17265]))) 0-: received signum (15), shutting down [2016-03-15 14:38:32.539509] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:38:32.543889] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:38:32.544501] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:32.544521] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:32.544861] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:38:32.544872] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:38:32.544878] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:38:32.544913] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:38:32.544964] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:32.544971] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:34.186345] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:38:34.191730] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:34.195610] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:34.198807] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:34.198848] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:34.198896] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:34.198904] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:34.199473] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:34.199508] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:34.199516] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:34.199778] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:34.199811] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:34.199818] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:38:34.203816] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:34.204667] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:34.204719] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:38:34.284395] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:34.284456] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:38:34.306738] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:34.306764] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:34.363409] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:34.363442] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:34.363456] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:34.363464] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:38:34.363507] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:38:34.381990] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:34.944484] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:38:35.437091] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:35.454898] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:35.454962] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:38:35.491412] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:35.491438] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:35.508440] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:35.508887] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:36.259786] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:37.599747] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:38:37.609998] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f539434947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f539461c182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f5395116265]))) 0-: received signum (15), shutting down [2016-03-15 14:38:37.621460] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:38:37.624802] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:38:37.625293] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:37.625305] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:37.625648] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:38:37.625662] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:38:37.625669] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:38:37.625705] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:38:37.625756] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:37.625764] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:39.252633] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:38:39.255841] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:39.258926] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:39.262058] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:39.262098] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:39.262145] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:39.262153] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:39.262712] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:39.262779] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:39.262800] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:39.263031] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:39.263066] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:39.263073] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:38:39.266631] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:39.267956] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:39.268011] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:38:39.341370] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:39.341432] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:38:39.364287] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:39.364306] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:39.381689] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:38:39.401547] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:39.401578] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:39.401586] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:39.401593] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:38:39.401639] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:38:39.420391] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:39.420412] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:40.163342] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:40.199395] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:40.199449] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:38:40.226821] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:40.226854] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:40.245658] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:40.264503] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:41.321803] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:42.686914] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:38:42.700166] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fde05c8947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fde05f5c182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fde06a56265]))) 0-: received signum (15), shutting down [2016-03-15 14:38:42.712407] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:38:42.716564] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:38:42.717246] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:42.717259] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:42.717596] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:38:42.717608] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:38:42.717614] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:38:42.717649] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:38:42.717700] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:42.717708] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:44.468688] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:38:44.471914] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:44.475103] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:44.493151] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:44.493195] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:44.493247] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:44.493254] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:44.493833] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:44.493869] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:44.493876] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:44.494155] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:44.494187] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:44.494203] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:38:44.498787] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:44.499537] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:44.499596] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:38:44.512935] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:44.529406] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:44.589665] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:44.589728] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:38:44.608864] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:44.608900] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:44.725199] I [glusterd-rpc-ops.c:234:__glusterd_probe_cbk] 0-glusterd: Received probe resp from uuid: acfc1794-9080-4eb0-8f69-3abe78bbee16, host: 172.31.27.27 [2016-03-15 14:38:44.758349] I [glusterd-rpc-ops.c:306:__glusterd_probe_cbk] 0-glusterd: Received resp to probe req [2016-03-15 14:38:44.804400] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:44.804461] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:38:44.822050] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:44.822067] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:44.843066] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:44.843097] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:44.843104] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:44.843111] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:38:44.843159] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:38:44.864180] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:38:44.888429] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:44.888465] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:44.888472] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:44.888479] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:38:44.888531] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:44.888549] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:46.555224] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:47.775481] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:38:47.786765] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f95c419847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f95c446b182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f95c4f65265]))) 0-: received signum (15), shutting down [2016-03-15 14:38:47.798473] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:38:47.801817] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:38:47.802319] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:47.802331] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:47.802668] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:38:47.802679] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:38:47.802685] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:38:47.802721] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:38:47.803770] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:47.803789] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:49.642818] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:38:49.645928] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:49.648866] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:49.651958] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:49.651997] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:49.652044] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:49.652052] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:49.652550] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:49.652585] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:49.652593] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:49.652814] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:49.652859] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:49.652872] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:38:49.658452] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:49.659714] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:49.659767] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:38:49.687854] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:49.800852] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:49.800929] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:38:49.825692] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:49.825716] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:49.848227] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:38:49.866769] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:38:49.889600] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:49.889636] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:49.889644] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:49.889651] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:38:49.889697] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:49.889719] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:50.440082] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:50.457789] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:50.457834] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:38:50.478613] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:50.478641] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:50.495427] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:50.556180] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:51.715695] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:52.862485] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:38:52.873029] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7feb224b147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7feb22784182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7feb2327e265]))) 0-: received signum (15), shutting down [2016-03-15 14:38:52.884345] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:38:52.888285] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:38:52.888885] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:52.888898] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:52.889241] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:38:52.889252] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:38:52.889258] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:38:52.889293] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:38:52.889344] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:52.889351] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:54.908905] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:38:54.916130] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:54.920630] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:54.925105] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:54.925147] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:54.925198] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:54.925205] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:54.925782] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:54.925823] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:54.925831] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:54.926045] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:54.926083] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:54.926091] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:38:54.930655] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:54.931727] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:54.931798] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:38:55.036841] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:55.036909] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:38:55.075398] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:55.075435] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:55.132623] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:38:55.605147] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:55.605186] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:55.605195] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:55.605202] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:38:55.605281] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:55.605313] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:55.605335] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:38:55.652145] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:55.669232] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:55.683801] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:55.683851] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:38:55.710631] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:55.710660] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:55.737306] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:55.787907] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:56.987820] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:57.946365] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:38:57.956840] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f04d217847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f04d244b182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f04d2f45265]))) 0-: received signum (15), shutting down [2016-03-15 14:38:57.969156] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:38:57.973179] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:38:57.973909] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:57.973926] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:57.974449] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:38:57.974466] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:38:57.974475] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:38:57.974523] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:38:57.974606] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:57.974621] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:59.693485] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:38:59.698405] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:59.703723] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:59.707023] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:59.707063] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:59.707114] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:59.707121] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:59.707699] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:59.707735] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:59.707742] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:59.707975] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:59.708012] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:59.708021] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:38:59.714494] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:59.715425] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:59.715482] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:38:59.735839] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:59.756775] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:59.890628] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:59.890698] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:38:59.915730] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:59.915774] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:59.932860] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:38:59.961468] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:38:59.997951] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:59.997986] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:59.997994] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:59.998000] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:38:59.998043] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:59.998083] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:39:00.043528] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:00.043554] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:00.114879] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:00.114915] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:00.114941] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:00.114960] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:00.114966] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:00.114973] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:39:00.115023] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:00.115045] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:01.771715] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:03.032881] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:39:03.043527] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f0e64a6747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f0e64d3a182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f0e65834265]))) 0-: received signum (15), shutting down [2016-03-15 14:39:03.056104] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:39:03.060188] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:39:03.060705] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:03.060718] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:03.061062] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:39:03.061074] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:39:03.061079] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:39:03.061115] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:39:03.061165] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:03.061173] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:04.706184] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:39:04.710711] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:04.714358] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:04.717723] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:04.717763] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:04.717812] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:04.717820] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:04.718385] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:04.718427] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:04.718436] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:04.718691] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:04.718727] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:04.718763] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:39:04.722708] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:04.723847] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:04.723930] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:39:04.759917] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:04.865852] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:04.865917] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:39:04.888075] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:04.888110] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:04.920138] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:39:04.945528] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:39:04.970421] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:04.970460] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:04.970469] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:04.970476] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:39:04.970545] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:04.970563] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:05.442991] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:05.458131] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:05.458189] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:39:05.477813] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:05.477835] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:05.517527] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:05.547653] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:06.780481] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:08.119478] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:39:08.130665] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f1f53b1347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f1f53de6182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f1f548e0265]))) 0-: received signum (15), shutting down [2016-03-15 14:39:08.142371] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:39:08.146484] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:39:08.147547] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:08.147566] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:08.148062] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:39:08.148079] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:39:08.148089] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:39:08.148141] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:39:08.148212] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:08.148223] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:09.760152] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:39:09.763003] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:09.766575] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:09.769558] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:09.769602] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:09.769653] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:09.769661] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:09.770256] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:09.770299] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:09.770308] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:09.770576] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:09.770608] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:09.770615] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:39:09.774413] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:09.775420] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:09.775475] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:39:09.865913] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:39:09.884021] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:09.884073] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:39:09.904353] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:09.904379] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:09.963676] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:09.963717] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:09.963737] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:09.963744] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:09.963751] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:39:10.168531] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:39:10.253872] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:10.253920] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:10.253928] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:10.253935] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:39:10.272673] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:10.287726] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:10.287769] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:39:10.339574] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:10.339601] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:10.360272] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:10.360300] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:11.419797] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:11.830427] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:13.203921] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:39:13.214149] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f122ce7b47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f122d14e182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f122dc48265]))) 0-: received signum (15), shutting down [2016-03-15 14:39:13.225876] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:39:13.228733] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:39:13.229235] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:13.229246] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:13.229586] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:39:13.229597] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:39:13.229603] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:39:13.229641] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:39:13.229695] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:13.229703] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:14.959654] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:39:14.964178] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:14.968290] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:14.973325] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:14.973377] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:14.973427] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:14.973435] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:14.973963] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:14.973999] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:14.974006] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:14.974230] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:14.974275] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:14.974288] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:39:14.978652] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:14.979508] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:14.979561] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:39:15.011320] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:15.086689] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:39:15.108567] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:15.108621] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:39:15.132941] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:15.132974] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:15.180161] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:15.180211] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:15.180232] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:15.180239] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:15.180245] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:39:15.827505] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:39:16.170203] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:16.186866] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:16.186920] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:39:16.237649] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:16.237677] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:16.273202] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:16.273231] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:17.035367] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:18.289858] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:39:18.300127] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7faf9368047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7faf93953182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7faf9444d265]))) 0-: received signum (15), shutting down [2016-03-15 14:39:18.311585] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:39:18.316139] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:39:18.316836] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:18.316853] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:18.317206] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:39:18.317218] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:39:18.317224] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:39:18.317257] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:39:18.317308] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:18.317316] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:20.068767] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:39:20.072796] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:20.076047] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:20.079488] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:20.079528] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:20.079579] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:20.079587] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:20.080072] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:20.080110] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:20.080118] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:20.080337] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:20.080376] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:20.080388] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:39:20.085026] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:20.086099] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:20.086152] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:39:20.118662] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:20.131399] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:20.131450] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:39:20.152628] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:20.152663] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:20.210880] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:39:20.232491] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:39:20.278875] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:20.278911] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:20.278918] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:20.278925] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:39:20.446067] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:20.461736] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:20.461792] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:39:20.499775] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:20.499819] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:20.529679] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:20.565553] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:22.143502] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:23.373672] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:39:23.384193] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fcb0147547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fcb01748182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fcb02242265]))) 0-: received signum (15), shutting down [2016-03-15 14:39:23.395794] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:39:23.397932] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:39:23.398428] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:23.398439] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:23.398864] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:39:23.398877] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:39:23.398883] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:39:23.398943] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:39:23.398998] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:23.399006] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:25.032826] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:39:25.036725] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:25.040284] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:25.043783] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:25.043823] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:25.043872] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:25.043879] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:25.044405] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:25.044444] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:25.044452] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:25.044672] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:25.044716] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:25.044729] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:39:25.048995] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:25.050093] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:25.050147] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:39:25.228259] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:39:25.254939] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:25.254981] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:25.254990] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:25.254997] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:39:25.264473] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:25.310212] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:25.310264] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:39:25.334392] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:25.334420] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:25.353538] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:25.353608] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:39:25.376685] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:25.376709] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:25.395602] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:25.395636] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:25.618187] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:25.661294] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:39:26.021497] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:26.021542] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:26.021550] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:26.021557] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:39:26.168193] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:26.168252] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:27.105017] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:28.466278] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:39:28.477659] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f5ee375f47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f5ee3a32182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f5ee452c265]))) 0-: received signum (15), shutting down [2016-03-15 14:39:28.488757] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:39:28.492681] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:39:28.493269] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:28.493282] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:28.493666] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:39:28.493678] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:39:28.493683] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:39:28.493719] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:39:28.493770] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:28.493778] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:30.118533] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:39:30.122139] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:30.125396] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:30.129936] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:30.129976] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:30.130030] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:30.130038] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:30.130616] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:30.130651] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:30.130658] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:30.130954] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:30.130987] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:30.130994] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:39:30.136306] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:30.137194] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:30.137272] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:39:30.151315] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:30.251550] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:39:30.298818] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:30.298876] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:39:30.357536] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:30.357569] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:30.484834] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:30.484871] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:39:30.532502] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:30.532540] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:30.532547] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:30.532554] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:39:30.555820] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:31.173324] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:31.202565] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:31.202621] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:39:31.218282] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:31.218299] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:31.236105] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:31.274181] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:32.192376] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:33.551535] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:39:33.562030] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f65b913c47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f65b940f182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f65b9f09265]))) 0-: received signum (15), shutting down [2016-03-15 14:39:33.573866] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:39:33.577344] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:39:33.577834] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:33.577846] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:33.578194] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:39:33.578211] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:39:33.578217] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:39:33.578253] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:39:33.578303] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:33.578311] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:35.174494] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:39:35.177757] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:35.181137] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:35.184406] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:35.184445] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:35.184492] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:35.184499] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:35.185062] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:35.185099] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:35.185107] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:35.185358] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:35.185396] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:35.185404] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:39:35.189322] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:35.189822] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:35.189930] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:39:35.204975] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:35.205040] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:39:35.224349] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:35.224403] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:35.279781] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:35.373328] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:39:35.394123] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:35.394167] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:35.394175] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:35.394182] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:39:35.394229] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:39:35.448592] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:35.462961] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:35.463011] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:39:35.481486] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:35.481523] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:35.499602] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:35.502638] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:37.244911] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:38.640711] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:39:38.651188] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f516782147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f5167af4182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f51685ee265]))) 0-: received signum (15), shutting down [2016-03-15 14:39:38.662692] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:39:38.666698] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:39:38.667590] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:38.667604] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:38.667947] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:39:38.667959] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:39:38.667965] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:39:38.668001] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:39:38.668051] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:38.668059] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:40.339524] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:39:40.342817] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:40.348630] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:40.366921] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:40.366966] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:40.367023] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:40.367031] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:40.367544] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:40.367582] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:40.367590] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:40.367813] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:40.367849] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:40.367857] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:39:40.372867] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:40.374046] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:40.374118] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:39:40.387702] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:40.387749] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:39:40.407545] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:40.407571] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:40.529542] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:39:40.568090] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:40.568133] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:40.568141] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:40.568148] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:39:40.568522] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:39:40.625353] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:41.449879] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:41.466524] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:41.466570] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:39:41.497953] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:41.497980] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:41.521531] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:41.552676] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:42.429421] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:43.725208] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:39:43.737322] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fc9b051947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fc9b07ec182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fc9b12e6265]))) 0-: received signum (15), shutting down [2016-03-15 14:39:43.748570] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:39:43.752555] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:39:43.753068] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:43.753080] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:43.753456] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:39:43.753468] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:39:43.753474] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:39:43.753508] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:39:43.753559] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:43.753567] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:45.378448] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:39:45.381286] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:45.385230] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:45.389727] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:45.389770] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:45.389856] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:45.389872] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:45.390518] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:45.390565] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:45.390579] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:45.390940] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:45.390987] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:45.391000] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:39:45.396215] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:45.396276] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:39:45.411166] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:45.513196] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:45.513278] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:39:45.536765] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:45.536803] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:45.556798] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:39:45.587554] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:45.587588] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:45.587596] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:45.587603] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:39:45.587653] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:39:45.610270] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:45.610298] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:46.176071] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:46.190768] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:46.190823] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:39:46.210574] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:46.210592] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:46.233584] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:46.239087] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:46.612736] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:47.468346] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:48.810327] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:39:48.821988] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f29763b747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f297668a182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f2977184265]))) 0-: received signum (15), shutting down [2016-03-15 14:39:48.834518] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:39:48.836848] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:39:48.837352] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:48.837363] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:48.837702] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:39:48.837713] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:39:48.837719] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:39:48.837754] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:39:48.837805] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:48.837813] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:50.437322] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:39:50.440473] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:50.444434] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:50.447248] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:50.447297] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:50.447355] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:50.447365] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:50.447915] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:50.447950] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:50.447958] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:50.448193] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:50.448230] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:50.448238] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:39:50.451843] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:50.452604] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:50.452659] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:39:50.470262] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:50.518000] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:39:50.548059] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:50.578474] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:50.578508] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:50.578515] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:50.578523] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:39:50.613080] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:50.613144] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:39:50.629949] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:50.629982] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:50.648870] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:50.648929] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:39:50.665960] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:50.665994] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:50.685905] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:39:50.706010] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:50.706043] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:50.706050] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:50.706057] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:39:50.706109] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:50.706127] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:50.706145] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:50.706157] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:52.508424] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:53.902552] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:39:53.913312] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f79dd02847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f79dd2fb182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f79dddf5265]))) 0-: received signum (15), shutting down [2016-03-15 14:39:53.925363] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:39:53.928334] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:39:53.929026] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:53.929044] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:53.929556] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:39:53.929574] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:39:53.929584] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:39:53.929636] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:39:53.929714] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:53.929729] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:55.576338] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:39:55.581034] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:55.585643] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:55.590129] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:55.590169] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:55.590217] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:55.590225] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:55.590767] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:55.590805] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:55.590813] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:55.591048] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:55.591092] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:55.591104] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:39:55.596870] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:55.597728] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:55.597786] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:39:55.713733] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:55.713798] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:39:55.760622] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:55.760659] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:55.794841] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:39:55.839246] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:55.839279] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:55.839286] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:55.839293] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:39:55.839372] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:39:55.870746] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:55.870804] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:55.890027] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:56.452828] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:56.469051] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:56.469102] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:39:56.867246] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:56.867283] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:56.938952] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:56.938987] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:57.653632] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:58.992428] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:39:59.002622] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f453968747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f453995a182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f453a454265]))) 0-: received signum (15), shutting down [2016-03-15 14:39:59.013979] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:39:59.017057] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:39:59.017563] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:59.017574] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:59.017919] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:39:59.017936] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:39:59.017942] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:39:59.017977] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:39:59.018027] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:59.018035] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:00.691920] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:40:00.696092] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:00.704454] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:00.708671] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:00.708711] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:00.708762] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:00.708769] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:00.709351] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:00.709387] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:00.709395] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:00.709656] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:00.709692] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:00.709700] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:40:00.713581] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:00.714401] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:00.714454] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:40:00.834913] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:40:00.878851] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:40:00.928067] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:00.928132] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:40:00.966433] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:00.966473] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:01.032268] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:01.032309] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:01.032336] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:01.032358] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:01.032364] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:01.032371] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:40:01.178909] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:01.240968] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:01.241034] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:40:01.258409] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:01.258429] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:01.297713] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:01.297756] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:01.615512] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:02.770987] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:04.077633] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:40:04.087933] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fdd17fc047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fdd18293182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fdd18d8d265]))) 0-: received signum (15), shutting down [2016-03-15 14:40:04.100074] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:40:04.104676] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:40:04.105201] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:04.105213] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:04.105548] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:40:04.105559] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:40:04.105565] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:40:04.105600] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:40:04.105651] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:04.105659] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:05.832975] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:40:05.836787] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:05.839790] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:05.842386] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:05.842426] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:05.842475] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:05.842482] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:05.843092] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:05.843135] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:05.843144] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:05.843363] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:05.843400] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:05.843408] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:40:05.882009] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:05.882521] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:05.882577] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:40:05.895770] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:05.909190] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:05.993123] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:05.993188] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:40:06.009622] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:06.009651] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:06.027021] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:40:06.048951] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:06.048984] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:06.048992] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:06.048999] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:40:06.066423] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:06.066496] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:40:06.083457] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:06.083483] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:06.122676] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:06.122704] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:40:06.201464] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:06.201505] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:06.201513] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:06.201520] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:40:06.201578] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:06.201599] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:07.939131] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:09.163544] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:40:09.174762] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f602cf2f47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f602d202182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f602dcfc265]))) 0-: received signum (15), shutting down [2016-03-15 14:40:09.186331] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:40:09.189315] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:40:09.190014] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:09.190033] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:09.190544] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:40:09.190561] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:40:09.190572] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:40:09.190627] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:40:09.190704] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:09.190719] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:10.806336] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:40:10.809408] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:10.813225] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:10.816345] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:10.816384] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:10.816432] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:10.816447] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:10.817119] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:10.817156] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:10.817164] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:10.817426] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:10.817476] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:10.817490] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:40:10.822267] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:10.823169] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:10.823228] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:40:10.843264] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:10.940699] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:40:10.957962] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:10.958024] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:40:10.977787] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:10.977822] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:10.997585] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:10.997617] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:10.997625] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:10.997632] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:40:10.997675] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:10.997700] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:40:11.015529] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:11.455918] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:11.487446] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:11.487510] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:40:11.520957] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:11.520994] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:11.547210] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:11.580210] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:12.878498] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:14.250098] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:40:14.260309] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd0dbfa347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd0dc276182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd0dcd70265]))) 0-: received signum (15), shutting down [2016-03-15 14:40:14.272549] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:40:14.276753] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:40:14.278220] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:14.278239] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:14.278630] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:40:14.278642] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:40:14.278648] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:40:14.278683] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:40:14.278754] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:14.278822] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:16.069528] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:40:16.072625] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:16.075489] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:16.078773] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:16.078813] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:16.078863] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:16.078871] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:16.079409] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:16.079444] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:16.079452] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:16.079683] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:16.079728] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:16.079737] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:40:16.083535] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:16.084718] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:16.084771] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:40:16.138966] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:16.139022] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:40:16.180104] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:16.180149] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:16.363679] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:40:16.391910] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:40:16.409352] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:16.409388] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:16.409397] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:16.409403] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:40:16.409447] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:16.409465] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:16.409473] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:16.409479] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:40:16.410119] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:16.429829] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:16.429879] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:40:16.447266] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:16.447292] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:16.462260] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:16.462784] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:16.902099] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:18.139661] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:19.332338] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:40:19.342816] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f13cf8e047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f13cfbb3182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f13d06ad265]))) 0-: received signum (15), shutting down [2016-03-15 14:40:19.354193] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:40:19.358153] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:40:19.358642] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:19.358654] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:19.359763] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:40:19.359820] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:40:19.359832] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:40:19.359918] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:40:19.359996] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:19.360011] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:21.011364] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:40:21.015523] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:21.019359] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:21.022851] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:21.022895] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:21.022943] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:21.022950] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:21.023475] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:21.023524] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:21.023537] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:21.023759] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:21.023802] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:21.023815] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:40:21.028377] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:21.029336] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:21.029389] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:40:21.046266] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:21.108041] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:21.108101] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:40:21.128409] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:21.128441] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:21.148133] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:40:21.173092] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:21.173124] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:21.173131] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:21.173138] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:40:21.195360] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:21.706148] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:40:22.182932] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:22.197288] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:22.197339] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:40:22.224461] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:22.224490] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:22.243635] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:22.243663] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:23.085483] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:24.415047] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:40:24.425668] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fe342aa047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fe342d73182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fe34386d265]))) 0-: received signum (15), shutting down [2016-03-15 14:40:24.436764] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:40:24.441181] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:40:24.441686] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:24.441698] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:24.442036] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:40:24.442047] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:40:24.442053] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:40:24.442088] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:40:24.442137] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:24.442145] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:26.093472] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:40:26.096539] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:26.099160] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:26.102034] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:26.102073] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:26.102123] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:26.102131] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:26.102708] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:26.102781] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:26.102792] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:26.103040] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:26.103072] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:26.103079] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:40:26.106784] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:26.107843] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:26.107913] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:40:26.125035] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:26.190822] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:26.190885] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:40:26.221537] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:26.221563] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:26.252281] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:40:26.281408] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:26.281441] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:26.281448] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:26.281455] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:40:26.298652] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:26.458208] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:40:26.489909] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:26.489940] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:26.489947] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:26.489954] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:40:26.506668] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:26.518265] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:26.518321] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:40:26.538768] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:26.538789] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:26.621363] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:26.770918] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:28.162998] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:29.500093] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:40:29.511515] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f979f3b447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f979f687182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f97a0181265]))) 0-: received signum (15), shutting down [2016-03-15 14:40:29.522825] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:40:29.526932] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:40:29.527633] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:29.527658] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:29.528168] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:40:29.528183] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:40:29.528194] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:40:29.528242] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:40:29.528318] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:29.528333] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:31.153768] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:40:31.157837] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:31.160853] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:31.164275] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:31.164314] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:31.164363] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:31.164371] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:31.164941] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:31.164983] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:31.164991] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:31.165220] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:31.165257] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:31.165266] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:40:31.168668] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:31.207435] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:40:31.280151] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:40:31.300420] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:31.300450] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:31.300457] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:31.300465] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:40:31.300509] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:40:31.324706] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:31.343558] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:31.343610] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:40:31.363565] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:31.363590] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:31.384491] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:31.384524] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:31.620704] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:32.459939] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:32.506919] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:32.506975] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:40:32.530655] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:32.530683] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:32.565033] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:32.565074] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:33.225877] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:34.582423] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:40:34.593612] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f8b419cd47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f8b41ca0182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f8b4279a265]))) 0-: received signum (15), shutting down [2016-03-15 14:40:34.605860] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:40:34.609667] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:40:34.610176] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:34.610187] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:34.610526] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:40:34.610537] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:40:34.610543] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:40:34.610579] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:40:34.610630] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:34.610638] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:36.222404] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:40:36.225709] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:36.229445] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:36.234136] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:36.234177] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:36.234227] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:36.234234] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:36.234829] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:36.234871] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:36.234880] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:36.235150] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:36.235183] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:36.235190] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:40:36.239790] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:36.241029] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:36.241083] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:40:36.256118] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:36.296653] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:40:36.311073] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:36.311124] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:40:36.331264] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:36.331300] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:36.373255] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:36.373308] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:36.373329] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:36.373336] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:36.373343] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:40:37.185260] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:40:37.225411] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:37.225446] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:37.225453] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:37.225461] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:40:37.245980] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:37.273175] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:37.273222] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:40:37.323160] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:37.323188] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:37.347629] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:37.347657] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:38.295674] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:39.670800] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:40:39.681140] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f6daca6547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f6dacd38182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f6dad832265]))) 0-: received signum (15), shutting down [2016-03-15 14:40:39.692769] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:40:39.694992] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:40:39.695706] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:39.695723] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:39.696187] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:40:39.696199] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:40:39.696205] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:40:39.696242] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:40:39.696292] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:39.696300] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:41.422706] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:40:41.425935] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:41.429134] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:41.432964] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:41.433005] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:41.433053] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:41.433060] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:41.433611] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:41.433646] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:41.433653] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:41.433887] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:41.433927] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:41.433936] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:40:41.438163] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:41.439323] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:41.439379] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:40:41.460776] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:41.460841] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:40:41.479790] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:41.479809] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:41.530183] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:41.600428] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:40:41.645631] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:41.645676] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:41.645684] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:41.645691] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:40:41.645752] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:40:41.712027] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:41.712059] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:41.712075] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:41.712083] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:40:41.712361] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:41.729711] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:41.729752] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:40:41.750424] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:41.750460] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:41.771704] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:41.771732] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:43.494905] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:44.758421] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:40:44.772002] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f78f537047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f78f5643182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f78f613d265]))) 0-: received signum (15), shutting down [2016-03-15 14:40:44.783894] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:40:44.786056] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:40:44.786560] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:44.786571] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:44.787351] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:40:44.787365] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:40:44.787372] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:40:44.787408] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:40:44.787460] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:44.787468] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:46.443954] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:40:46.448414] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:46.451740] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:46.455098] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:46.455141] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:46.455194] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:46.455201] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:46.455773] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:46.455820] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:46.455828] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:46.456090] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:46.456123] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:46.456130] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:40:46.460046] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:46.461263] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:46.461320] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:40:46.688136] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:46.688221] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:40:46.804966] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:46.805002] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:46.854011] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:40:46.884668] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:46.884705] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:46.884713] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:46.884720] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:40:46.884767] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:40:46.918386] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:46.918431] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:46.919165] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:47.462888] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:47.491658] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:47.491712] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:40:47.510598] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:47.510616] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:47.530436] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:47.552484] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:48.516715] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:49.848780] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:40:49.859964] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f44f106147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f44f1334182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f44f1e2e265]))) 0-: received signum (15), shutting down [2016-03-15 14:40:49.872156] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:40:49.875280] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:40:49.875982] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:49.875999] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:49.876511] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:40:49.876529] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:40:49.876539] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:40:49.876590] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:40:49.876672] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:49.876686] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:51.531965] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:40:51.535042] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:51.538544] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:51.541961] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:51.542007] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:51.542124] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:51.542135] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:51.542648] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:51.542685] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:51.542693] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:51.542952] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:51.542986] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:51.542994] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:40:51.547198] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:51.548210] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:51.548265] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:40:51.565034] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:51.565085] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:40:51.585006] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:51.585026] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:51.634819] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:40:51.734931] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:51.734970] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:51.734977] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:51.734984] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:40:52.188158] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:40:52.230135] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:52.230171] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:52.230178] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:52.230185] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:40:52.273055] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:52.286997] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:52.287044] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:40:52.305320] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:52.305347] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:52.323007] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:52.324164] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:52.624368] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:53.603622] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:54.938981] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:40:54.949676] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f9c9d67247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f9c9d945182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f9c9e43f265]))) 0-: received signum (15), shutting down [2016-03-15 14:40:54.961191] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:40:54.964620] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:40:54.965128] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:54.965139] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:54.965479] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:40:54.965490] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:40:54.965496] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:40:54.965531] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:40:54.965581] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:54.965589] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:56.564368] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:40:56.568400] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:56.571233] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:56.582821] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:56.582865] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:56.582916] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:56.582924] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:56.583567] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:56.583607] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:56.583615] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:56.583827] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:56.583864] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:56.583873] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:40:56.588836] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:56.589056] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:56.589136] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:40:56.602563] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:56.618942] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:56.618993] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:40:56.659570] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:56.659597] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:56.712113] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:56.747342] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:40:56.765320] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:56.765349] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:56.765357] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:56.765364] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:40:56.781795] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:56.781856] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:40:56.800887] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:56.800912] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:56.835969] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:40:56.855451] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:56.855500] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:56.855519] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:56.855526] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:56.855533] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:40:56.855575] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:58.645143] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:00.026080] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:41:00.036742] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd78efdf47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd78f2b2182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd78fdac265]))) 0-: received signum (15), shutting down [2016-03-15 14:41:00.048328] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:41:00.052552] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:41:00.053095] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:00.053107] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:00.053451] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:41:00.053463] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:41:00.053469] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:41:00.053504] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:41:00.053565] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:00.053579] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:01.719656] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:41:01.725631] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:01.729885] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:01.733418] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:01.733462] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:01.733513] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:01.733521] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:01.734102] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:01.734145] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:01.734155] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:01.734406] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:01.734439] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:01.734446] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:41:01.738992] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:01.739695] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:01.739747] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:41:01.757513] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:01.825522] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:01.825625] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:41:01.846431] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:01.846458] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:01.946794] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:01.946843] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:01.946851] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:01.946858] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:41:01.946906] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:41:01.969285] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:41:01.989341] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:02.465987] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:02.481118] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:02.481167] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:41:02.502346] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:02.502369] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:02.523093] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:02.524815] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:03.796705] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:05.110709] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:41:05.121072] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd0e65a447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd0e6877182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd0e7371265]))) 0-: received signum (15), shutting down [2016-03-15 14:41:05.133237] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:41:05.135507] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:41:05.136009] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:05.136021] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:05.136363] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:41:05.136374] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:41:05.136379] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:41:05.136421] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:41:05.136472] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:05.136480] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:06.906841] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:41:06.913137] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:06.916326] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:06.919060] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:06.919101] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:06.919150] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:06.919157] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:06.919700] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:06.919735] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:06.919742] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:06.919966] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:06.920004] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:06.920017] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:41:06.923904] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:06.925160] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:06.925213] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:41:07.013598] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:41:07.061229] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:07.061292] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:41:07.090770] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:07.090799] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:07.146890] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:07.146919] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:41:07.180761] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:07.180802] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:07.180810] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:07.180817] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:41:07.180868] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:07.199446] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:07.215565] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:07.215607] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:41:07.233398] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:07.233432] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:07.253487] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:07.270199] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:07.628297] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:08.979955] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:10.196382] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:41:10.207708] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f0b5194347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f0b51c16182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f0b52710265]))) 0-: received signum (15), shutting down [2016-03-15 14:41:10.218951] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:41:10.222658] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:41:10.223757] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:10.223777] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:10.224285] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:41:10.224302] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:41:10.224313] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:41:10.224379] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:41:10.224456] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:10.224470] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:12.069866] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:41:12.073640] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:12.083418] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:12.090292] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:12.090343] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:12.090394] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:12.090401] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:12.090992] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:12.091034] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:12.091043] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:12.091265] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:12.091310] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:12.091321] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:41:12.095319] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:12.096551] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:12.096604] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:41:12.119575] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:12.223799] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:41:12.243627] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:12.243684] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:41:12.263271] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:12.263296] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:12.286762] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:12.286806] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:12.286827] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:12.286835] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:12.286842] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:41:12.286883] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:41:12.327974] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:13.192747] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:13.210349] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:13.210400] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:41:13.267162] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:13.267191] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:13.291095] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:13.320213] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:14.153028] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:15.281332] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:41:15.291482] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f55f130247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f55f15d5182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f55f20cf265]))) 0-: received signum (15), shutting down [2016-03-15 14:41:15.303598] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:41:15.305813] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:41:15.306328] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:15.306340] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:15.306682] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:41:15.306692] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:41:15.306698] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:41:15.306755] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:41:15.306814] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:15.306822] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:16.967518] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:41:16.974338] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:16.978829] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:16.982506] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:16.982556] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:16.982606] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:16.982613] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:16.983279] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:16.983321] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:16.983331] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:16.983612] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:16.983662] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:16.983671] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:41:16.987459] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:16.988738] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:16.988792] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:41:17.013007] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:17.164834] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:17.164903] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:41:17.182585] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:17.182604] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:17.201431] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:41:17.220074] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:17.220108] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:17.220116] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:17.220124] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:41:17.220169] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:41:17.250450] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:17.250477] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:17.468728] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:17.489009] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:17.489057] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:41:17.534916] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:17.534945] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:17.553511] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:17.553545] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:19.043835] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:20.368130] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:41:20.378246] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f91b41e147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f91b44b4182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f91b4fae265]))) 0-: received signum (15), shutting down [2016-03-15 14:41:20.390784] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:41:20.393105] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:41:20.393629] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:20.393641] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:20.393983] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:41:20.393995] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:41:20.394000] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:41:20.394036] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:41:20.394087] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:20.394096] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:21.963838] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:41:21.967638] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:21.972580] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:21.976056] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:21.976097] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:21.976145] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:21.976152] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:21.976654] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:21.976700] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:21.976711] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:21.976986] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:21.977030] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:21.977043] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:41:21.984069] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:21.985170] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:21.985245] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:41:22.047522] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:41:22.074947] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:22.075011] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:41:22.095345] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:22.095370] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:22.111633] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:22.111660] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:22.111668] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:22.111675] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:41:22.134760] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:22.193920] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:41:22.216024] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:22.216052] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:22.216059] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:22.216066] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:41:22.240571] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:22.271497] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:22.271548] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:41:22.290472] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:22.290490] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:22.305920] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:22.326569] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:22.631904] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:24.040735] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:25.458670] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:41:25.469126] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f033c22747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f033c4fa182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f033cff4265]))) 0-: received signum (15), shutting down [2016-03-15 14:41:25.480254] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:41:25.484118] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:41:25.484698] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:25.484711] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:25.485049] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:41:25.485060] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:41:25.485066] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:41:25.485101] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:41:25.485151] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:25.485158] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:27.054192] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:41:27.060415] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:27.065065] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:27.069263] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:27.069309] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:27.069358] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:27.069366] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:27.069904] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:27.069941] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:27.069949] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:27.070170] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:27.070214] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:27.070227] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:41:27.074930] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:27.075822] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:27.075875] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:41:27.089608] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:27.119431] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:41:27.230532] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:27.230605] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:41:27.257894] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:27.257928] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:27.302389] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:27.302437] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:27.302457] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:27.302464] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:27.302471] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:41:27.544908] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:41:28.195630] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:28.211428] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:28.211476] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:41:28.228666] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:28.228701] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:28.251627] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:28.292970] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:29.132300] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:30.541855] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:41:30.553212] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f0e8bce047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f0e8bfb3182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f0e8caad265]))) 0-: received signum (15), shutting down [2016-03-15 14:41:30.565096] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:41:30.568621] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:41:30.569125] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:30.569137] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:30.569489] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:41:30.569500] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:41:30.569506] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:41:30.569542] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:41:30.569593] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:30.569601] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:32.267359] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:41:32.272721] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:32.277797] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:32.280915] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:32.280955] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:32.281006] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:32.281014] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:32.281591] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:32.281629] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:32.281637] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:32.281881] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:32.281918] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:32.281926] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:41:32.286103] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:32.287439] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:32.287512] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:41:32.299763] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:32.632208] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:32.632270] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:41:32.657706] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:32.657740] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:32.704116] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:41:32.751939] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:41:32.781479] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:32.781511] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:32.781519] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:32.781526] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:41:32.781606] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:32.781626] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:32.781648] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:32.781660] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:32.781666] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:32.781673] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:41:32.782372] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:32.809147] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:32.809194] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:41:32.830000] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:32.830039] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:32.875445] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:32.975575] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:34.343365] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:35.635007] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:41:35.645296] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7eff1ee3d47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7eff1f110182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7eff1fc0a265]))) 0-: received signum (15), shutting down [2016-03-15 14:41:35.656790] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:41:35.660832] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:41:35.661367] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:35.661379] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:35.661713] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:41:35.661724] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:41:35.661736] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:41:35.661771] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:41:35.661824] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:35.661832] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:37.345270] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:41:37.347796] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:37.350899] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:37.353537] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:37.353580] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:37.353632] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:37.353640] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:37.354228] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:37.354264] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:37.354271] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:37.354535] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:37.354571] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:37.354579] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:41:37.358067] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:37.359038] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:37.359092] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:41:37.414179] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:41:37.431111] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:37.431169] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:41:37.457019] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:37.457047] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:37.480614] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:37.480654] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:37.480683] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:37.480690] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:37.480696] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:41:37.632295] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:37.962418] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:41:38.472941] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:38.490318] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:38.490367] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:41:38.512860] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:38.512887] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:38.535789] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:38.548421] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:39.414379] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:40.719173] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:41:40.729479] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd5912d847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd5915ab182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd5920a5265]))) 0-: received signum (15), shutting down [2016-03-15 14:41:40.741857] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:41:40.744123] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:41:40.744624] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:40.744635] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:40.744971] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:41:40.744982] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:41:40.744988] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:41:40.745023] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:41:40.745073] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:40.745080] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:43.728836] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:41:43.738939] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:43.754641] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:43.768555] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:43.768610] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:43.768662] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:43.768670] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:43.769243] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:43.769280] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:43.769287] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:43.769552] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:43.769587] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:43.769594] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:41:43.781811] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:43.782847] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:43.782921] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:41:43.869572] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:44.381123] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:44.400176] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:44.400239] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:41:44.418486] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:44.418511] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:44.435919] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:41:44.470302] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:41:44.486472] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:44.486524] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:41:44.504316] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:44.504347] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:44.529389] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:44.529423] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:44.529438] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:44.529445] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:41:44.529494] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:44.529515] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:44.529536] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:44.529547] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:44.529552] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:44.529558] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:41:44.529591] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:44.529610] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:45.818768] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:41:45.839190] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f37ccc4447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f37ccf17182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f37cda11265]))) 0-: received signum (15), shutting down [2016-03-15 14:41:45.856321] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:41:45.859409] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:41:45.860131] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:45.860149] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:45.860655] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:41:45.860672] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:41:45.860683] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:41:45.860735] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:41:45.860818] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:45.860833] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:47.520240] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:41:47.527256] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:47.530806] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:47.553496] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:47.553544] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:47.553597] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:47.553604] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:47.554124] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:47.554166] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:47.554175] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:47.554398] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:47.554439] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:47.554452] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:41:47.562893] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:47.562950] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:41:47.576963] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:47.589896] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:47.624648] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:47.624711] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:41:47.647866] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:47.647895] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:47.720255] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:47.720321] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:41:47.754895] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:47.754925] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:47.792981] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:47.793014] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:47.793022] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:47.793029] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:41:47.793084] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:41:47.819524] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:47.819552] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:47.819566] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:47.819573] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:41:47.819616] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:41:47.840037] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:47.840066] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:47.840080] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:48.792564] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:50.848989] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:50.931628] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:41:50.942804] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f1d5523a47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f1d5550d182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f1d56007265]))) 0-: received signum (15), shutting down [2016-03-15 14:41:50.954111] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:41:50.956958] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:41:50.957668] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:50.957685] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:50.958183] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:41:50.958200] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:41:50.958209] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:41:50.958260] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:41:50.958335] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:50.958349] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:52.566292] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:41:52.569516] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:52.572610] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:52.575592] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:52.575631] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:52.575679] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:52.575687] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:52.576243] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:52.576280] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:52.576288] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:52.576527] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:52.576569] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:52.576578] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:41:52.582104] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:52.582151] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:41:52.690098] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:52.690177] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:41:52.712597] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:52.712627] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:52.731615] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:41:52.764183] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:52.764214] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:52.764222] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:52.764228] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:41:52.764276] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:41:52.816048] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:52.816081] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:52.816660] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:52.905667] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:53.476000] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:53.509123] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:53.509178] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:41:53.548125] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:53.548157] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:53.626342] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:53.626370] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:54.961653] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:56.014901] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:41:56.025252] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f69c495647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f69c4c29182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f69c5723265]))) 0-: received signum (15), shutting down [2016-03-15 14:41:56.036813] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:41:56.040897] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:41:56.041425] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:56.041437] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:56.041827] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:41:56.041841] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:41:56.041846] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:41:56.041883] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:41:56.041934] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:56.041942] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:59.252513] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:41:59.255960] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:59.258847] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:59.265142] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:59.265183] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:59.265234] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:59.265241] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:59.265734] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:59.265773] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:59.265781] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:59.265995] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:59.266030] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:59.266037] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:41:59.270324] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:59.271065] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:59.271117] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:41:59.296245] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:59.336045] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:59.336101] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:41:59.355621] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:59.355651] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:59.388550] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:41:59.407195] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:59.407267] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:41:59.434116] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:59.434151] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:59.485280] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:59.485317] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:59.485324] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:59.485331] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:41:59.485379] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:59.485403] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:59.485418] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:59.485424] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:59.485430] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:41:59.485464] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:41:59.555356] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:59.581069] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:01.098022] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:42:01.108145] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f636eb9347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f636ee66182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f636f960265]))) 0-: received signum (15), shutting down [2016-03-15 14:42:01.120287] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:42:01.122519] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:42:01.123056] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:01.123070] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:01.123444] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:42:01.123455] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:42:01.123461] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:42:01.123497] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:42:01.123549] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:01.123556] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:03.059768] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:42:03.063108] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:03.206418] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:03.213096] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:03.213146] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:03.213199] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:03.213207] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:03.213781] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:03.213818] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:03.213825] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:03.214044] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:03.214081] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:03.214092] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:42:03.225222] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:03.225893] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:03.225948] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:42:03.379176] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:03.410224] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:03.519733] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:42:03.571513] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:03.571549] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:03.571557] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:03.571564] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:42:03.571611] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:42:03.617282] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:03.617362] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:42:03.684938] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:03.684966] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:03.724382] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:03.724429] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:03.724475] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:42:03.741480] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:03.741508] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:03.763455] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:03.763502] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:03.763521] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:03.763528] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:03.763534] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:42:03.763584] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:03.763598] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:05.282557] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:06.186625] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:42:06.197428] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f36fadd847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f36fb0ab182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f36fbba5265]))) 0-: received signum (15), shutting down [2016-03-15 14:42:06.209451] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:42:06.213072] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:42:06.213613] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:06.213625] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:06.213978] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:42:06.213990] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:42:06.213996] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:42:06.214031] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:42:06.214082] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:06.214090] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:07.917917] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:42:07.921551] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:07.925207] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:07.932563] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:07.932602] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:07.932650] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:07.932658] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:07.933284] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:07.933323] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:07.933330] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:07.933558] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:07.933593] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:07.933601] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:42:07.954267] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:07.955457] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:07.955512] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:42:07.972206] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:08.064152] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:08.064250] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:42:08.122162] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:08.122196] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:08.140465] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:42:08.165199] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:08.165233] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:08.165241] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:08.165248] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:42:08.165290] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:42:08.188292] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:08.188321] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:08.478777] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:08.511754] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:08.511809] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:42:08.562647] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:08.562670] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:08.598971] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:08.599001] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:10.011316] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:11.277509] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:42:11.288905] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa7e4be747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa7e4eba182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa7e59b4265]))) 0-: received signum (15), shutting down [2016-03-15 14:42:11.300962] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:42:11.303164] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:42:11.303673] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:11.303688] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:11.304042] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:42:11.304053] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:42:11.304064] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:42:11.304101] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:42:11.304152] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:11.304160] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:12.885867] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:42:12.888854] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:12.895731] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:12.898946] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:12.898985] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:12.899036] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:12.899044] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:12.899542] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:12.899579] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:12.899587] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:12.899800] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:12.899837] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:12.899846] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:42:12.903828] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:12.905142] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:12.905195] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:42:12.985583] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:12.985650] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:42:13.010216] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:13.010251] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:13.106506] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:42:13.131300] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:13.131335] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:13.131349] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:13.131357] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:42:13.131416] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:42:13.155083] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:13.204608] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:13.237614] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:13.237663] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:42:13.266611] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:13.266632] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:13.284897] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:13.305598] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:13.638867] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:14.960059] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:16.367013] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:42:16.377276] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7eff68f3b47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7eff6920e182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7eff69d08265]))) 0-: received signum (15), shutting down [2016-03-15 14:42:16.388741] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:42:16.392825] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:42:16.393455] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:16.393469] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:16.393815] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:42:16.393826] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:42:16.393832] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:42:16.393867] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:42:16.393918] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:16.393926] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:17.997417] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:42:18.000375] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:18.003235] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:18.006047] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:18.006097] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:18.006148] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:18.006156] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:18.006754] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:18.006797] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:18.006805] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:18.007059] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:18.007097] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:18.007105] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:42:18.010728] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:18.011352] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:18.011437] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:42:18.027241] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:18.086225] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:18.086291] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:42:18.113823] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:18.113857] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:18.176571] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:42:18.233357] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:18.233399] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:18.233423] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:18.233429] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:18.233436] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:42:18.233486] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:42:19.205734] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:19.219628] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:19.219686] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:42:19.240659] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:19.240688] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:19.260521] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:19.354019] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:20.068170] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:21.451252] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:42:21.461683] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa4eabd647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa4eaea9182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa4eb9a3265]))) 0-: received signum (15), shutting down [2016-03-15 14:42:21.473543] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:42:21.477667] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:42:21.478387] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:21.478406] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:21.478972] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:42:21.478991] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:42:21.479001] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:42:21.479054] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:42:21.479132] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:21.479147] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:23.146822] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:42:23.149872] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:23.152504] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:23.155422] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:23.155464] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:23.155512] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:23.155520] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:23.156094] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:23.156131] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:23.156139] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:23.156363] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:23.156401] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:23.156412] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:42:23.162596] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:23.163273] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:23.163340] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:42:23.210716] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:23.311193] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:42:23.329453] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:23.329511] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:42:23.346502] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:23.346524] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:23.377628] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:42:23.396747] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:23.396790] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:23.396832] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:23.396843] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:23.396850] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:42:23.396893] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:23.481614] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:23.531794] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:23.531849] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:42:23.550638] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:23.550657] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:23.574455] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:23.574654] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:25.218620] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:26.537754] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:42:26.548097] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f6fa5b1047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f6fa5de3182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f6fa68dd265]))) 0-: received signum (15), shutting down [2016-03-15 14:42:26.560039] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:42:26.562236] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:42:26.562761] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:26.562777] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:26.563142] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:42:26.563154] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:42:26.563160] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:42:26.563195] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:42:26.563246] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:26.563254] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:28.120652] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:42:28.124103] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:28.126923] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:28.129743] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:28.129781] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:28.129832] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:28.129840] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:28.130338] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:28.130376] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:28.130383] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:28.130605] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:28.130641] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:28.130649] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:42:28.136095] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:28.211565] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:42:28.298331] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:42:28.349651] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:28.349685] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:28.349693] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:28.349700] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:42:28.349745] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:42:28.366533] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:28.379207] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:28.379251] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:42:28.398756] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:28.398775] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:28.415829] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:28.426086] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:28.641474] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:29.482963] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:29.497009] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:29.497057] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:42:29.531828] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:29.531856] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:29.548852] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:29.548879] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:30.192859] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:31.625358] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:42:31.636515] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f07e24c147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f07e2794182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f07e328e265]))) 0-: received signum (15), shutting down [2016-03-15 14:42:31.648025] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:42:31.651312] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:42:31.652360] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:31.652377] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:31.652773] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:42:31.652785] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:42:31.652791] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:42:31.652827] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:42:31.652878] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:31.652886] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:33.241906] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:42:33.245298] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:33.249659] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:33.253135] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:33.253177] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:33.253226] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:33.253234] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:33.253814] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:33.253857] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:33.253866] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:33.254084] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:33.254116] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:33.254123] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:42:33.259793] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:33.261569] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:33.261623] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:42:33.277240] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:33.312575] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:33.312638] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:42:33.345235] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:33.345271] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:33.404283] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:42:33.432127] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:42:33.498957] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:33.499002] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:33.499010] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:33.499017] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:42:33.499074] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:34.208507] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:34.225885] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:34.225942] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:42:34.253643] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:34.253667] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:34.281648] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:34.349987] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:35.317401] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:36.713354] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:42:36.723856] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f621f26347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f621f536182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f6220030265]))) 0-: received signum (15), shutting down [2016-03-15 14:42:36.737040] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:42:36.741321] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:42:36.741850] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:36.741862] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:36.742194] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:42:36.742205] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:42:36.742210] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:42:36.742245] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:42:36.742298] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:36.742306] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:38.302962] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:42:38.306908] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:38.310811] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:38.313524] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:38.313564] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:38.313615] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:38.313623] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:38.314111] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:38.314148] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:38.314156] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:38.314387] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:38.314424] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:38.314432] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:42:38.318963] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:38.319639] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:38.319692] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:42:38.332387] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:38.402698] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:38.402834] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:42:38.445743] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:38.445777] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:38.490008] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:42:38.509285] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:38.509316] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:38.509324] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:38.509331] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:42:38.509381] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:42:38.528416] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:38.904321] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:38.905096] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:38.920916] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:38.920974] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:42:38.944713] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:38.944748] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:38.969109] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:39.244920] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:40.375262] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:41.798898] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:42:41.809330] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f13c280a47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f13c2add182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f13c35d7265]))) 0-: received signum (15), shutting down [2016-03-15 14:42:41.821479] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:42:41.824568] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:42:41.825261] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:41.825279] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:41.825799] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:42:41.825817] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:42:41.825827] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:42:41.825879] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:42:41.825957] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:41.825971] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:43.426764] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:42:43.430046] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:43.449593] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:43.452813] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:43.452854] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:43.452916] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:43.452927] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:43.453475] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:43.453510] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:43.453517] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:43.453748] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:43.453791] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:43.453801] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:42:43.459526] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:43.460454] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:43.460511] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:42:43.527440] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:43.527514] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:42:43.545980] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:43.546013] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:43.583603] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:43.583636] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:42:43.604117] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:42:43.622808] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:43.622838] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:43.622845] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:43.622852] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:42:43.633371] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:43.643918] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:44.485663] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:44.526458] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:44.526517] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:42:44.551642] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:44.551668] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:44.573544] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:44.573570] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:45.515579] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:46.889505] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:42:46.902948] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f0ca2fbc47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f0ca328f182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f0ca3d89265]))) 0-: received signum (15), shutting down [2016-03-15 14:42:46.914459] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:42:46.916946] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:42:46.917456] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:46.917468] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:46.917809] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:42:46.917820] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:42:46.917826] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:42:46.917862] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:42:46.917913] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:46.917921] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:48.607575] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:42:48.612497] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:48.618716] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:48.622964] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:48.623008] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:48.623060] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:48.623068] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:48.623687] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:48.623723] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:48.623731] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:48.623988] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:48.624025] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:48.624032] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:42:48.628421] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:48.629119] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:48.629174] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:42:48.677172] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:48.677260] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:42:48.700735] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:48.700768] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:48.739481] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:48.739515] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:48.739522] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:48.739529] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:42:48.754436] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:42:48.781217] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:49.210970] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:42:49.276563] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:49.276608] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:49.276616] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:49.276623] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:42:49.313495] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:49.326368] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:49.326412] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:42:49.346287] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:49.346312] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:49.365143] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:49.371907] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:49.644950] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:50.685292] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:51.977350] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:42:51.988852] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa92a9c447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa92ac97182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa92b791265]))) 0-: received signum (15), shutting down [2016-03-15 14:42:52.000172] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:42:52.004172] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:42:52.004837] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:52.004850] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:52.005191] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:42:52.005202] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:42:52.005208] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:42:52.005242] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:42:52.005292] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:52.005300] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:53.622136] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:42:53.625150] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:53.629397] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:53.632240] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:53.632279] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:53.632324] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:53.632331] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:53.632928] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:53.632964] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:53.632972] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:53.633243] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:53.633275] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:53.633283] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:42:53.637199] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:53.637814] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:53.637865] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:42:53.650571] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:53.690707] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:53.690774] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:42:53.734939] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:53.734970] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:53.767310] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:53.796448] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:53.796480] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:53.796488] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:53.796495] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:42:53.796535] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:42:53.854008] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:53.854068] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:42:53.878324] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:53.878351] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:53.895756] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:53.895784] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:42:53.914629] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:53.914659] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:53.914666] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:53.914673] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:42:53.914720] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:53.914771] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:55.700201] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:57.059945] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:42:57.070460] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f219990847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f2199bdb182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f219a6d5265]))) 0-: received signum (15), shutting down [2016-03-15 14:42:57.082196] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:42:57.085515] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:42:57.086568] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:57.086587] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:57.087126] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:42:57.087144] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:42:57.087154] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:42:57.087208] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:42:57.087291] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:57.087306] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:59.089632] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:42:59.093932] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:59.097724] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:59.108265] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:59.108309] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:59.108360] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:59.108368] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:59.108899] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:59.108934] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:59.108942] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:59.109167] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:59.109205] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:59.109216] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:42:59.113771] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:59.114796] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:59.114865] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:42:59.132416] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:59.221232] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:59.221298] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:42:59.242038] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:59.242072] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:59.280775] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:59.280805] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:42:59.320635] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:42:59.344363] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:59.344403] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:59.344423] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:59.344430] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:59.344437] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:42:59.488865] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:59.506134] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:59.506184] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:42:59.553303] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:59.553326] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:59.569285] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:59.570189] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:01.170560] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:02.149091] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:43:02.159679] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa71533147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa715604182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa7160fe265]))) 0-: received signum (15), shutting down [2016-03-15 14:43:02.171606] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:43:02.173848] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:43:02.174366] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:02.174377] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:02.174718] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:43:02.174729] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:43:02.174758] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:43:02.174800] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:43:02.174850] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:02.174858] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:03.985413] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:43:03.989081] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:03.993939] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:03.997592] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:03.997630] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:03.997680] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:03.997687] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:03.998255] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:03.998292] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:03.998300] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:03.998521] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:03.998597] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:03.998607] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:43:04.005773] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:04.006772] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:04.006832] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:43:04.065744] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:43:04.183353] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:04.183427] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:43:05.216400] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:05.216445] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:05.239882] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:05.239931] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:05.239956] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:05.239963] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:05.239970] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:43:05.301892] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:43:05.328930] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:05.328966] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:05.328974] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:05.328981] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:43:05.329626] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:05.390184] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:05.406512] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:05.406566] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:43:05.552435] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:05.552470] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:05.570361] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:05.570385] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:06.062574] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:07.236873] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:43:07.247602] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f78db0b547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f78db388182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f78dbe82265]))) 0-: received signum (15), shutting down [2016-03-15 14:43:07.259164] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:43:07.263043] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:43:07.263673] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:07.263686] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:07.264040] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:43:07.264058] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:43:07.264064] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:43:07.264099] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:43:07.264150] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:07.264158] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:08.907432] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:43:08.910889] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:08.914149] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:08.917392] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:08.917433] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:08.917483] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:08.917490] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:08.918023] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:08.918058] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:08.918066] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:08.918293] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:08.918330] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:08.918339] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:43:08.922754] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:08.923487] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:08.923541] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:43:08.936866] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:08.985204] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:08.985283] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:43:09.004550] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:09.004568] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:09.140872] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:09.140909] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:09.140923] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:09.140931] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:43:09.140976] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:43:09.160861] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:43:09.190053] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:10.215641] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:11.484812] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:11.484932] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:11.484988] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:43:11.511258] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:11.511280] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:11.534127] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:11.542701] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:12.448625] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:43:12.459928] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd192d9847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd19306b182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd193b65265]))) 0-: received signum (15), shutting down [2016-03-15 14:43:12.471217] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:43:12.474131] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:43:12.475723] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:12.475742] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:12.476130] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:43:12.476143] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:43:12.476149] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:43:12.476185] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:43:12.476237] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:12.476245] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:14.072167] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:43:14.077536] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:14.083900] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:14.087146] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:14.087187] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:14.087239] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:14.087246] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:14.087813] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:14.087848] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:14.087856] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:14.088088] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:14.088134] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:14.088144] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:43:14.092004] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:14.092819] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:14.092872] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:43:14.104854] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:14.141354] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:14.141413] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:43:14.159702] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:14.159735] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:14.240756] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:43:14.264526] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:43:14.280859] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:14.280896] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:14.280903] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:14.280910] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:43:14.280963] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:14.491673] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:14.505571] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:14.505614] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:43:14.522676] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:14.522700] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:14.591297] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:14.591330] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:16.149048] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:17.536432] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:43:17.547343] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f4507dd347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f45080a6182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f4508ba0265]))) 0-: received signum (15), shutting down [2016-03-15 14:43:17.559337] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:43:17.563299] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:43:17.563986] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:17.564000] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:17.564378] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:43:17.564389] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:43:17.564395] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:43:17.564432] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:43:17.564484] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:17.564492] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:19.221740] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:43:19.224992] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:19.229393] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:19.233345] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:19.233387] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:19.233436] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:19.233444] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:19.234021] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:19.234058] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:19.234066] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:19.234320] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:19.234352] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:19.234359] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:43:19.238722] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:19.239609] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:19.239663] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:43:19.303979] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:19.304041] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:43:19.325652] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:19.325686] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:19.342779] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:19.342805] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:43:19.379714] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:43:19.417669] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:19.417703] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:19.417711] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:19.417718] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:43:19.417763] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:19.650272] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:20.492746] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:20.547899] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:20.547956] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:43:20.618254] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:20.618291] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:20.639943] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:20.639987] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:21.296568] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:22.622285] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:43:22.632545] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fb17cfad47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fb17d280182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fb17dd7a265]))) 0-: received signum (15), shutting down [2016-03-15 14:43:22.643789] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:43:22.647504] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:43:22.648008] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:22.648020] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:22.648353] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:43:22.648364] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:43:22.648370] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:43:22.648405] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:43:22.648457] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:22.648465] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:24.322822] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:43:24.326212] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:24.330194] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:24.335273] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:24.335313] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:24.335363] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:24.335370] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:24.335907] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:24.335942] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:24.335950] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:24.336170] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:24.336218] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:24.336232] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:43:24.343567] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:24.344238] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:24.344293] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:43:24.361602] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:24.407102] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:24.407168] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:43:24.433766] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:24.433801] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:24.487417] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:24.487453] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:24.487461] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:24.487468] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:43:24.487514] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:43:24.528070] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:43:24.552017] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:25.218958] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:25.261152] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:25.261213] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:43:25.279823] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:25.279848] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:25.303469] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:25.334022] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:26.400714] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:27.707642] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:43:27.718100] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fafa152547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fafa17f8182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fafa22f2265]))) 0-: received signum (15), shutting down [2016-03-15 14:43:27.730290] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:43:27.734186] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:43:27.734704] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:27.734716] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:27.735649] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:43:27.735667] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:43:27.735678] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:43:27.735736] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:43:27.735812] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:27.735826] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:29.358894] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:43:29.362487] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:29.365878] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:29.369514] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:29.369555] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:29.369606] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:29.369614] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:29.370189] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:29.370225] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:29.370232] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:29.370460] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:29.370504] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:29.370517] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:43:29.376558] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:29.377294] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:29.377347] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:43:29.393123] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:29.508123] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:29.508188] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:43:29.562555] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:29.562602] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:29.628934] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:43:29.655493] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:43:29.673807] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:29.673838] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:29.673846] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:29.673853] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:43:29.703020] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:29.703046] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:29.703103] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:29.703127] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:29.703135] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:29.703142] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:43:29.703892] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:29.722252] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:29.722298] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:43:29.749503] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:29.749536] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:29.772622] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:29.808421] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:31.434544] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:32.792800] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:43:32.804274] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f579ea2f47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f579ed02182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f579f7fc265]))) 0-: received signum (15), shutting down [2016-03-15 14:43:32.816263] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:43:32.820259] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:43:32.821140] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:32.821297] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:32.821854] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:43:32.821871] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:43:32.821882] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:43:32.821933] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:43:32.822058] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:32.822073] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:34.518958] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:43:34.522197] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:34.525825] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:34.532058] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:34.532103] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:34.532155] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:34.532163] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:34.532684] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:34.532742] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:34.532755] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:34.533000] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:34.533044] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:34.533057] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:43:34.536858] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:34.537896] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:34.537950] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:43:34.600174] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:34.600237] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:43:34.616804] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:34.616823] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:34.651995] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:34.652022] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:43:34.667684] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:43:34.684201] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:34.684230] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:34.684238] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:34.684245] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:43:34.684287] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:35.125689] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:35.495603] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:35.508401] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:35.508452] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:43:35.526942] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:35.526969] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:35.556044] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:35.594308] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:36.593053] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:37.876873] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:43:37.887492] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f53cadcc47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f53cb09f182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f53cbb99265]))) 0-: received signum (15), shutting down [2016-03-15 14:43:37.898865] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:43:37.902945] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:43:37.903559] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:37.903573] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:37.903916] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:43:37.903928] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:43:37.903934] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:43:37.903969] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:43:37.904020] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:37.904028] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:39.606709] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:43:39.610473] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:39.613437] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:39.616287] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:39.616328] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:39.616379] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:39.616386] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:39.616975] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:39.617017] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:39.617026] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:39.617294] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:39.617326] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:39.617333] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:43:39.621098] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:39.622107] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:39.622159] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:43:39.658374] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:43:39.677971] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:39.678040] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:43:39.752162] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:39.752191] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:40.049155] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:40.049201] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:40.049227] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:40.049235] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:40.049242] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:43:40.221067] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:43:40.262578] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:40.262614] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:40.262622] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:40.262629] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:43:40.281247] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:40.297916] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:40.297961] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:43:40.319597] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:40.319619] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:40.336860] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:40.338636] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:40.653703] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:41.677689] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:42.962724] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:43:42.973389] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f92255ad47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f9225880182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f922637a265]))) 0-: received signum (15), shutting down [2016-03-15 14:43:42.984882] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:43:42.988163] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:43:42.988675] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:42.988687] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:42.989025] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:43:42.989036] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:43:42.989042] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:43:42.989077] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:43:42.989127] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:42.989135] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:44.810360] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:43:44.814192] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:44.817152] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:44.820649] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:44.820698] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:44.820748] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:44.820755] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:44.821348] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:44.821386] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:44.821393] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:44.821616] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:44.821653] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:44.821662] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:43:44.825472] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:44.826717] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:44.826828] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:43:44.841312] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:44.858336] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:44.889119] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:44.889188] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:43:44.924850] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:44.924884] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:44.942959] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:43:44.961077] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:44.961112] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:44.961123] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:44.961130] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:43:44.981234] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:44.981346] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:43:44.998074] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:44.998101] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:45.112693] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:45.112734] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:43:45.128344] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:45.128370] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:45.128377] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:45.128384] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:43:45.128443] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:45.128465] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:46.882112] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:48.047832] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:43:48.059862] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd253d6b47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd25403e182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd254b38265]))) 0-: received signum (15), shutting down [2016-03-15 14:43:48.072054] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:43:48.074333] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:43:48.074923] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:48.074937] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:48.075278] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:43:48.075289] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:43:48.075294] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:43:48.075330] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:43:48.075380] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:48.075388] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:49.670561] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:43:49.673557] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:49.677484] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:49.681055] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:49.681096] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:49.681146] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:49.681154] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:49.681709] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:49.681743] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:49.681751] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:49.681978] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:49.682020] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:49.682033] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:43:49.685492] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:49.686258] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:49.686311] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:43:49.701765] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:49.738155] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:49.738216] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:43:49.757605] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:49.757631] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:49.879179] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:49.879214] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:49.879222] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:49.879229] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:43:49.901360] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:43:49.919101] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:50.498063] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:43:50.534212] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:50.534258] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:50.534265] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:50.534284] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:43:50.543280] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:50.559004] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:50.559059] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:43:50.588998] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:50.589033] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:50.607703] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:50.614596] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:51.742959] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:53.139167] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:43:53.150508] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fc96b78047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fc96ba53182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fc96c54d265]))) 0-: received signum (15), shutting down [2016-03-15 14:43:53.162665] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:43:53.166621] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:43:53.167371] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:53.167385] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:53.167731] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:43:53.167742] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:43:53.167748] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:43:53.167785] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:43:53.167835] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:53.167843] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:54.773524] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:43:54.776944] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:54.780309] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:54.784329] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:54.784369] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:54.784418] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:54.784425] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:54.785010] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:54.785051] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:54.785060] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:54.785290] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:54.785327] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:54.785336] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:43:54.791564] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:54.791836] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:54.791887] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:43:54.859992] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:43:54.892405] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:54.892462] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:43:54.914883] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:54.914909] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:55.000544] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:55.000588] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:55.000611] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:55.000617] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:55.000624] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:43:55.224130] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:43:55.256636] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:55.256668] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:55.256675] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:55.256682] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:43:55.267342] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:55.278931] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:55.278976] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:43:55.296154] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:55.296188] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:55.318780] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:55.323146] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:55.656553] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:56.849222] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:58.226948] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:43:58.238217] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fefce80047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fefcead3182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fefcf5cd265]))) 0-: received signum (15), shutting down [2016-03-15 14:43:58.249839] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:43:58.253882] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:43:58.254593] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:58.254610] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:58.255554] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:43:58.255568] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:43:58.255574] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:43:58.255609] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:43:58.255660] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:58.255668] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:59.925556] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:43:59.929567] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:59.934489] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:59.937810] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:59.937853] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:59.937902] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:59.937910] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:59.938452] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:59.938489] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:59.938497] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:59.938722] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:59.938790] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:59.938804] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:43:59.943609] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:59.944233] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:59.944285] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:43:59.958980] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:59.994953] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:59.995019] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:44:00.014264] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:00.014291] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:00.041870] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:44:00.079591] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:00.079637] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:00.079661] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:00.079668] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:00.079675] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:44:01.225314] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:44:01.262455] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:01.262490] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:01.262497] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:01.262504] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:44:01.294497] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:01.308905] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:01.308952] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:44:01.332593] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:01.332627] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:01.351735] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:01.355351] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:02.001604] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:03.312310] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:44:03.322964] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fba5f8d747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fba5fbaa182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fba606a4265]))) 0-: received signum (15), shutting down [2016-03-15 14:44:03.334718] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:44:03.339148] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:44:03.340164] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:03.340180] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:03.340520] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:44:03.340532] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:44:03.340538] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:44:03.340572] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:44:03.340623] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:03.340631] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:05.001131] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:44:05.004230] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:05.007360] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:05.012772] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:05.012813] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:05.012863] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:05.012870] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:05.013419] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:05.013475] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:05.013484] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:05.013715] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:05.013750] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:05.013757] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:44:05.019028] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:05.019847] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:05.019899] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:44:05.172929] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:05.216969] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:05.217038] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:44:05.247422] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:05.247460] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:05.305622] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:44:05.329682] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:05.329719] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:05.329727] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:05.329734] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:44:05.329779] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:44:05.363435] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:05.363464] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:05.501749] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:05.629273] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:05.629329] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:44:05.677137] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:05.677187] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:05.752908] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:05.752947] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:07.076364] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:08.396724] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:44:08.407302] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f69bb2d847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f69bb5ab182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f69bc0a5265]))) 0-: received signum (15), shutting down [2016-03-15 14:44:08.420323] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:44:08.424488] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:44:08.424999] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:08.425011] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:08.425357] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:44:08.425368] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:44:08.425374] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:44:08.425409] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:44:08.425460] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:08.425468] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:10.069903] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:44:10.072625] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:10.075061] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:10.077351] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:10.077390] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:10.077439] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:10.077447] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:10.077943] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:10.077982] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:10.077990] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:10.078207] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:10.078245] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:10.078254] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:44:10.099958] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:10.100279] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:10.100338] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:44:10.112282] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:10.112342] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:44:10.131362] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:10.131395] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:10.203022] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:44:10.251932] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:10.251968] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:10.251976] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:10.251983] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:44:10.252036] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:44:10.276123] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:10.288254] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:10.288304] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:44:10.304251] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:10.304276] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:10.322938] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:10.338308] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:10.659267] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:12.157823] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:13.483836] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:44:13.495456] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f708f02b47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f708f2fe182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f708fdf8265]))) 0-: received signum (15), shutting down [2016-03-15 14:44:13.507323] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:44:13.511925] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:44:13.512631] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:13.512648] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:13.513119] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:44:13.513132] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:44:13.513138] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:44:13.513179] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:44:13.513231] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:13.513239] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:15.093869] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:44:15.096667] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:15.099639] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:15.102356] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:15.102397] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:15.102448] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:15.102456] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:15.103010] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:15.103046] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:15.103053] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:15.103286] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:15.103331] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:15.103343] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:44:15.107728] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:15.108580] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:15.108637] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:44:15.133406] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:15.172221] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:44:15.191482] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:15.191538] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:44:15.210582] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:15.210605] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:15.264742] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:15.264787] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:15.264810] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:15.264824] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:15.264831] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:44:16.145948] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:44:16.241936] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:16.283346] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:16.283404] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:44:16.299543] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:16.299568] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:16.316539] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:16.334015] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:17.164843] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:18.570205] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:44:18.580863] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7eff1393247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7eff13c05182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7eff146ff265]))) 0-: received signum (15), shutting down [2016-03-15 14:44:18.592827] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:44:18.595147] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:44:18.595686] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:18.595699] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:18.596036] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:44:18.596047] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:44:18.596053] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:44:18.596092] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:44:18.596143] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:18.596151] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:20.296541] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:44:20.299695] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:20.302931] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:20.305825] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:20.305866] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:20.305916] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:20.305932] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:20.306504] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:20.306540] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:20.306547] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:20.306818] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:20.306852] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:20.306859] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:44:20.311378] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:20.312339] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:20.312393] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:44:20.334617] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:20.352265] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:20.352318] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:44:20.374167] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:20.374189] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:20.453272] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:20.453310] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:20.453323] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:20.453334] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:44:20.453387] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:44:20.490754] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:44:20.567615] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:20.624663] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:20.624721] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:44:20.686579] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:20.686620] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:20.707302] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:20.707332] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:22.368248] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:23.657680] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:44:23.668317] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ff55f50c47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ff55f7df182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ff5602d9265]))) 0-: received signum (15), shutting down [2016-03-15 14:44:23.680264] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:44:23.683419] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:44:23.684128] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:23.684146] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:23.684658] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:44:23.684676] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:44:23.684685] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:44:23.684737] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:44:23.684812] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:23.684827] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:25.387122] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:44:25.390818] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:25.393734] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:25.396700] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:25.396740] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:25.396789] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:25.396796] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:25.397352] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:25.397387] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:25.397394] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:25.397611] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:25.397648] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:25.397658] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:44:25.405574] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:25.406643] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:25.406695] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:44:25.503628] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:25.503701] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:44:25.530068] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:25.530098] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:25.550868] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:44:25.570838] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:25.570869] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:25.570877] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:25.570884] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:44:25.570930] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:44:25.609891] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:25.609919] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:25.661922] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:26.505481] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:26.525599] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:26.525653] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:44:26.544842] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:26.544864] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:26.579116] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:26.610317] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:27.463526] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:28.744194] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:44:28.754659] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fb316a8e47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fb316d61182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fb31785b265]))) 0-: received signum (15), shutting down [2016-03-15 14:44:28.766876] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:44:28.770874] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:44:28.771464] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:28.771477] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:28.771823] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:44:28.771835] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:44:28.771841] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:44:28.771877] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:44:28.771928] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:28.771936] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:30.416959] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:44:30.420680] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:30.432950] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:30.440951] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:30.440993] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:30.441041] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:30.441049] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:30.441572] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:30.441609] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:30.441617] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:30.441846] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:30.441890] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:30.441907] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:44:30.446333] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:30.447594] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:30.447651] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:44:30.523915] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:30.523996] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:44:30.561565] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:30.561601] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:30.637443] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:30.637481] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:30.637489] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:30.637496] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:44:30.711873] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:44:30.769423] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:31.231085] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:44:31.275806] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:31.275838] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:31.275846] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:31.275853] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:44:31.284833] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:31.303006] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:31.303053] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:44:31.343895] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:31.343924] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:31.408852] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:31.408885] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:31.662677] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:32.502864] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:33.830178] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:44:33.841551] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f60c7e1147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f60c80e4182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f60c8bde265]))) 0-: received signum (15), shutting down [2016-03-15 14:44:33.852851] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:44:33.856854] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:44:33.857366] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:33.857377] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:33.857715] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:44:33.857727] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:44:33.857732] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:44:33.857766] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:44:33.857820] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:33.857828] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:35.472932] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:44:35.475968] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:35.478728] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:35.482606] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:35.482646] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:35.482692] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:35.482700] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:35.483313] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:35.483356] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:35.483364] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:35.483621] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:35.483654] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:35.483660] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:44:35.488056] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:35.488716] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:35.488768] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:44:35.500456] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:35.547010] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:35.547076] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:44:35.623614] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:35.623654] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:35.736357] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:35.736390] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:35.736398] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:35.736405] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:44:35.736451] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:44:35.766869] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:44:35.815548] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:35.815582] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:35.815590] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:35.815597] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:44:35.816037] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:35.830186] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:35.830230] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:44:35.849586] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:35.849612] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:35.872154] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:35.872182] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:37.543888] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:38.915603] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:44:38.926016] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fe7f14e647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fe7f17b9182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fe7f22b3265]))) 0-: received signum (15), shutting down [2016-03-15 14:44:38.937325] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:44:38.941419] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:44:38.941966] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:38.941978] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:38.942320] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:44:38.942337] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:44:38.942343] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:44:38.942378] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:44:38.942427] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:38.942435] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:40.565922] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:44:40.568709] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:40.571788] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:40.576697] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:40.576737] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:40.576783] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:40.576790] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:40.577277] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:40.577314] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:40.577322] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:40.577541] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:40.577575] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:40.577583] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:44:40.581620] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:40.582667] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:40.582719] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:44:40.660549] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:40.660611] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:44:40.678155] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:40.678173] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:40.731251] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:44:40.748247] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:40.748275] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:40.748288] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:40.748296] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:44:40.748364] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:40.749075] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:40.878927] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:44:41.508315] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:41.522904] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:41.522952] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:44:41.558971] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:41.559005] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:41.575940] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:41.575968] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:42.638790] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:44.001864] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:44:44.012147] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f74943ac47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f749467f182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f7495179265]))) 0-: received signum (15), shutting down [2016-03-15 14:44:44.023889] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:44:44.028432] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:44:44.029006] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:44.029019] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:44.029362] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:44:44.029373] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:44:44.029379] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:44:44.029415] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:44:44.029465] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:44.029473] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:45.680809] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:44:45.685246] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:45.688118] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:45.691347] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:45.691388] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:45.691436] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:45.691443] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:45.692045] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:45.692090] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:45.692100] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:45.692366] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:45.692398] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:45.692406] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:44:45.696034] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:45.697169] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:45.697222] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:44:45.849608] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:44:45.868221] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:45.868278] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:44:45.892470] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:45.892497] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:45.910686] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:45.910716] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:45.910724] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:45.910730] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:44:45.910785] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:44:45.931353] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:46.234498] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:46.253228] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:46.253281] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:44:46.273814] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:46.273840] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:46.308215] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:46.311988] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:46.665472] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:46.743476] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:47.753065] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:49.085691] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:44:49.098969] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fafc393547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fafc3c08182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fafc4702265]))) 0-: received signum (15), shutting down [2016-03-15 14:44:49.110168] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:44:49.114123] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:44:49.114626] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:49.114638] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:49.115505] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:44:49.115524] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:44:49.115534] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:44:49.115585] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:44:49.115662] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:49.115675] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:50.805772] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:44:50.809076] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:50.813678] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:50.819390] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:50.819431] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:50.819482] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:50.819490] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:50.820046] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:50.820089] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:50.820099] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:50.820342] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:50.820388] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:50.820396] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:44:50.830168] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:50.830837] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:50.830891] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:44:50.852713] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:50.871961] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:50.917257] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:50.917322] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:44:50.934353] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:50.934370] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:50.971193] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:50.971260] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:44:51.005618] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:51.005655] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:51.025509] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:51.025536] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:51.025543] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:51.025550] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:44:51.025590] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:44:51.043300] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:51.043329] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:51.043336] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:51.043343] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:44:51.043393] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:44:51.098871] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:51.098905] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:51.098920] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:52.886587] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:54.171979] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:44:54.183409] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f5239b0547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f5239dd8182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f523a8d2265]))) 0-: received signum (15), shutting down [2016-03-15 14:44:54.195136] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:44:54.199519] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:44:54.200221] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:54.200242] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:54.200759] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:44:54.200776] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:44:54.200786] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:44:54.200837] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:44:54.200913] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:54.200927] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:55.781858] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:44:55.785147] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:55.788823] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:55.792680] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:55.792719] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:55.792769] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:55.792777] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:55.793382] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:55.793425] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:55.793434] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:55.793696] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:55.793729] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:55.793736] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:44:55.797792] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:55.798624] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:55.798676] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:44:55.814786] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:55.976996] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:55.977062] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:44:55.999615] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:55.999650] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:56.036750] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:44:56.057094] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:56.057129] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:56.057137] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:56.057143] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:44:56.057194] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:44:56.087264] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:56.087288] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:56.511234] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:56.530073] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:56.530124] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:44:56.548500] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:56.548520] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:56.571838] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:56.624740] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:57.854425] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:59.256375] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:44:59.266811] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fabf09f347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fabf0cc6182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fabf17c0265]))) 0-: received signum (15), shutting down [2016-03-15 14:44:59.279506] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:44:59.284775] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:44:59.285480] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:59.285498] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:59.285992] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:44:59.286005] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:44:59.286011] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:44:59.286047] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:44:59.286099] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:59.286107] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:00.934817] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:45:00.938193] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:00.943996] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:00.947079] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:00.947120] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:00.947168] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:00.947176] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:00.947759] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:00.947795] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:00.947804] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:00.948032] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:00.948106] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:00.948116] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:45:00.952369] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:00.953124] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:00.953188] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:45:01.027809] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:01.027869] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:45:01.042735] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:01.042760] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:01.076520] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:45:01.096041] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:01.096074] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:01.096081] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:01.096088] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:45:01.096134] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:01.236810] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:45:01.285674] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:01.285715] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:01.285723] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:01.285730] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:45:01.333105] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:01.391524] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:01.391587] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:45:01.413722] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:01.413763] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:01.434849] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:01.434890] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:01.668247] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:03.009224] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:04.342226] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:45:04.352890] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f149c64547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f149c918182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f149d412265]))) 0-: received signum (15), shutting down [2016-03-15 14:45:04.365238] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:45:04.368361] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:45:04.369063] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:04.369081] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:04.369591] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:45:04.369612] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:45:04.369624] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:45:04.369677] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:45:04.369754] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:04.369768] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:06.195257] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:45:06.237375] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:06.282966] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:06.294900] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:06.294952] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:06.295004] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:06.295011] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:06.295527] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:06.295563] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:06.295571] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:06.295789] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:06.295834] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:06.295847] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:45:06.303149] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:06.303785] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:06.303870] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:45:06.348669] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:06.403136] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:45:06.469081] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:06.469143] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:45:06.535251] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:06.535289] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:06.584677] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:06.584725] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:06.584749] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:06.584756] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:06.584763] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:45:07.238066] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:45:07.275771] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:07.275804] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:07.275811] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:07.275818] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:45:07.276218] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:07.297973] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:07.298021] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:45:07.327155] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:07.327182] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:07.376465] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:07.376504] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:08.360380] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:09.432464] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:45:09.443334] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ff57799e47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ff577c71182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ff57876b265]))) 0-: received signum (15), shutting down [2016-03-15 14:45:09.454894] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:45:09.459153] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:45:09.459862] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:09.459881] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:09.460410] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:45:09.460427] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:45:09.460437] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:45:09.460491] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:45:09.460569] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:09.460583] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:11.361761] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:45:11.369827] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:11.373513] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:11.378189] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:11.378234] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:11.378284] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:11.378292] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:11.378920] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:11.378958] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:11.378966] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:11.379227] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:11.379259] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:11.379266] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:45:11.384676] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:11.385053] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:11.385156] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:45:11.404253] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:11.429792] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:11.429861] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:45:11.457108] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:11.457148] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:11.586715] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:45:11.707977] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:45:11.790500] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:11.790537] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:11.790545] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:11.790552] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:45:11.790619] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:11.790632] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:11.790638] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:11.790644] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:45:11.791508] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:11.827419] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:11.827478] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:45:11.853519] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:11.853547] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:11.887724] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:11.887756] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:13.442483] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:14.517531] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:45:14.529167] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa9e690647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa9e6bd9182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa9e76d3265]))) 0-: received signum (15), shutting down [2016-03-15 14:45:14.540725] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:45:14.544293] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:45:14.544809] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:14.544821] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:14.545164] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:45:14.545175] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:45:14.545188] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:45:14.545224] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:45:14.545273] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:14.545281] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:16.373027] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:45:16.377637] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:16.380657] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:16.384608] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:16.384649] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:16.384698] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:16.384706] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:16.385274] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:16.385312] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:16.385320] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:16.385553] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:16.385592] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:16.385601] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:45:16.394153] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:16.395449] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:16.395506] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:45:16.496340] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:16.496412] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:45:16.528520] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:16.528550] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:16.667712] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:45:16.717338] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:16.717376] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:16.717393] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:16.717400] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:45:16.764712] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:45:16.795802] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:16.795833] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:16.796535] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:17.515643] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:17.557553] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:17.557617] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:45:17.671475] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:17.671518] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:17.738653] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:17.738688] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:18.451018] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:19.608786] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:45:19.619349] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f78a677b47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f78a6a4e182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f78a7548265]))) 0-: received signum (15), shutting down [2016-03-15 14:45:19.631257] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:45:19.635732] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:45:19.636688] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:19.636710] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:19.637303] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:45:19.637321] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:45:19.637332] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:45:19.637426] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:45:19.637497] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:19.637509] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:21.558557] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:45:21.572223] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:21.583855] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:21.609134] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:21.609186] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:21.609238] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:21.609251] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:21.609796] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:21.609831] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:21.609839] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:21.610061] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:21.610104] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:21.610113] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:45:21.625532] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:21.626612] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:21.626666] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:45:21.691780] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:21.988356] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:21.988424] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:45:22.031254] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:22.031294] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:22.075470] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:45:22.102768] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:45:22.137142] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:22.137176] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:22.137184] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:22.137191] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:45:22.137247] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:22.137272] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:22.241498] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:22.272278] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:22.272335] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:45:22.304349] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:22.304381] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:22.347741] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:22.347783] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:23.681829] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:24.692226] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:45:24.702465] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ffc9d8a547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ffc9db78182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ffc9e672265]))) 0-: received signum (15), shutting down [2016-03-15 14:45:24.714216] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:45:24.716362] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:45:24.716863] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:24.716875] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:24.717212] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:45:24.717223] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:45:24.717229] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:45:24.717264] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:45:24.717314] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:24.717322] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:26.602857] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:45:26.607760] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:26.622438] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:26.644226] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:26.644272] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:26.644323] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:26.644331] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:26.644880] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:26.644915] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:26.644923] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:26.645156] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:26.645193] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:26.645202] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:45:26.665094] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:26.665554] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:26.665615] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:45:26.768059] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:26.768139] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:45:26.980480] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:26.980520] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:27.012625] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:27.065248] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:27.192041] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:45:27.222639] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:27.222672] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:27.222680] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:27.222687] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:45:27.222767] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:45:27.259592] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:27.259646] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:45:27.290421] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:27.290448] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:27.326505] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:27.326532] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:27.326546] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:27.326553] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:45:27.326596] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:27.326617] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:28.722462] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:29.778716] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:45:29.789105] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ff31afa247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ff31b275182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ff31bd6f265]))) 0-: received signum (15), shutting down [2016-03-15 14:45:29.801048] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:45:29.804106] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:45:29.804665] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:29.804682] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:29.805097] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:45:29.805109] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:45:29.805115] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:45:29.805151] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:45:29.805203] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:29.805210] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:31.429183] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:45:31.433575] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:31.436764] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:31.439784] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:31.439823] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:31.439872] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:31.439879] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:31.440477] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:31.440512] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:31.440520] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:31.440780] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:31.440812] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:31.440818] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:45:31.445132] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:31.446230] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:31.446283] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:45:31.510992] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:31.511053] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:45:31.558627] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:31.558663] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:31.657585] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:31.657619] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:31.657627] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:31.657635] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:45:31.678407] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:45:31.697371] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:31.698047] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:32.254356] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:45:32.518592] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:32.558120] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:32.558182] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:45:32.598638] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:32.598668] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:32.616564] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:32.616592] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:33.502115] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:34.865662] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:45:34.877582] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fede846147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fede8734182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fede922e265]))) 0-: received signum (15), shutting down [2016-03-15 14:45:34.888891] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:45:34.892684] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:45:34.893296] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:34.893312] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:34.894713] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:45:34.894745] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:45:34.894759] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:45:34.894810] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:45:34.894887] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:34.894901] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:36.580569] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:45:36.583909] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:36.587947] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:36.591662] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:36.591704] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:36.591754] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:36.591761] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:36.592292] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:36.592327] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:36.592334] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:36.592554] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:36.592592] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:36.592604] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:45:36.603333] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:36.604699] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:36.604752] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:45:36.661752] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:36.661824] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:45:36.688096] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:36.688129] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:36.750192] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:45:36.770371] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:45:36.819890] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:36.819936] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:36.819943] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:36.819950] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:45:36.820001] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:37.244606] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:37.257739] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:37.257784] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:45:37.278447] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:37.278472] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:37.303206] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:37.316870] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:37.674506] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:38.660518] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:39.949545] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:45:39.960043] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f61c03a447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f61c0677182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f61c1171265]))) 0-: received signum (15), shutting down [2016-03-15 14:45:39.972050] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:45:39.976219] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:45:39.976741] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:39.976753] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:39.977094] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:45:39.977106] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:45:39.977118] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:45:39.977153] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:45:39.977204] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:39.977212] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:41.693333] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:45:41.696696] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:41.699730] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:41.703489] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:41.703530] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:41.703578] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:41.703586] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:41.704172] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:41.704215] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:41.704225] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:41.704485] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:41.704521] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:41.704528] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:45:41.708757] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:41.709585] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:41.709636] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:45:41.727965] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:41.742908] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:41.801183] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:45:41.825882] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:41.825937] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:45:41.844040] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:41.844075] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:41.887019] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:41.887088] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:45:41.906508] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:41.906541] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:41.928114] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:41.928152] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:41.928173] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:41.928180] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:41.928187] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:45:41.928239] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:45:41.951749] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:41.951776] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:41.951782] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:41.951789] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:45:41.973599] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:41.973630] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:43.766354] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:45.036810] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:45:45.047587] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fcaf7e4847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fcaf811b182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fcaf8c15265]))) 0-: received signum (15), shutting down [2016-03-15 14:45:45.060122] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:45:45.064065] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:45:45.064664] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:45.064678] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:45.065100] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:45:45.065113] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:45:45.065120] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:45:45.065156] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:45:45.065206] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:45.065220] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:46.796382] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:45:46.806216] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:46.811729] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:46.816618] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:46.816661] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:46.816711] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:46.816719] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:46.817279] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:46.817314] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:46.817322] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:46.817553] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:46.817592] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:46.817605] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:45:46.839912] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:46.840739] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:46.840798] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:45:46.870845] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:46.974506] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:46.974572] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:45:47.008883] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:47.008916] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:47.050174] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:45:47.073562] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:47.073592] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:47.073600] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:47.073607] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:45:47.130429] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:47.520925] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:45:47.620584] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:47.620631] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:47.620639] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:47.620646] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:45:47.693951] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:47.724079] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:47.724136] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:45:47.762881] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:47.762926] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:47.799897] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:47.876127] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:48.897727] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:50.123730] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:45:50.137196] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fcdcc22a47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fcdcc4fd182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fcdccff7265]))) 0-: received signum (15), shutting down [2016-03-15 14:45:50.149568] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:45:50.153099] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:45:50.153615] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:50.153626] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:50.153970] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:45:50.153981] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:45:50.153987] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:45:50.154022] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:45:50.154071] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:50.154079] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:51.763601] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:45:51.792618] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:51.799237] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:51.808294] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:51.808345] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:51.808396] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:51.808403] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:51.809003] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:51.809059] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:51.809094] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:51.809412] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:51.809446] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:51.809453] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:45:51.819415] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:51.820397] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:51.820450] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:45:51.947420] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:51.947486] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:45:51.965748] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:51.965774] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:52.020116] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:45:52.072259] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:52.072300] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:52.072307] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:52.072315] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:45:52.072362] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:45:52.155798] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:52.155829] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:52.247369] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:52.262659] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:52.262712] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:45:52.305988] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:52.306023] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:52.328303] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:52.352981] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:52.677095] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:53.876239] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:55.217051] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:45:55.229316] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fcdf67ef47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fcdf6ac2182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fcdf75bc265]))) 0-: received signum (15), shutting down [2016-03-15 14:45:55.241130] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:45:55.244739] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:45:55.245247] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:55.245259] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:55.245602] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:45:55.245614] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:45:55.245620] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:45:55.245658] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:45:55.245709] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:55.245717] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:56.856252] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:45:56.860098] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:56.863135] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:56.866577] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:56.866616] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:56.866665] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:56.866673] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:56.867202] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:56.867241] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:56.867258] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:56.867480] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:56.867515] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:56.867523] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:45:56.871877] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:56.872660] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:56.872713] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:45:56.901305] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:56.916520] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:56.974895] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:56.974956] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:45:56.993785] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:56.993813] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:57.033075] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:45:57.069175] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:57.069247] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:45:57.089130] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:57.089164] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:57.109982] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:45:57.178440] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:57.178476] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:57.178484] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:57.178491] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:45:57.178547] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:57.178580] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:57.178592] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:57.178598] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:57.178604] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:45:57.178635] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:57.182979] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:58.928482] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:00.312153] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:46:00.322775] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa6468cc47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa646b9f182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa647699265]))) 0-: received signum (15), shutting down [2016-03-15 14:46:00.334397] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:46:00.338516] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:46:00.339826] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:00.339847] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:00.340325] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:46:00.340337] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:46:00.340344] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:46:00.340379] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:46:00.340431] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:00.340439] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:02.102995] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:46:02.106457] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:02.109743] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:02.112554] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:02.112595] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:02.112646] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:02.112654] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:02.113244] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:02.113287] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:02.113297] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:02.113535] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:02.113601] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:02.113610] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:46:02.116846] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:02.117739] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:02.117817] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:46:02.132515] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:02.483705] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:02.483775] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:46:02.542608] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:02.542643] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:02.613261] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:46:02.652212] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:02.652284] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:46:02.676727] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:02.676759] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:02.676767] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:02.676774] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:46:02.677227] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:02.755773] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:02.755835] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:46:02.775087] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:02.775113] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:02.802234] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:02.802261] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:02.802275] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:02.802282] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:46:02.802325] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:02.802344] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:02.802363] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:04.174245] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:05.396999] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:46:05.407342] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f1047aaa47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f1047d7d182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f1048877265]))) 0-: received signum (15), shutting down [2016-03-15 14:46:05.419624] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:46:05.423701] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:46:05.424319] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:05.424333] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:05.424680] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:46:05.424692] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:46:05.424697] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:46:05.424733] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:46:05.424785] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:05.424793] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:07.194040] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:46:07.200212] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:07.214178] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:07.217681] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:07.217721] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:07.217771] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:07.217778] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:07.218391] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:07.218439] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:07.218450] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:07.218666] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:07.218711] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:07.218722] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:46:07.223810] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:07.224189] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:07.224240] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:46:07.244095] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:07.244141] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:46:07.294351] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:07.294386] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:07.359815] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:46:07.411323] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:07.411360] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:07.411368] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:07.411375] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:46:07.411424] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:46:07.429626] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:07.429654] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:07.429662] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:07.429668] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:46:07.430055] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:07.446930] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:07.446976] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:46:07.474560] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:07.474595] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:07.499843] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:07.502087] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:07.679915] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:09.280483] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:10.482141] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:46:10.492559] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f401f04047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f401f313182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f401fe0d265]))) 0-: received signum (15), shutting down [2016-03-15 14:46:10.504518] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:46:10.507456] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:46:10.508155] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:10.508173] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:10.508676] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:46:10.508693] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:46:10.508704] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:46:10.508772] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:46:10.508849] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:10.508864] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:12.312739] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:46:12.319920] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:12.326328] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:12.333557] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:12.333600] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:12.333649] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:12.333656] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:12.334190] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:12.334226] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:12.334233] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:12.334451] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:12.334496] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:12.334507] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:46:12.343154] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:12.344233] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:12.344288] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:46:12.382094] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:12.496128] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:46:12.529860] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:12.529927] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:46:12.561208] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:12.561246] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:12.625732] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:12.625777] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:12.625800] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:12.625807] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:12.625814] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:46:12.625858] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:46:12.662663] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:13.251478] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:13.273458] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:13.273512] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:46:13.324346] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:13.324378] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:13.356004] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:13.366095] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:14.399704] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:15.572500] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:46:15.583811] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f84c570947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f84c59dc182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f84c64d6265]))) 0-: received signum (15), shutting down [2016-03-15 14:46:15.595594] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:46:15.599675] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:46:15.600251] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:15.600264] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:15.600612] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:46:15.600624] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:46:15.600630] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:46:15.600665] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:46:15.600717] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:15.600725] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:17.375254] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:46:17.378866] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:17.382450] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:17.385225] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:17.385264] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:17.385311] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:17.385319] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:17.385892] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:17.385930] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:17.385938] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:17.386200] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:17.386233] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:17.386241] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:46:17.390285] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:17.391238] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:17.391352] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:46:17.406662] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:17.406740] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:46:17.425905] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:17.425948] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:17.463157] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:17.546157] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:46:17.609212] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:17.609251] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:17.609259] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:17.609266] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:46:17.609958] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:17.624820] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:17.624862] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:46:17.643688] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:17.643715] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:17.758686] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:17.758728] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:46:17.777193] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:17.777222] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:17.777229] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:17.777236] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:46:19.446615] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:20.656933] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:46:20.667154] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f2040a1547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f2040ce8182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f20417e2265]))) 0-: received signum (15), shutting down [2016-03-15 14:46:20.678504] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:46:20.682489] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:46:20.683753] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:20.683773] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:20.684316] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:46:20.684334] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:46:20.684341] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:46:20.684377] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:46:20.684430] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:20.684439] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:22.308915] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:46:22.331946] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:22.335552] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:22.341106] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:22.341151] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:22.341203] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:22.341211] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:22.341794] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:22.341832] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:22.341839] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:22.342076] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:22.342112] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:22.342120] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:46:22.351532] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:22.352633] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:22.352688] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:46:22.506757] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:22.506826] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:46:22.578319] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:22.578354] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:22.600145] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:46:22.633893] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:22.633929] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:22.633944] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:22.633952] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:46:22.633998] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:46:22.658257] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:22.658281] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:22.682625] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:23.528365] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:23.611913] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:23.611977] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:46:23.669912] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:23.669949] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:23.688700] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:23.688744] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:24.407557] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:25.739736] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:46:25.749905] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f395651547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f39567e8182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f39572e2265]))) 0-: received signum (15), shutting down [2016-03-15 14:46:25.761729] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:46:25.763918] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:46:25.764423] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:25.764434] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:25.764768] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:46:25.764779] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:46:25.764785] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:46:25.764821] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:46:25.764872] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:25.764881] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:27.416825] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:46:27.422261] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:27.426551] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:27.430052] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:27.430108] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:27.430181] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:27.430193] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:27.430857] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:27.430921] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:27.430932] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:27.431180] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:27.431217] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:27.431225] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:46:27.435768] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:27.436844] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:27.436899] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:46:27.515057] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:27.515123] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:46:27.535270] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:27.535300] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:27.607618] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:46:27.627383] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:27.627418] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:27.627425] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:27.627433] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:46:27.627492] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:28.000022] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:46:28.254372] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:28.271536] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:28.271587] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:46:28.293924] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:28.293952] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:28.317563] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:28.336651] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:28.683760] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:29.492595] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:30.826840] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:46:30.837503] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f7e6892d47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f7e68c00182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f7e696fa265]))) 0-: received signum (15), shutting down [2016-03-15 14:46:30.848746] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:46:30.852687] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:46:30.853232] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:30.853245] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:30.854090] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:46:30.854108] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:46:30.854118] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:46:30.854171] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:46:30.854247] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:30.854261] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:32.532487] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:46:32.536746] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:32.542081] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:32.550155] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:32.550196] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:32.550245] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:32.550252] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:32.550773] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:32.550811] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:32.550819] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:32.551057] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:32.551102] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:32.551125] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:46:32.557701] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:32.558899] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:32.558956] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:46:32.594393] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:32.608104] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:32.650884] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:32.650954] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:46:32.717298] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:32.717325] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:32.783447] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:32.783504] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:46:32.847576] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:32.847611] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:32.869065] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:32.869097] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:32.869105] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:32.869111] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:46:32.869155] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:46:32.897488] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:46:32.916999] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:32.917026] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:32.917033] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:32.917046] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:46:32.917107] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:32.917121] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:34.614662] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:35.911479] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:46:35.922727] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fbcdb13047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fbcdb403182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fbcdbefd265]))) 0-: received signum (15), shutting down [2016-03-15 14:46:35.933989] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:46:35.937020] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:46:35.937524] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:35.937536] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:35.937876] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:46:35.937887] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:46:35.937893] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:46:35.937929] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:46:35.937979] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:35.937987] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:37.703381] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:46:37.708028] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:37.711105] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:37.713936] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:37.713977] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:37.714026] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:37.714033] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:37.714611] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:37.714645] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:37.714653] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:37.714934] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:37.714968] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:37.714975] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:46:37.718885] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:37.719634] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:37.719700] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:46:37.747138] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:37.984407] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:37.984495] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:46:38.044532] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:38.044570] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:38.090102] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:46:38.125967] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:46:38.187304] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:38.187345] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:38.187353] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:38.187359] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:46:38.187422] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:38.187444] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:38.531538] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:38.549693] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:38.549754] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:46:38.580628] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:38.580661] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:38.608646] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:38.610882] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:39.775511] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:40.999510] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:46:41.009694] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fabd8ac447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fabd8d97182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fabd9891265]))) 0-: received signum (15), shutting down [2016-03-15 14:46:41.020876] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:46:41.024873] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:46:41.025379] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:41.025391] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:41.025726] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:46:41.025737] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:46:41.025743] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:46:41.025779] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:46:41.025844] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:41.025854] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:42.755768] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:46:42.777329] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:42.785849] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:42.799696] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:42.799741] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:42.799793] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:42.799800] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:42.800383] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:42.800425] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:42.800434] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:42.800688] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:42.800720] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:42.800727] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:46:42.834863] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:42.835548] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:42.835604] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:46:42.937472] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:46:42.988813] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:42.988875] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:46:43.049491] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:43.049517] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:43.127554] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:43.127602] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:43.127626] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:43.127634] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:43.127640] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:46:43.256748] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:46:43.291110] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:43.291142] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:43.291149] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:43.291156] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:46:43.320363] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:43.342751] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:43.342818] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:46:43.376449] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:43.376469] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:43.416716] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:43.416749] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:44.157312] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:44.890446] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:46.083961] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:46:46.094394] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f2e128b647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f2e12b89182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f2e13683265]))) 0-: received signum (15), shutting down [2016-03-15 14:46:46.106431] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:46:46.110315] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:46:46.111123] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:46.111138] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:46.111589] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:46:46.111601] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:46:46.111607] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:46:46.111646] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:46:46.111697] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:46.111705] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:47.883220] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:46:47.889499] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:47.893232] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:47.897786] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:47.897830] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:47.897884] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:47.897893] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:47.898499] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:47.898543] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:47.898553] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:47.898796] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:47.898835] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:47.898844] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:46:47.904478] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:47.905141] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:47.905192] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:46:47.925288] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:47.925348] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:46:48.020152] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:48.020208] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:48.346199] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:48.638711] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:48.713316] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:46:48.793234] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:48.793272] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:48.793285] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:48.793297] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:46:48.793362] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:46:48.817189] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:48.817216] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:48.817223] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:48.817230] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:46:48.817281] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:48.817321] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:46:48.838210] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:48.838235] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:48.866751] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:48.866787] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:49.961090] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:51.168638] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:46:51.181205] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd18ae0247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd18b0d5182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd18bbcf265]))) 0-: received signum (15), shutting down [2016-03-15 14:46:51.192558] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:46:51.196608] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:46:51.197126] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:51.197138] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:51.197466] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:46:51.197483] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:46:51.197489] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:46:51.197523] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:46:51.197572] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:51.197580] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:52.907024] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:46:52.910255] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:52.914827] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:52.918687] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:52.918728] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:52.918812] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:52.918821] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:52.919383] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:52.919421] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:52.919429] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:52.919654] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:52.919692] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:52.919701] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:46:52.926075] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:52.927272] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:52.927327] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:46:52.956303] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:53.123557] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:46:53.216689] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:53.216749] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:46:53.293165] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:53.293201] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:53.347379] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:46:53.433329] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:53.433377] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:53.433401] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:53.433408] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:53.433415] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:46:53.433458] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:53.534371] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:53.558713] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:53.558797] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:46:53.593574] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:53.593603] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:53.642417] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:53.867278] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:54.982602] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:56.253340] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:46:56.264724] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fca270c047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fca27393182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fca27e8d265]))) 0-: received signum (15), shutting down [2016-03-15 14:46:56.276296] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:46:56.280256] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:46:56.280862] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:56.280875] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:56.281224] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:46:56.281235] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:46:56.281241] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:46:56.281276] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:46:56.281326] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:56.281334] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:57.975664] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:46:58.018808] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:58.075916] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:58.123316] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:58.123373] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:58.123425] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:58.123433] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:58.124022] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:58.124066] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:58.124075] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:58.124342] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:58.124376] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:58.124383] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:46:58.177283] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:58.177532] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:58.177590] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:46:58.221572] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:58.221630] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:46:58.289901] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:58.289930] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:58.438455] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:46:58.464951] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:46:58.485364] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:58.485399] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:58.485407] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:58.485414] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:46:58.485458] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:58.485484] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:58.485491] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:58.485497] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:46:58.485958] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:58.516117] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:58.516164] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:46:58.566859] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:58.566888] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:58.648854] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:58.648896] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:58.688952] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:00.235275] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:01.337931] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:47:01.348518] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f90cef9c47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f90cf26f182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f90cfd69265]))) 0-: received signum (15), shutting down [2016-03-15 14:47:01.360438] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:47:01.364543] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:47:01.365055] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:01.365067] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:01.365403] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:47:01.365414] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:47:01.365421] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:47:01.365455] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:47:01.365505] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:01.365513] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:03.218404] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:47:03.222132] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:03.225340] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:03.230936] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:03.230982] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:03.231034] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:03.231051] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:03.231645] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:03.231683] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:03.231691] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:03.231915] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:03.231959] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:03.231972] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:47:03.235718] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:03.236637] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:03.236691] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:47:03.263369] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:03.375036] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:47:03.431594] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:03.431675] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:47:03.496735] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:03.496771] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:03.731729] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:03.731786] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:03.731811] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:03.731818] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:03.731825] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:47:04.260895] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:47:04.311549] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:04.311595] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:04.311602] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:04.311621] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:47:04.348576] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:04.364913] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:04.364966] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:47:04.390008] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:04.390038] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:04.426833] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:04.426876] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:05.292436] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:06.422058] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:47:06.432733] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f7403c8c47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f7403f5f182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f7404a59265]))) 0-: received signum (15), shutting down [2016-03-15 14:47:06.444295] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:47:06.448388] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:47:06.448941] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:06.448953] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:06.449289] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:47:06.449301] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:47:06.449307] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:47:06.449341] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:47:06.449391] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:06.449399] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:08.319911] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:47:08.344961] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:08.368217] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:08.376821] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:08.376867] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:08.376917] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:08.376924] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:08.377508] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:08.377546] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:08.377554] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:08.377822] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:08.377855] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:08.377862] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:47:08.388380] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:08.389295] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:08.389352] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:47:08.461338] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:08.644420] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:47:08.676582] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:08.676641] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:47:08.707679] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:08.707713] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:08.795643] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:08.795685] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:08.795709] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:08.795717] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:08.795724] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:47:08.795780] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:47:08.825047] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:08.825080] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:08.825088] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:08.825096] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:47:08.826011] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:08.841344] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:08.841394] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:47:08.860534] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:08.860569] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:08.894266] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:08.894297] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:10.446353] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:11.506060] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:47:11.516564] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa2a6ba547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa2a6e78182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa2a7972265]))) 0-: received signum (15), shutting down [2016-03-15 14:47:11.528403] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:47:11.532200] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:47:11.532711] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:11.532724] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:11.533636] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:47:11.533653] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:47:11.533664] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:47:11.533719] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:47:11.533856] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:11.533872] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:13.563294] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:47:13.584346] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:13.597106] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:13.633006] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:13.633059] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:13.633110] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:13.633117] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:13.633692] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:13.633728] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:13.633736] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:13.633953] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:13.633998] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:13.634011] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:47:13.647312] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:13.647923] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:13.647977] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:47:13.704005] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:13.943861] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:13.943926] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:47:13.988223] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:13.988260] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:14.044288] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:47:14.080343] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:14.080410] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:47:14.109200] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:14.109236] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:14.132572] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:47:14.156872] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:14.156903] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:14.156911] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:14.156917] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:47:14.156964] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:14.156990] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:14.157003] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:14.157009] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:14.157015] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:47:14.157058] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:14.157078] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:14.157124] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:14.157390] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:15.703693] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:16.593929] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:47:16.605352] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f635855647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f6358829182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f6359323265]))) 0-: received signum (15), shutting down [2016-03-15 14:47:16.616536] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:47:16.620536] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:47:16.621077] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:16.621090] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:16.621434] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:47:16.621445] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:47:16.621451] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:47:16.621485] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:47:16.621536] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:16.621544] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:18.453546] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:47:18.480646] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:18.495895] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:18.512845] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:18.512896] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:18.512946] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:18.512954] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:18.513537] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:18.513581] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:18.513591] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:18.513849] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:18.513882] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:18.513889] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:47:18.526563] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:18.527309] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:18.527366] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:47:18.699334] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:18.776084] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:47:18.806038] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:47:18.831804] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:18.831864] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:47:18.862044] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:18.862073] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:18.889094] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:18.889135] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:18.889157] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:18.889164] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:18.889170] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:47:18.925523] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:19.264382] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:19.300342] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:19.300396] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:47:19.339561] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:19.339589] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:19.369652] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:19.370893] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:20.582900] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:21.676986] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:47:21.687163] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fedba5c347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fedba896182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fedbb390265]))) 0-: received signum (15), shutting down [2016-03-15 14:47:21.699598] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:47:21.704310] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:47:21.704828] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:21.704840] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:21.705181] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:47:21.705192] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:47:21.705198] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:47:21.705233] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:47:21.705285] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:21.705294] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:23.586171] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:47:23.611886] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:23.635739] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:23.648006] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:23.648052] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:23.648104] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:23.648112] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:23.648670] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:23.648706] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:23.648714] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:23.648953] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:23.648987] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:23.648994] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:47:23.663680] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:23.664445] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:23.664499] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:47:23.734721] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:23.851166] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:23.851237] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:47:23.888615] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:23.888643] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:24.032127] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:24.122719] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:24.122755] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:24.122762] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:24.122770] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:47:24.122841] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:24.122883] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:47:24.175919] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:24.175954] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:24.203639] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:47:24.224788] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:47:24.257704] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:24.257735] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:24.257742] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:24.257748] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:47:24.257795] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:24.257816] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:25.720819] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:26.762157] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:47:26.772335] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f75c3c5047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f75c3f23182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f75c4a1d265]))) 0-: received signum (15), shutting down [2016-03-15 14:47:26.784759] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:47:26.788607] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:47:26.789114] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:26.789125] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:26.789474] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:47:26.789485] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:47:26.789490] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:47:26.789526] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:47:26.789577] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:26.789585] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:28.760209] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:47:28.765325] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:28.770073] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:28.776229] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:28.776269] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:28.776320] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:28.776327] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:28.776896] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:28.776939] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:28.776949] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:28.777208] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:28.777240] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:28.777246] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:47:28.783026] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:28.784075] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:28.784131] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:47:28.800689] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:28.866075] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:28.866157] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:47:28.884991] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:28.885025] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:28.905316] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:28.905345] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:47:28.925170] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:47:28.942782] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:28.942812] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:28.942819] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:28.942826] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:47:28.942869] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:29.541198] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:29.556131] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:29.556179] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:47:29.573741] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:29.573766] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:29.591576] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:29.604914] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:30.839438] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:31.848279] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:47:31.858889] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7faae307047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7faae3343182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7faae3e3d265]))) 0-: received signum (15), shutting down [2016-03-15 14:47:31.870905] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:47:31.875209] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:47:31.875937] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:31.875954] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:31.876487] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:47:31.876504] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:47:31.876514] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:47:31.876597] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:47:31.876675] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:31.876690] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:33.635827] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:47:33.640967] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:33.645360] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:33.650452] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:33.650504] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:33.650555] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:33.650563] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:33.651229] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:33.651268] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:33.651276] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:33.651503] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:33.651545] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:33.651556] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:47:33.657186] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:33.657712] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:33.657766] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:47:33.680200] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:33.680256] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:47:33.778660] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:33.778693] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:34.130588] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:47:34.272497] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:47:34.298778] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:34.298823] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:34.298839] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:34.298846] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:47:34.299577] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:34.356405] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:34.356470] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:47:34.413592] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:34.413639] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:34.436520] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:34.436552] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:34.695452] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:35.714116] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:36.933675] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:47:36.945374] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fbecf04a47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fbecf31d182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fbecfe17265]))) 0-: received signum (15), shutting down [2016-03-15 14:47:36.956727] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:47:36.960681] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:47:36.962152] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:36.962170] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:36.962688] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:47:36.962705] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:47:36.962716] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:47:36.962785] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:47:36.962860] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:36.962874] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:38.668402] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:47:38.673807] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:38.678529] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:38.685610] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:38.685651] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:38.685699] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:38.685706] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:38.686260] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:38.686295] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:38.686302] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:38.686527] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:38.686563] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:38.686571] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:47:38.706847] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:38.707844] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:38.707898] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:47:38.738651] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:38.738718] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:47:38.789813] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:38.789842] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:38.850052] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:38.917373] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:38.964920] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:47:38.985918] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:38.985952] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:38.985959] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:38.985966] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:47:39.013708] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:39.013777] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:47:39.054855] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:39.054892] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:39.090079] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:47:39.149753] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:39.149786] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:39.149793] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:39.149800] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:47:39.149856] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:39.149878] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:40.763772] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:42.017359] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:47:42.027653] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fc74bac647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fc74bd99182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fc74c893265]))) 0-: received signum (15), shutting down [2016-03-15 14:47:42.040249] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:47:42.042444] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:47:42.043267] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:42.043285] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:42.043915] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:47:42.043931] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:47:42.043941] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:47:42.043994] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:47:42.044073] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:42.044087] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:43.861005] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:47:43.868250] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:43.877698] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:43.882279] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:43.882320] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:43.882370] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:43.882378] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:43.882993] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:43.883037] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:43.883046] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:43.883304] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:43.883338] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:43.883354] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:47:43.891768] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:43.892647] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:43.892701] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:47:43.930887] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:43.982370] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:43.982433] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:47:44.100499] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:44.100533] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:44.279517] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:44.279558] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:44.279578] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:44.279585] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:44.279592] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:47:44.279631] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:47:44.301201] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:47:44.544147] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:44.568498] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:44.568546] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:47:44.592729] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:44.592757] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:44.650336] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:44.650372] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:45.948205] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:47.108246] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:47:47.118590] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ff6258f947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ff625bcc182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ff6266c6265]))) 0-: received signum (15), shutting down [2016-03-15 14:47:47.130315] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:47:47.134274] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:47:47.135207] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:47.135227] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:47.135725] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:47:47.135742] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:47:47.135752] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:47:47.135803] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:47:47.135879] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:47.135892] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:49.162851] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:47:49.176551] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:49.187425] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:49.192305] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:49.192347] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:49.192398] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:49.192405] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:49.192951] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:49.192987] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:49.192995] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:49.193224] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:49.193265] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:49.193277] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:47:49.200496] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:49.201178] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:49.201248] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:47:49.283319] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:49.283388] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:47:49.355016] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:49.355064] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:49.434001] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:49.434036] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:49.434044] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:49.434051] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:47:49.494739] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:47:49.570546] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:47:49.672749] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:49.672789] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:49.672807] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:49.672813] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:49.672820] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:47:49.673302] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:49.716298] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:49.716360] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:47:49.789159] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:49.789203] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:49.845440] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:49.845475] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:49.845738] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:51.257959] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:52.191215] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:47:52.203203] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f828251047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f82827e3182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f82832dd265]))) 0-: received signum (15), shutting down [2016-03-15 14:47:52.214646] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:47:52.218429] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:47:52.219488] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:52.219507] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:52.220014] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:47:52.220031] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:47:52.220042] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:47:52.220096] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:47:52.220172] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:52.220186] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:54.252747] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:47:54.258222] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:54.269913] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:54.275558] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:54.275601] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:54.275652] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:54.275660] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:54.276209] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:54.276243] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:54.276251] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:54.276477] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:54.276513] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:54.276521] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:47:54.283802] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:54.284732] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:54.284785] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:47:54.307664] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:54.327053] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:54.415775] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:54.415843] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:47:54.475767] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:54.475805] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:54.564524] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:47:54.660449] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:47:54.772789] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:54.772830] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:54.772838] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:54.772846] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:47:54.772896] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:54.772936] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:47:54.845203] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:54.845232] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:54.902434] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:54.902469] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:54.902505] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:54.902531] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:54.902542] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:54.902553] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:47:54.902612] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:54.902635] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:56.341461] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:57.278054] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:47:57.289371] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f3cf22c247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f3cf2595182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f3cf308f265]))) 0-: received signum (15), shutting down [2016-03-15 14:47:57.300973] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:47:57.304919] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:47:57.305542] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:57.305555] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:57.305906] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:47:57.305919] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:47:57.305925] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:47:57.305960] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:47:57.306010] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:57.306017] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:59.007704] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:47:59.011466] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:59.023353] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:59.037439] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:59.037486] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:59.037538] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:59.037545] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:59.038116] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:59.038161] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:59.038170] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:59.038394] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:59.038432] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:59.038441] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:47:59.042088] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:59.042900] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:59.042954] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:47:59.087426] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:59.418508] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:59.418573] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:47:59.465263] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:59.465297] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:59.489557] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:47:59.513739] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:47:59.560193] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:59.560233] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:59.560241] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:59.560248] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:47:59.560300] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:59.582420] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:59.583161] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:59.602479] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:59.602525] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:47:59.630028] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:59.630062] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:59.656248] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:59.656278] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:01.100000] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:02.363827] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:48:02.374175] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f3900be147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f3900eb4182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f39019ae265]))) 0-: received signum (15), shutting down [2016-03-15 14:48:02.385838] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:48:02.389918] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:48:02.390517] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:02.390530] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:02.390967] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:48:02.390979] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:48:02.390992] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:48:02.391031] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:48:02.391086] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:02.391094] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:04.267129] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:48:04.272523] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:04.275650] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:04.278850] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:04.278892] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:04.278941] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:04.278948] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:04.279926] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:04.279969] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:04.279978] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:04.280202] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:04.280240] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:04.280248] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:48:04.289666] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:04.290401] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:04.290455] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:48:04.322620] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:04.361391] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:04.361453] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:48:04.392525] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:04.392560] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:04.504086] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:48:04.572018] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:48:04.606054] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:04.606128] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:48:04.633881] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:04.633909] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:04.715277] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:04.715311] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:04.715339] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:04.715364] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:04.715371] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:04.715378] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:48:04.715430] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:04.715443] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:04.715449] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:04.715456] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:48:04.716166] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:06.346918] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:07.449515] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:48:07.460032] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ffbd726a47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ffbd753d182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ffbd8037265]))) 0-: received signum (15), shutting down [2016-03-15 14:48:07.471926] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:48:07.474031] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:48:07.474528] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:07.474540] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:07.474907] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:48:07.474919] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:48:07.474925] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:48:07.474960] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:48:07.475011] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:07.475019] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:09.417117] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:48:09.447460] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:09.482546] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:09.492705] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:09.492755] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:09.492806] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:09.492814] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:09.493346] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:09.493400] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:09.493410] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:09.493693] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:09.493740] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:09.493753] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:48:09.507291] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:09.508253] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:09.508309] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:48:09.557088] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:09.622527] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:09.622587] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:48:09.654722] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:09.654758] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:09.771041] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:09.771076] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:09.771084] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:09.771092] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:48:09.771136] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:48:09.852558] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:10.273994] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:48:10.338479] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:10.338514] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:10.338521] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:10.338528] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:48:10.370665] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:10.458717] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:10.458803] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:48:10.564027] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:10.564059] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:10.649795] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:10.649827] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:11.564442] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:12.542401] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:48:12.553295] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f803b97b47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f803bc4e182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f803c748265]))) 0-: received signum (15), shutting down [2016-03-15 14:48:12.564813] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:48:12.568664] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:48:12.569275] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:12.569289] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:12.569630] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:48:12.569642] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:48:12.569647] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:48:12.569682] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:48:12.569733] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:12.569740] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:14.700606] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:48:14.708723] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:14.720151] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:14.728189] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:14.728241] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:14.728291] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:14.728298] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:14.728859] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:14.728902] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:14.728912] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:14.729137] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:14.729175] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:14.729183] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:48:14.736474] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:14.737234] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:14.737287] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:48:14.765183] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:14.765241] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:48:14.797885] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:14.797920] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:14.830795] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:14.926581] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:48:14.996831] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:15.443723] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:15.443761] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:15.443769] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:15.443776] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:48:15.443822] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:48:15.476601] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:15.476636] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:15.476643] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:15.476650] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:48:15.476695] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:15.476735] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:48:15.513801] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:15.513828] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:15.563152] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:15.563185] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:16.794234] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:17.626528] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:48:17.637636] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fe92249847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fe92276b182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fe923265265]))) 0-: received signum (15), shutting down [2016-03-15 14:48:17.648750] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:48:17.652795] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:48:17.653289] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:17.653301] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:17.653644] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:48:17.653656] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:48:17.653662] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:48:17.653696] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:48:17.653779] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:17.653788] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:19.700571] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:48:19.705434] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:19.709122] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:19.712352] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:19.712392] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:19.712441] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:19.712448] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:19.713001] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:19.713046] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:19.713055] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:19.713291] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:19.713330] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:19.713339] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:48:19.717688] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:19.718315] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:19.718386] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:48:19.751380] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:19.929919] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:19.929985] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:48:19.962304] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:19.962331] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:19.986541] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:48:20.013589] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:48:20.060610] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:20.060656] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:20.060664] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:20.060671] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:48:20.060719] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:20.060735] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:20.551143] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:20.624690] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:20.624749] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:48:20.722909] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:20.722951] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:20.810807] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:20.810840] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:21.773870] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:22.713119] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:48:22.723460] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f096b1b647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f096b489182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f096bf83265]))) 0-: received signum (15), shutting down [2016-03-15 14:48:22.738221] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:48:22.741274] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:48:22.741996] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:22.742046] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:22.742568] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:48:22.742584] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:48:22.742594] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:48:22.742650] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:48:22.742726] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:22.743225] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:24.686850] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:48:24.695533] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:24.702289] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:24.714946] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:24.714995] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:24.715047] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:24.715054] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:24.715635] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:24.715670] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:24.715678] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:24.715919] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:24.715956] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:24.715964] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:48:24.736372] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:24.737436] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:24.737495] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:48:24.774016] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:24.774074] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:48:24.819010] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:24.819040] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:24.996741] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:48:25.109825] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:48:25.161620] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:25.161660] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:25.161668] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:25.161675] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:48:25.277431] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:25.371643] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:25.371703] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:48:25.468241] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:25.468274] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:25.529228] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:25.529266] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:25.704400] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:26.792729] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:27.805339] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:48:27.815832] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f01d2afa47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f01d2dcd182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f01d38c7265]))) 0-: received signum (15), shutting down [2016-03-15 14:48:27.827951] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:48:27.830935] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:48:27.831636] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:27.831653] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:27.832158] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:48:27.832175] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:48:27.832185] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:48:27.832238] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:48:27.832314] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:27.832328] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:30.267568] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:48:30.275870] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:30.281230] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:30.289545] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:30.289588] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:30.289638] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:30.289646] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:30.290245] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:30.290283] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:30.290291] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:30.290522] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:30.290567] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:30.290581] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:48:30.294662] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:30.295338] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:30.295393] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:48:30.314846] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:30.314903] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:48:30.351109] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:30.351169] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:30.373434] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:30.406737] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:30.429404] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:48:30.451008] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:30.451038] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:30.451045] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:30.451052] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:48:30.480450] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:48:30.569988] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:30.570061] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:48:30.590758] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:30.590777] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:30.618703] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:30.618743] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:30.618760] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:30.618766] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:30.618773] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:48:30.618832] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:32.352723] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:32.896375] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:48:32.907098] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f19afba647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f19afe79182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f19b0973265]))) 0-: received signum (15), shutting down [2016-03-15 14:48:32.918776] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:48:32.922469] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:48:32.924261] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:32.924279] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:32.924801] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:48:32.924818] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:48:32.924828] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:48:32.924881] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:48:32.924959] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:32.924974] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:34.840209] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:48:34.851232] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:34.866935] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:34.877338] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:34.877380] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:34.877433] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:34.877440] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:34.878029] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:34.878065] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:34.878073] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:34.878326] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:34.878363] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:34.878370] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:48:34.914107] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:34.914836] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:34.914894] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:48:35.019590] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:35.216349] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:48:35.340684] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:48:35.428276] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:35.428343] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:48:35.527835] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:35.527893] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:35.706948] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:35.706986] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:35.707014] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:35.707039] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:35.707047] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:35.707054] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:48:35.707876] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:35.752834] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:35.752894] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:48:35.827639] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:35.827674] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:35.897514] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:35.897552] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:36.970339] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:37.980943] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:48:37.992243] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f96be6ed47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f96be9c0182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f96bf4ba265]))) 0-: received signum (15), shutting down [2016-03-15 14:48:38.004697] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:48:38.008764] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:48:38.009396] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:38.009410] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:38.009739] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:48:38.009750] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:48:38.009756] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:48:38.009790] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:48:38.009841] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:38.009849] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:39.788621] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:48:39.792338] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:39.795738] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:39.798905] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:39.798945] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:39.798992] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:39.798999] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:39.799576] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:39.799617] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:39.799627] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:39.799888] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:39.799920] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:39.799927] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:48:39.804880] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:39.805566] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:39.805620] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:48:39.872429] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:48:39.893638] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:39.893698] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:48:39.939592] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:39.939621] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:40.097122] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:40.097171] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:40.097196] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:40.097203] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:40.097210] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:48:40.279834] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:48:40.372593] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:40.372625] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:40.372632] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:40.372639] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:48:40.397881] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:40.413869] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:40.413915] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:48:40.433445] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:40.433477] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:40.453098] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:40.501981] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:40.707427] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:41.861476] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:43.066192] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:48:43.076418] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f214ab0047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f214add3182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f214b8cd265]))) 0-: received signum (15), shutting down [2016-03-15 14:48:43.087850] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:48:43.090892] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:48:43.092027] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:43.092045] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:43.092547] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:48:43.092564] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:48:43.092574] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:48:43.092626] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:48:43.092703] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:43.092713] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:44.988340] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:48:45.029912] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:45.034522] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:45.039314] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:45.039374] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:45.039429] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:45.039436] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:45.040052] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:45.040113] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:45.040127] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:45.040384] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:45.040416] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:45.040423] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:48:45.049945] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:45.050367] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:45.050421] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:48:45.089503] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:45.373042] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:45.373123] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:48:45.434671] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:45.434700] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:45.486374] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:45.559075] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:45.559112] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:45.559120] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:45.559127] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:48:45.559173] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:48:45.601839] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:45.601876] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:45.601883] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:45.601900] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:48:45.601963] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:48:45.626759] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:45.626828] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:48:45.692049] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:45.692077] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:45.841388] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:45.841433] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:47.107180] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:48.149982] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:48:48.160666] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa2092a447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa209577182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa20a071265]))) 0-: received signum (15), shutting down [2016-03-15 14:48:48.172470] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:48:48.174676] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:48:48.175205] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:48.175217] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:48.175597] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:48:48.175608] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:48:48.175614] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:48:48.175650] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:48:48.175700] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:48.175708] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:50.054887] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:48:50.059143] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:50.062851] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:50.067125] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:50.067167] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:50.067217] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:50.067225] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:50.067796] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:50.067832] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:50.067849] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:50.068070] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:50.068137] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:50.068146] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:48:50.073703] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:50.074962] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:50.075037] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:48:50.093235] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:50.135854] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:50.135911] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:48:50.159525] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:50.159557] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:50.272976] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:48:50.300311] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:50.300345] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:50.300352] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:50.300359] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:48:50.300415] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:50.556634] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:48:50.660226] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:50.660269] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:50.660276] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:50.660283] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:48:50.660776] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:50.716197] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:50.716255] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:48:50.765241] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:50.765285] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:50.854398] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:50.854429] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:52.129963] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:53.238988] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:48:53.254610] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f2fe36aa47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f2fe397d182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f2fe4477265]))) 0-: received signum (15), shutting down [2016-03-15 14:48:53.266654] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:48:53.268843] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:48:53.269358] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:53.269370] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:53.269716] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:48:53.269727] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:48:53.269733] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:48:53.269770] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:48:53.269821] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:53.269830] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:55.139437] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:48:55.147145] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:55.151059] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:55.157055] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:55.157115] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:55.157188] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:55.157203] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:55.157881] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:55.157929] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:55.157942] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:55.158278] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:55.158343] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:55.158364] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:48:55.166237] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:55.167026] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:55.167110] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:48:55.197526] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:55.197580] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:48:55.231410] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:55.231447] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:55.324252] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:48:55.442863] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:48:55.497527] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:55.497581] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:55.497595] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:55.497603] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:48:55.497649] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:55.497666] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:55.497673] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:55.497679] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:48:55.498144] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:55.518258] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:55.518302] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:48:55.540266] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:55.540292] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:55.559925] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:55.570516] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:55.709753] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:57.222932] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:58.335270] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:48:58.346573] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ff2dc1c547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ff2dc498182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ff2dcf92265]))) 0-: received signum (15), shutting down [2016-03-15 14:48:58.358139] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:48:58.360708] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:48:58.361212] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:58.361224] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:58.361573] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:48:58.361584] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:48:58.361589] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:48:58.361625] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:48:58.361675] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:58.361682] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:00.286783] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:49:00.297753] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:00.301345] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:00.306895] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:00.306939] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:00.306988] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:00.306995] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:00.307521] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:00.307560] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:00.307568] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:00.307793] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:00.307824] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:00.307831] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:49:00.321165] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:00.322797] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:00.322873] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:49:00.379086] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:00.507018] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:00.507101] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:49:00.533304] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:00.533327] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:00.666936] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:00.666980] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:49:00.666991] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:49:00.667006] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:49:00.667071] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:49:00.711509] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:00.711546] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:49:01.284391] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:01.307677] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:01.307728] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:49:01.342075] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:01.342107] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:01.363704] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:01.409856] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:02.378995] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:03.423286] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:49:03.434057] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f073e50747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f073e7da182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f073f2d4265]))) 0-: received signum (15), shutting down [2016-03-15 14:49:03.446404] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:49:03.449686] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:49:03.450393] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:03.450410] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:03.451087] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:49:03.451111] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:49:03.451121] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:49:03.451176] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:49:03.451254] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:03.451269] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:05.524439] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:49:05.533996] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:05.544707] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:05.556124] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:05.556171] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:05.556222] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:05.556229] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:05.556792] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:05.556828] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:05.556835] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:05.557058] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:05.557102] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:05.557115] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:49:05.574569] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:05.575552] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:05.575635] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:49:05.609856] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:05.710967] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:05.790873] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:05.790953] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:49:05.813396] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:05.813426] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:05.837154] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:49:05.857235] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:05.857266] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:49:05.857273] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:49:05.857280] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:49:05.857326] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:49:05.877694] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:05.877721] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:49:05.877727] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:49:05.877733] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:49:05.877782] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:05.877805] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:05.877843] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:49:05.896300] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:05.896326] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:05.917489] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:05.917520] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:05.917535] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:07.632043] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:08.512103] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:49:08.522421] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fe65df0c47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fe65e1df182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fe65ecd9265]))) 0-: received signum (15), shutting down [2016-03-15 14:49:08.534876] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:49:08.537964] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:49:08.538673] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:08.538691] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:08.539408] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:49:08.539463] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:49:08.539473] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:49:08.539528] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:49:08.539608] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:08.539623] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:10.270190] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:49:10.273267] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:10.280497] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:10.285338] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:10.285380] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:10.285431] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:10.285439] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:10.286086] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:10.286129] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:10.286138] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:10.286392] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:10.286425] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:10.286431] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:49:10.290290] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:10.291490] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:10.291546] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:49:10.311944] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:10.497747] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:10.497829] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:49:10.590768] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:10.590814] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:10.666347] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:49:10.717282] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:49:10.786114] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:10.786151] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:49:10.786159] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:49:10.786166] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:49:10.786208] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:10.786246] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:49:10.814810] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:10.814843] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:10.870467] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:10.870506] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:10.870534] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:10.870553] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:49:10.870560] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:49:10.870567] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:49:10.870644] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:10.870659] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:10.871359] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:12.346690] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:13.599625] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:49:13.610014] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f417d55947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f417d82c182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f417e326265]))) 0-: received signum (15), shutting down [2016-03-15 14:49:13.622291] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:49:13.624647] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:49:13.625159] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:13.625170] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:13.625508] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:49:13.625519] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:49:13.625525] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:49:13.625561] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:49:13.625611] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:13.625619] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:15.419570] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:49:15.427842] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:15.431341] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:15.439760] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:15.439804] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:15.439852] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:15.439860] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:15.440440] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:15.440475] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:15.440483] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:15.440733] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:15.440771] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:15.440778] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:49:15.446929] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:15.447810] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:15.447883] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:49:15.469515] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:15.562433] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:49:15.867549] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:15.867630] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:49:15.939421] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:15.939458] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:16.022262] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:16.022309] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:16.022333] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:49:16.022340] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:49:16.022347] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:49:16.286941] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:49:16.348430] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:16.348461] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:49:16.348469] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:49:16.348476] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:49:16.360717] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:16.374892] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:16.374936] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:49:16.398636] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:16.398654] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:16.438736] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:16.460078] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:17.502600] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:18.688701] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:49:18.699960] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd951d9247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd952065182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd952b5f265]))) 0-: received signum (15), shutting down [2016-03-15 14:49:18.712413] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:49:18.716503] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:49:18.717137] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:18.717151] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:18.717496] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:49:18.717513] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:49:18.717519] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:49:18.717554] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:49:18.717604] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:18.717611] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:20.404246] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:49:20.472281] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:20.486317] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:20.492502] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:20.492548] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:20.492601] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:20.492609] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:20.493215] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:20.493250] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:20.493258] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:20.493508] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:20.493540] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:20.493547] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:49:20.500725] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:20.501420] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:20.501474] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:49:20.520007] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:20.520065] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:49:20.541582] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:20.541606] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:20.566260] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:20.644826] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:49:20.770686] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:49:20.797616] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:20.797653] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:49:20.797660] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:49:20.797667] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:49:20.797723] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:20.797741] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:49:20.797748] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:49:20.797754] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:49:20.798198] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:20.839186] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:20.839241] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:49:20.878335] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:20.878364] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:20.901762] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:20.901794] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:22.558006] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:23.773648] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:49:23.784012] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f5c6201847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f5c622eb182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f5c62de5265]))) 0-: received signum (15), shutting down [2016-03-15 14:49:23.795428] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:49:23.799308] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:49:23.799901] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:23.799914] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:23.800254] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:49:23.800265] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:49:23.800271] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:49:23.800306] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:49:23.800357] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:23.800365] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:25.578316] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:49:25.589560] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:25.597540] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:25.605734] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:25.605778] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:25.605826] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:25.605834] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:25.606413] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:25.606449] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:25.606457] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:25.606709] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:25.606763] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:25.606775] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:49:25.621707] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:25.623062] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:25.623119] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:49:25.813740] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:25.813808] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:49:25.833987] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:25.834022] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:25.857120] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:49:25.890762] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:25.890802] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:49:25.890810] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:49:25.890817] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:49:25.890861] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:49:25.912772] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:25.912804] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:25.913705] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:26.564313] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:26.585248] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:26.585302] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:49:26.608667] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:26.608695] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:26.672378] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:26.672424] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:27.679886] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:28.857973] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:49:28.868660] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd0b07c047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd0b0a93182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd0b158d265]))) 0-: received signum (15), shutting down [2016-03-15 14:49:28.880749] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:49:28.884917] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:49:28.885448] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:28.885460] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:28.885827] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:49:28.885841] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:49:28.885848] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:49:28.885884] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:49:28.885935] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:28.885943] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:30.672424] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:49:30.679077] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:30.683708] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:30.689550] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:30.689598] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:30.689651] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:30.689658] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:30.690250] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:30.690294] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:30.690304] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:30.690561] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:30.690593] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:30.690600] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:49:30.695689] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:30.696616] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:30.696670] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:49:30.846113] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:30.846185] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:49:30.874442] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:30.874478] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:30.917069] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:49:30.948703] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:30.948742] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:49:30.948750] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:49:30.948757] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:49:30.948813] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:49:30.988669] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:30.988708] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:31.290951] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:31.323792] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:31.323849] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:49:31.347139] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:31.347167] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:31.401176] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:31.401216] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:31.716097] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:32.752272] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:33.943617] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:49:33.954098] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ff8b480047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ff8b4ad3182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ff8b55cd265]))) 0-: received signum (15), shutting down [2016-03-15 14:49:33.966212] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:49:33.970412] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:49:33.971685] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:33.971705] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:33.972260] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:49:33.972273] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:49:33.972279] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:49:33.972314] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:49:33.972365] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:33.972373] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:35.755052] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:49:35.758802] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:35.769515] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:35.774460] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:35.774502] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:35.774552] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:35.774559] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:35.775151] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:35.775190] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:35.775197] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:35.775417] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:35.775462] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:35.775474] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:49:35.779489] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:35.780495] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:35.780549] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:49:35.795762] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:35.809253] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:35.849053] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:35.849114] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:49:35.881994] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:35.882030] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:36.010956] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:49:36.083990] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:36.084039] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:49:36.084047] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:49:36.084054] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:49:36.084113] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:49:36.131296] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:36.131356] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:49:36.166774] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:36.166804] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:36.204393] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:36.204432] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:36.204457] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:36.204476] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:49:36.204483] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:49:36.204498] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:49:36.224558] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:37.835879] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:39.028338] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:49:39.039627] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fdc4569347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fdc45966182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fdc46460265]))) 0-: received signum (15), shutting down [2016-03-15 14:49:39.051957] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:49:39.055422] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:49:39.056779] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:39.056797] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:39.057236] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:49:39.057249] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:49:39.057255] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:49:39.057293] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:49:39.057347] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:39.057355] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:40.876787] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:49:40.881176] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:40.885633] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:40.889666] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:40.889717] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:40.889779] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:40.889788] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:40.890390] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:40.890429] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:40.890437] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:40.890659] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:40.890723] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:40.890738] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:49:40.896844] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:40.897845] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:40.897900] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:49:41.062125] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:41.075686] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:49:41.112874] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:41.112943] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:49:41.173127] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:41.173158] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:41.191028] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:41.191078] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:41.191101] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:49:41.191108] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:49:41.191114] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:49:41.191159] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:49:41.213613] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:41.567575] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:41.587064] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:41.587122] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:49:41.607276] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:41.607305] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:41.625440] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:41.648435] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:42.954421] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:44.113759] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:49:44.124261] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fcc0848047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fcc08753182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fcc0924d265]))) 0-: received signum (15), shutting down [2016-03-15 14:49:44.135716] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:49:44.140174] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:49:44.140873] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:44.140890] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:44.141243] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:49:44.141254] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:49:44.141260] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:49:44.141294] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:49:44.141345] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:44.141353] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:45.961449] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:49:45.966044] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:45.970992] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:45.975030] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:45.975071] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:45.975118] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:45.975125] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:45.975697] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:45.975734] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:45.975742] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:45.975966] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:45.976004] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:45.976013] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:49:45.981970] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:45.983049] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:45.983107] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:49:46.023489] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:46.023545] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:49:46.259477] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:46.259526] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:46.658146] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:49:46.718933] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:49:46.803641] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:46.803678] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:49:46.803686] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:49:46.803693] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:49:46.803764] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:46.803777] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:49:46.803783] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:49:46.803794] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:49:46.804364] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:46.864321] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:46.916586] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:46.916644] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:49:46.990317] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:46.990357] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:47.022299] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:47.022329] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:48.038460] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:49.198396] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:49:49.208882] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f7b9ea5347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f7b9ed26182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f7b9f820265]))) 0-: received signum (15), shutting down [2016-03-15 14:49:49.220459] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:49:49.225744] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:49:49.226472] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:49.226500] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:49.227046] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:49:49.227064] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:49:49.227074] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:49:49.227117] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:49:49.227174] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:49.227183] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:50.968033] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:49:50.973310] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:50.976816] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:50.980590] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:50.980631] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:50.980681] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:50.980689] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:50.981216] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:50.981252] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:50.981259] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:50.981479] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:50.981518] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:50.981529] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:49:50.986228] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:50.987028] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:50.987083] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:49:51.018426] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:51.046464] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:51.213971] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:51.214069] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:49:51.282346] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:51.282385] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:51.357294] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:49:51.410671] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:49:51.461659] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:51.461696] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:49:51.461704] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:49:51.461711] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:49:51.461761] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:51.461802] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:49:51.482065] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:51.482090] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:51.506598] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:51.506629] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:51.506653] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:51.506669] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:49:51.506675] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:49:51.506682] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:49:51.506738] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:51.506764] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:53.042503] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:54.280590] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:49:54.295772] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f3df7aff47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f3df7dd2182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f3df88cc265]))) 0-: received signum (15), shutting down [2016-03-15 14:49:54.307531] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:49:54.312083] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:49:54.312788] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:54.312807] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:54.313292] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:49:54.313312] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:49:54.313319] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:49:54.313354] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:49:54.313405] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:54.313413] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:56.139357] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:49:56.144664] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:56.148752] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:56.151725] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:56.151765] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:56.151814] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:56.151821] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:56.152383] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:56.152419] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:56.152427] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:56.152673] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:56.152711] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:56.152719] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:49:56.168742] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:56.169736] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:56.169792] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:49:56.184208] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:56.252877] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:56.252939] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:49:56.289048] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:56.289082] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:56.330315] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:56.330350] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:49:56.330366] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:49:56.330374] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:49:56.330415] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:49:56.353670] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:49:56.377379] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:56.570147] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:56.621340] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:56.621399] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:49:56.696906] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:56.696942] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:56.777158] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:56.777193] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:58.224831] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:59.369312] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:49:59.380961] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f5599cf147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f5599fc4182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f559aabe265]))) 0-: received signum (15), shutting down [2016-03-15 14:49:59.392665] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:49:59.396626] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:49:59.397135] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:59.397147] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:59.397486] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:49:59.397498] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:49:59.397503] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:49:59.397538] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:49:59.397589] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:59.397597] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:50:01.252968] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:50:01.257023] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:01.260420] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:01.265601] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:01.265641] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:01.265691] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:01.265698] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:50:01.266286] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:01.266324] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:01.266332] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:50:01.266558] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:01.266595] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:01.266604] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:50:01.272927] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:50:01.274112] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:01.274164] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:50:01.435114] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:50:01.569737] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:01.569827] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:50:01.634314] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:50:01.634355] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:50:01.789453] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:01.789502] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:01.789527] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:50:01.789535] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:50:01.789542] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:50:01.789578] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:50:01.831708] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:01.831745] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:50:01.831763] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:50:01.831770] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:50:01.832719] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:01.885482] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:01.918551] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:01.918626] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:50:01.977762] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:50:01.977803] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:50:02.026056] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:02.026093] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:03.331441] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:50:04.456435] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:50:04.466778] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f73f2acf47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f73f2da2182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f73f389c265]))) 0-: received signum (15), shutting down [2016-03-15 14:50:04.478930] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:50:04.483731] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:50:04.484441] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:50:04.484458] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:50:04.484821] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:50:04.484833] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:50:04.484839] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:50:04.484873] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:50:04.484924] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:50:04.484932] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:50:06.410679] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:50:06.414184] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:06.417916] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:06.421578] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:06.421620] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:06.421670] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:06.421687] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:50:06.422255] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:06.422292] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:06.422300] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:50:06.422522] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:06.422593] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:06.422603] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:50:06.428138] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:50:06.429331] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:06.429386] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:50:06.442762] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:06.534831] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:06.534914] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:50:06.575868] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:50:06.575895] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:50:06.598547] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:50:06.670767] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:06.670803] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:06.670816] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:50:06.723843] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:06.723880] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:50:06.723887] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:50:06.723895] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:50:07.297319] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:07.328781] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:07.328849] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:50:07.361222] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:50:07.361258] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:50:07.397451] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:07.397485] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:08.485108] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:50:09.542388] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:50:09.553362] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f7a46ada47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f7a46dad182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f7a478a7265]))) 0-: received signum (15), shutting down [2016-03-15 14:50:09.565262] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:50:09.568809] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:50:09.569316] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:50:09.569328] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:50:09.569665] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:50:09.569676] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:50:09.569682] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:50:09.569717] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:50:09.569767] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:50:09.569775] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:50:11.475823] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:50:11.483302] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:11.497521] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:11.502794] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:11.502837] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:11.502887] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:11.502894] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:50:11.503438] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:11.503474] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:11.503483] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:50:11.503704] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:11.503745] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:11.503757] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:50:11.511192] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:50:11.512063] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:11.512135] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:50:11.536768] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:11.728925] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:50:11.762204] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:50:11.788030] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:11.788091] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:50:11.859041] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:50:11.859076] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:50:11.904450] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:11.904482] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:11.904507] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:11.904531] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:50:11.904538] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:50:11.904545] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:50:11.904620] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:11.904637] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:50:11.904647] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:50:11.904654] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:50:11.905432] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:11.925500] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:11.925544] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:50:11.951904] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:50:11.951930] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:50:11.991472] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:11.991501] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:13.568783] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:50:14.630909] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:50:14.641228] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fae6939147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fae69664182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fae6a15e265]))) 0-: received signum (15), shutting down [2016-03-15 14:50:14.653562] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:50:14.656635] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:50:14.657352] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:50:14.657369] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:50:14.657885] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:50:14.657903] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:50:14.657915] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:50:14.657970] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:50:14.658049] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:50:14.658063] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:50:16.582025] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:50:16.594287] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:16.602675] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:16.613257] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:16.613324] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:16.613379] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:16.613387] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:50:16.613950] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:16.613985] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:16.613993] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:50:16.614216] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:16.614261] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:16.614274] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:50:16.627681] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:50:16.628735] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:16.628794] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:50:16.894722] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:16.894827] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:50:16.966537] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:50:16.966567] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:50:17.143326] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:17.143424] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:50:17.168346] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:17.217688] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:50:17.295494] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:17.295530] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:50:17.295538] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:50:17.295545] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:50:17.321524] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:17.574317] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:17.621410] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:17.621467] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:50:17.648218] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:50:17.648247] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:50:17.677407] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:17.702821] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:18.684767] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:50:19.719716] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:50:19.731096] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f6c8fb8947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f6c8fe5c182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f6c90956265]))) 0-: received signum (15), shutting down [2016-03-15 14:50:19.742987] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:50:19.745211] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:50:19.745712] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:50:19.745723] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:50:19.746059] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:50:19.746070] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:50:19.746075] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:50:19.746111] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:50:19.746160] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:50:19.746168] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:50:21.494726] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:50:21.498829] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:21.501649] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:21.506029] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:21.506069] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:21.506116] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:21.506123] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:50:21.506661] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:21.506697] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:21.506704] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:50:21.506959] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:21.506997] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:21.507005] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:50:21.510137] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:50:21.510914] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:21.510966] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:50:21.614764] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:50:21.672591] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:21.672653] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:50:21.739913] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:50:21.739943] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:50:21.867038] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:21.867085] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:21.867109] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:50:21.867116] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:50:21.867123] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:50:21.867164] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:50:21.909372] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:22.300069] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:22.341000] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:22.341077] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:50:22.398806] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:50:22.398843] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:50:22.450973] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:22.451004] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:22.725020] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:23.567322] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:50:24.812208] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:50:24.823100] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f7f5c37047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f7f5c643182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f7f5d13d265]))) 0-: received signum (15), shutting down [2016-03-15 14:50:24.834721] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:50:24.838867] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:50:24.839443] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:50:24.839456] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:50:24.839804] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:50:24.839816] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:50:24.839823] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:50:24.839858] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:50:24.839912] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:50:24.839920] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:50:26.550912] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:50:26.554765] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:26.557682] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:26.561042] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:26.561083] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:26.561134] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:26.561142] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:50:26.561673] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:26.561707] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:26.561715] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:50:26.561934] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:26.561974] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:26.561987] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:50:26.566727] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:50:26.567911] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:26.567964] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:50:26.616356] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:26.656213] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:26.656279] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:50:26.680894] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:50:26.680919] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:50:26.786451] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:50:26.820236] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:26.820270] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:50:26.820278] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:50:26.820285] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:50:26.820328] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:50:26.914008] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:26.914048] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:26.914068] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:50:26.914075] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:50:26.914082] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:50:26.914794] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:26.979890] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:26.979952] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:50:27.080067] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:50:27.080107] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:50:27.113692] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:27.113721] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:28.623266] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:50:29.897365] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:50:29.907646] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f3bfe76447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f3bfea37182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f3bff531265]))) 0-: received signum (15), shutting down [2016-03-15 14:50:29.919349] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:50:29.922690] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:50:29.923397] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:50:29.923416] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:50:29.923971] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:50:29.923989] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:50:29.924000] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:50:29.924060] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:50:29.924134] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:50:29.924147] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:50:31.704537] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:50:31.708326] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:31.729537] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:31.733036] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:31.733079] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:31.733129] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:31.733137] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:50:31.733693] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:31.733741] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:31.733755] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:50:31.733996] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:31.734039] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:31.734052] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:50:31.744698] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:50:31.745414] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:31.745485] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:50:31.778572] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:31.836138] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:31.836201] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:50:31.881586] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:50:31.881616] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:50:31.911009] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:31.911041] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:50:31.911049] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:50:31.911065] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:50:31.933204] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:50:31.956488] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:32.570849] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:50:32.629612] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:32.650898] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:32.650949] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:50:32.699605] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:50:32.699635] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:50:32.722296] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:32.722326] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:33.800971] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:50:34.981864] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:50:34.992353] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f96c226247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f96c2535182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f96c302f265]))) 0-: received signum (15), shutting down [2016-03-15 14:50:35.004985] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:50:35.007969] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:50:35.008675] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:50:35.008692] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:50:35.009196] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:50:35.009214] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:50:35.009225] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:50:35.009278] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:50:35.009355] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:50:35.009369] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:50:36.742712] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:50:36.749639] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:36.753132] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:36.759046] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:36.759090] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:36.759150] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:36.759160] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:50:36.759725] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:36.759761] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:36.759769] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:50:36.760009] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:36.760047] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:36.760055] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:50:36.769109] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:50:36.770270] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:36.770324] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:50:36.852519] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:36.852584] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:50:36.893360] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:50:36.893392] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:50:36.935682] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:50:36.956287] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:36.956320] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:50:36.956327] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:50:36.956334] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:50:36.956387] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:50:36.992346] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:36.992373] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:37.303053] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:37.337625] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:37.337688] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:50:37.411659] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:50:37.411692] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:50:37.453978] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:37.454011] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:37.727761] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:38.825762] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:50:40.073439] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:50:40.085169] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f592961e47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f59298f1182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f592a3eb265]))) 0-: received signum (15), shutting down [2016-03-15 14:50:40.096900] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:50:40.100904] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:50:40.101420] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:50:40.101431] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:50:40.101774] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:50:40.101785] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:50:40.101790] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:50:40.101825] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:50:40.101876] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:50:40.101884] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:50:41.876510] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:50:41.880995] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:41.884718] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:41.888364] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:41.888405] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:41.888452] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:41.888459] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:50:41.889025] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:41.889061] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:41.889068] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:50:41.889317] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:41.889354] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:41.889361] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:50:41.893533] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:50:41.894785] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:41.894838] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:50:41.912885] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:41.951064] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:42.005177] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:42.005242] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:50:42.034444] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:50:42.034474] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:50:42.113144] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:50:42.153953] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:42.153992] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:50:42.154000] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:50:42.154006] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:50:42.154056] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:50:42.234590] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:42.234666] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:50:42.260111] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:50:42.260136] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:50:42.278921] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:42.278952] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:42.278977] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:42.278995] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:50:42.279010] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:50:42.279017] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:50:42.279058] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:43.949425] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:50:45.159027] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:50:45.169165] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fb3dba9247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fb3dbd65182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fb3dc85f265]))) 0-: received signum (15), shutting down [2016-03-15 14:50:45.181229] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:50:45.183417] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:50:45.183917] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:50:45.183928] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:50:45.184268] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:50:45.184280] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:50:45.184286] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:50:45.184321] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:50:45.184372] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:50:45.184380] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:50:46.983312] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:50:46.986792] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:46.995225] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:46.998480] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:46.998523] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:46.998574] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:46.998582] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:50:46.999173] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:46.999210] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:46.999218] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:50:46.999442] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:46.999482] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:46.999494] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:50:47.003993] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:50:47.005222] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:47.005287] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:50:47.023138] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:47.085101] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:47.085169] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:50:47.107233] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:50:47.107252] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:50:47.189337] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:50:47.224618] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:47.224672] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:50:47.224687] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:50:47.224696] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:50:47.224742] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:47.579456] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:50:47.699467] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:47.699501] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:50:47.699508] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:50:47.699515] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:50:47.700107] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:47.760424] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:47.760478] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:50:47.807818] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:50:47.807847] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:50:47.846541] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:47.846574] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:49.061618] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:50:50.246330] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:50:50.256797] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd9d594747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd9d5c1a182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd9d6714265]))) 0-: received signum (15), shutting down [2016-03-15 14:50:50.268055] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:50:50.272145] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:50:50.272733] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:50:50.272747] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:50:50.273084] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:50:50.273096] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:50:50.273101] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:50:50.273137] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:50:50.273187] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:50:50.273195] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:50:52.049867] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:50:52.063483] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:52.076823] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:52.099864] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:52.099914] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:52.099964] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:52.099971] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:50:52.100480] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:52.100518] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:52.100526] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:50:52.100746] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:52.100790] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:52.100804] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:50:52.110205] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:50:52.111302] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:52.111359] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:50:52.262160] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:52.262250] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:50:52.283527] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:50:52.283562] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:50:52.319498] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:50:52.352326] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:52.352364] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:50:52.352372] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:50:52.352379] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:50:52.352435] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:50:52.383715] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:52.383741] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:52.384371] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:52.408975] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:52.409022] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:50:52.473568] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:50:52.473597] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:50:52.493337] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:52.493360] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:52.730492] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:54.167190] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:50:55.329763] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:50:55.341705] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fb7824a147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fb782774182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fb78326e265]))) 0-: received signum (15), shutting down [2016-03-15 14:50:55.353169] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:50:55.356578] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:50:55.357079] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:50:55.357091] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:50:55.357431] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:50:55.357442] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:50:55.357447] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:50:55.357482] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:50:55.357534] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:50:55.357542] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:50:57.055263] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:50:57.065974] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:57.111175] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:57.131775] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:50:57.131826] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:57.131879] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:57.131887] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:50:57.132463] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:57.132500] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:57.132508] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:50:57.132736] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:50:57.132781] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:50:57.132794] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:50:57.137746] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:50:57.138524] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:57.138578] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:50:57.155981] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:57.189071] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:57.189140] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:50:57.209427] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:50:57.209460] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:50:57.290828] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:57.290862] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:50:57.290870] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:50:57.290877] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:50:57.290919] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:50:57.327040] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:57.435282] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:50:58.307044] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:50:58.407718] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:58.407781] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:50:58.467373] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:50:58.467398] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:50:58.491061] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:50:58.491091] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:50:59.194002] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:51:00.417263] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:51:00.429041] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f3ed3d3147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f3ed4004182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f3ed4afe265]))) 0-: received signum (15), shutting down [2016-03-15 14:51:00.440648] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:51:00.446152] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:51:00.447146] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:51:00.447160] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:51:00.447503] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:51:00.447514] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:51:00.447520] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:51:00.447556] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:51:00.447614] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:51:00.447623] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:51:02.557264] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:51:02.570126] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:02.581142] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:02.642406] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:02.642466] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:02.642519] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:02.642527] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:51:02.643174] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:02.643212] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:02.643220] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:51:02.643496] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:02.643529] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:02.643536] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:51:02.662534] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:51:02.663401] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:02.663456] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:51:02.745939] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:02.812960] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:03.072083] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:03.072150] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:51:03.095265] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:51:03.095300] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:51:03.118415] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:03.118476] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:51:03.160719] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:51:03.160760] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:51:03.262877] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:51:03.297419] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:51:03.320355] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:03.320388] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:51:03.320396] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:51:03.320403] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:51:03.320454] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:03.320471] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:51:03.320481] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:51:03.320488] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:51:03.320523] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:03.320543] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:03.320559] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:03.320571] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:04.719423] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:51:05.500609] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:51:05.511161] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f45db66e47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f45db941182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f45dc43b265]))) 0-: received signum (15), shutting down [2016-03-15 14:51:05.523101] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:51:05.527591] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:51:05.528238] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:51:05.528252] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:51:05.528598] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:51:05.528610] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:51:05.528616] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:51:05.528651] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:51:05.528702] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:51:05.528710] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:51:07.279493] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:51:07.283059] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:07.290989] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:07.295369] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:07.295411] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:07.295462] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:07.295470] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:51:07.296031] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:07.296068] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:07.296076] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:51:07.296328] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:07.296365] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:07.296372] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:51:07.300740] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:51:07.301714] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:07.301767] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:51:07.368956] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:51:07.410428] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:07.410467] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:51:07.410475] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:51:07.410482] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:51:07.511699] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:07.511760] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:51:07.530426] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:51:07.530453] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:51:07.552751] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:07.566161] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:51:07.587233] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:07.587281] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:07.587327] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:51:07.609468] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:51:07.609497] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:51:07.653300] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:07.653339] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:51:07.653346] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:51:07.653353] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:51:07.653404] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:07.732899] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:07.832037] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:07.880736] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:09.357559] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:51:10.584876] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:51:10.595590] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f75617e447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f7561ab7182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f75625b1265]))) 0-: received signum (15), shutting down [2016-03-15 14:51:10.607537] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:51:10.610530] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:51:10.611381] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:51:10.611400] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:51:10.611915] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:51:10.611933] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:51:10.611943] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:51:10.611995] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:51:10.612077] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:51:10.612092] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:51:12.592082] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:51:12.594946] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:12.598142] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:12.603222] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:12.603264] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:12.603313] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:12.603320] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:51:12.603889] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:12.603925] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:12.603932] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:51:12.604161] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:12.604203] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:12.604212] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:51:12.610972] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:51:12.612246] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:12.612298] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:51:12.632219] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:12.646504] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:12.646557] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:51:12.715768] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:51:12.715792] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:51:12.779651] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:51:12.881115] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:12.881152] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:51:12.881160] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:51:12.881167] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:51:13.309482] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:51:13.368349] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:13.368408] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:51:13.368416] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:51:13.368423] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:51:13.395169] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:13.432275] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:13.432328] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:51:13.508500] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:51:13.508530] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:51:13.567458] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:13.567490] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:14.667777] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:51:15.673669] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:51:15.683876] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f13acf6747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f13ad23a182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f13add34265]))) 0-: received signum (15), shutting down [2016-03-15 14:51:15.695700] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:51:15.697847] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:51:15.698350] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:51:15.698362] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:51:15.698699] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:51:15.698710] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:51:15.698716] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:51:15.698775] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:51:15.698830] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:51:15.698838] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:51:17.474765] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:51:17.499296] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:17.541080] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:17.570566] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:17.570625] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:17.570677] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:17.570696] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:51:17.571330] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:17.571368] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:17.571375] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:51:17.571633] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:17.571671] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:17.571678] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:51:17.601757] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:51:17.602462] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:17.602516] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:51:17.664569] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:17.664649] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:51:17.742427] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:51:17.742458] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:51:17.786490] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:17.871759] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:17.949165] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:51:17.993615] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:17.993657] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:51:17.993665] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:51:17.993672] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:51:17.993727] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:51:18.082515] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:18.082549] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:51:18.082557] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:51:18.082573] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:51:18.082624] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:18.082667] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:51:18.128653] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:51:18.128677] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:51:18.168072] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:18.168103] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:19.658363] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:51:20.761044] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:51:20.772575] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f1ded7a047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f1deda73182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f1dee56d265]))) 0-: received signum (15), shutting down [2016-03-15 14:51:20.784293] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:51:20.789126] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:51:20.789624] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:51:20.789658] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:51:20.790071] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:51:20.790084] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:51:20.790094] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:51:20.790144] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:51:20.790219] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:51:20.790231] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:51:22.959892] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:51:22.968743] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:22.977637] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:22.990260] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:22.990306] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:22.990356] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:22.990363] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:51:22.990926] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:22.990963] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:22.990971] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:51:22.991192] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:22.991247] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:22.991261] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:51:23.000459] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:51:23.001560] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:23.001618] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:51:23.063265] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:23.208397] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:51:23.245298] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:23.245367] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:51:23.299051] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:51:23.299088] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:51:23.348000] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:23.348054] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:23.348076] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:51:23.348083] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:51:23.348089] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:51:23.348130] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:23.348150] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:51:23.587200] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:23.651498] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:23.651555] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:51:23.744527] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:51:23.744567] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:51:23.861284] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:23.861325] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:25.057215] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:51:25.847171] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:51:25.857478] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd4d9f8847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd4da25b182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd4dad55265]))) 0-: received signum (15), shutting down [2016-03-15 14:51:25.868914] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:51:25.872891] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:51:25.873429] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:51:25.873441] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:51:25.873783] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:51:25.873794] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:51:25.873800] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:51:25.873834] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:51:25.873885] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:51:25.873893] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:51:27.739389] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:51:27.744610] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:27.747872] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:27.751572] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:27.751613] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:27.751660] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:27.751668] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:51:27.752181] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:27.752216] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:27.752224] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:51:27.752438] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:27.752483] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:27.752496] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:51:27.756755] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:51:27.757837] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:27.757892] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:51:27.815229] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:51:27.850228] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:27.850292] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:51:27.870021] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:51:27.870048] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:51:27.909155] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:27.909192] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:27.909213] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:51:27.909220] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:51:27.909227] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:51:28.257792] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:51:28.368091] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:28.398826] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:28.398879] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:51:28.581078] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:51:28.581121] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:51:28.612570] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:28.612599] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:28.736704] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:29.813386] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:51:30.932304] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:51:30.942588] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fbcc117547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fbcc1448182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fbcc1f42265]))) 0-: received signum (15), shutting down [2016-03-15 14:51:30.953832] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:51:30.957925] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:51:30.958442] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:51:30.958454] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:51:30.959238] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:51:30.959257] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:51:30.959268] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:51:30.959399] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:51:30.959621] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:51:30.959637] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:51:32.703794] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:51:32.707828] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:32.710964] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:32.715432] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:32.715471] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:32.715520] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:32.715527] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:51:32.716087] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:32.716122] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:32.716129] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:51:32.716353] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:32.716397] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:32.716410] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:51:32.721025] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:51:32.722086] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:32.722160] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:51:32.742816] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:32.758253] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:32.798254] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:32.798320] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:51:32.818453] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:51:32.818480] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:51:32.888999] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:51:32.927699] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:32.927734] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:51:32.927741] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:51:32.927748] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:51:32.927801] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:51:32.949926] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:32.949985] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:51:32.973599] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:51:32.973626] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:51:33.003251] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:33.003282] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:33.003306] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:33.003325] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:51:33.003331] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:51:33.003338] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:51:33.003383] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:34.777851] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:51:36.016468] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:51:36.026683] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fed2d5b247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fed2d885182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fed2e37f265]))) 0-: received signum (15), shutting down [2016-03-15 14:51:36.038268] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:51:36.042354] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:51:36.043183] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:51:36.043203] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:51:36.043730] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:51:36.043747] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:51:36.043757] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:51:36.043811] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:51:36.043892] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:51:36.043906] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:51:37.842467] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:51:37.845770] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:37.848981] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:37.852253] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:37.852292] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:37.852339] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:37.852346] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:51:37.852835] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:37.852873] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:37.852881] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:51:37.853105] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:37.853142] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:37.853155] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:51:37.857311] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:51:37.858378] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:37.858431] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:51:37.874624] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:37.916912] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:37.916983] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:51:37.957427] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:51:37.957460] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:51:38.026500] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:38.026549] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:51:38.026557] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:51:38.026565] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:51:38.026623] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:51:38.063357] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:38.589481] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:51:38.660332] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:38.660364] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:51:38.660372] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:51:38.660379] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:51:38.660868] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:38.694951] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:38.695002] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:51:38.759871] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:51:38.759901] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:51:38.861336] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:38.861371] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:39.913326] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:51:41.100431] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:51:41.111757] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fc799ef847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fc79a1cb182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fc79acc5265]))) 0-: received signum (15), shutting down [2016-03-15 14:51:41.123583] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:51:41.125728] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:51:41.126224] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:51:41.126236] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:51:41.126576] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:51:41.126588] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:51:41.126593] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:51:41.126637] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:51:41.126688] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:51:41.126697] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:51:42.812646] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:51:42.815961] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:42.819298] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:42.822503] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:42.822542] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:42.822590] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:42.822598] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:51:42.823155] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:42.823191] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:42.823199] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:51:42.823414] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:42.823451] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:42.823459] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:51:42.828218] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:51:42.829400] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:42.829452] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:51:42.971178] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:51:42.995292] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:42.995355] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:51:43.046130] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:51:43.046166] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:51:43.072230] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:43.072267] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:43.072288] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:51:43.072303] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:51:43.072311] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:51:43.072356] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:51:43.092243] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:43.315874] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:43.369451] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:43.369513] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:51:43.414528] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:51:43.414557] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:51:43.455838] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:43.455869] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:43.739244] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:44.883946] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:51:46.190501] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:51:46.201018] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fab7507947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fab7534c182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fab75e46265]))) 0-: received signum (15), shutting down [2016-03-15 14:51:46.212768] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:51:46.216252] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:51:46.216759] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:51:46.216770] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:51:46.217121] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:51:46.217132] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:51:46.217138] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:51:46.217174] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:51:46.217241] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:51:46.218512] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:51:48.339817] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:51:48.345598] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:48.350190] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:48.353331] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:48.353403] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:48.353455] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:48.353463] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:51:48.354096] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:48.354135] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:48.354143] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:51:48.354409] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:48.354449] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:48.354458] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:51:48.360042] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:51:48.360961] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:48.361015] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:51:48.374843] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:48.374903] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:51:48.393321] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:51:48.393340] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:51:48.412602] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:48.457706] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:48.480015] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:51:48.506269] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:48.506302] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:51:48.506309] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:51:48.506316] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:51:48.550067] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:48.550140] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:51:48.613456] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:51:48.613502] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:51:48.655721] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:51:48.682403] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:48.682460] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:48.682480] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:51:48.682502] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:51:48.682516] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:51:50.416574] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:51:51.261308] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:51:51.291723] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:51:51.302560] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fed13e1447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fed140e7182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fed14be1265]))) 0-: received signum (15), shutting down [2016-03-15 14:51:51.314916] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:51:51.317111] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:51:51.317607] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:51:51.317620] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:51:51.317960] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:51:51.317971] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:51:51.317977] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:51:51.318012] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:51:51.318063] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:51:51.318071] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:51:53.014466] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:51:53.017700] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:53.021706] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:53.025127] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:53.025167] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:53.025216] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:53.025223] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:51:53.025719] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:53.025756] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:53.025772] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:51:53.025994] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:53.026037] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:53.026051] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:51:53.041857] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:51:53.043210] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:53.043265] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:51:53.058027] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:53.101526] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:51:53.278752] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:53.278818] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:51:53.381860] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:51:53.381898] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:51:53.448682] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:53.448713] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:51:53.508343] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:53.508381] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:51:53.508389] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:51:53.508396] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:51:53.508444] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:53.593183] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:53.645734] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:53.645789] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:51:53.680325] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:51:53.680365] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:51:53.702214] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:53.704361] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:55.098786] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:51:56.388977] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:51:56.402928] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f847982e47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f8479b01182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f847a5fb265]))) 0-: received signum (15), shutting down [2016-03-15 14:51:56.415383] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:51:56.417559] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:51:56.418059] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:51:56.418071] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:51:56.418834] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:51:56.418848] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:51:56.418854] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:51:56.418891] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:51:56.418952] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:51:56.419919] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:51:58.162363] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:51:58.179030] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:58.213253] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:58.222828] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:51:58.222876] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:58.222928] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:58.222936] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:51:58.223527] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:58.223571] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:58.223580] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:51:58.223805] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:51:58.223867] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:51:58.223880] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:51:58.234673] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:51:58.235267] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:51:58.236672] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:58.236719] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:51:58.526894] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:51:58.575780] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:51:58.595936] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:58.595988] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:51:58.650684] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:51:58.650714] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:51:58.676338] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:58.676362] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:58.676385] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:58.676406] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:51:58.676412] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:51:58.676419] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:51:58.676497] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:58.676511] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:51:58.676517] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:51:58.676524] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:51:58.677159] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:58.701116] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:51:58.701173] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:51:58.731647] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:51:58.731673] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:51:58.750887] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:51:58.751617] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:51:58.776545] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:00.292494] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:52:01.485468] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:52:01.496990] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f25cea1e47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f25cecf1182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f25cf7eb265]))) 0-: received signum (15), shutting down [2016-03-15 14:52:01.509026] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:52:01.511285] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:52:01.511796] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:52:01.511807] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:52:01.512152] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:52:01.512163] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:52:01.512169] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:52:01.512205] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:52:01.512271] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:52:01.512285] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:52:03.343063] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:52:03.348752] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:03.352919] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:03.357458] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:03.357502] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:03.357555] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:03.357563] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:52:03.358123] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:03.358166] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:03.358175] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:52:03.358396] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:03.358437] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:03.358454] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:52:03.362643] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:52:03.363516] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:03.363570] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:52:03.391422] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:03.497364] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:03.497441] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:52:03.516262] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:52:03.516297] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:52:03.536652] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:52:03.553370] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:03.553409] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:52:03.553417] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:52:03.553424] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:52:03.553474] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:52:03.576033] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:03.576061] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:04.321490] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:04.337097] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:04.337146] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:52:04.362988] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:52:04.363017] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:52:04.382836] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:04.405104] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:05.421421] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:52:06.573245] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:52:06.583865] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f8db265c47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f8db292f182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f8db3429265]))) 0-: received signum (15), shutting down [2016-03-15 14:52:06.595243] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:52:06.599284] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:52:06.599882] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:52:06.599896] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:52:06.600237] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:52:06.600248] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:52:06.600254] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:52:06.600288] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:52:06.600338] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:52:06.600346] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:52:08.296486] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:52:08.299799] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:08.303368] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:08.307162] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:08.307203] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:08.307251] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:08.307258] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:52:08.307819] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:08.307861] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:08.307871] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:52:08.308126] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:08.308158] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:08.308165] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:52:08.324954] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:52:08.325681] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:08.325736] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:52:08.364735] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:08.364790] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:52:08.389179] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:52:08.389205] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:52:08.423669] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:08.520628] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:52:08.543955] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:08.543996] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:52:08.544003] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:52:08.544010] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:52:08.544067] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:52:08.596493] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:08.610359] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:08.610418] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:52:08.637335] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:52:08.637381] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:52:08.658093] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:08.658124] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:10.382202] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:52:11.659865] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:52:11.670053] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f8d620e047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f8d623b3182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f8d62ead265]))) 0-: received signum (15), shutting down [2016-03-15 14:52:11.681293] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:52:11.684789] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:52:11.685299] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:52:11.685311] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:52:11.685656] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:52:11.685668] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:52:11.685673] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:52:11.685709] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:52:11.685767] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:52:11.685775] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:52:13.442362] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:52:13.446526] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:13.450009] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:13.453480] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:13.453520] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:13.453567] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:13.453574] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:52:13.454152] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:13.454189] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:13.454196] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:52:13.454430] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:13.454471] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:13.454479] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:52:13.458695] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:52:13.459811] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:13.459865] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:52:13.570538] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:13.570601] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:52:13.609925] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:52:13.609952] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:52:13.648334] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:52:13.666844] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:13.666877] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:52:13.666885] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:52:13.666892] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:52:13.666943] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:52:13.685797] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:13.685821] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:14.148199] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:14.597522] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:14.618712] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:14.618772] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:52:14.642694] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:52:14.642722] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:52:14.677518] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:14.694185] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:15.514979] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:52:16.748209] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:52:16.758629] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f9f4eb9747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f9f4ee6a182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f9f4f964265]))) 0-: received signum (15), shutting down [2016-03-15 14:52:16.769811] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:52:16.773795] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:52:16.774396] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:52:16.774409] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:52:16.774792] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:52:16.774818] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:52:16.774824] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:52:16.774861] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:52:16.774912] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:52:16.774920] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:52:18.383669] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:52:18.390440] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:18.394936] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:18.399050] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:18.399091] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:18.399151] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:18.399159] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:52:18.399733] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:18.399775] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:18.399785] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:52:18.400043] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:18.400075] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:18.400082] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:52:18.405162] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:52:18.405953] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:18.406005] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:52:18.421529] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:18.463606] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:18.463665] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:52:18.483256] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:52:18.483279] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:52:18.528054] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:18.528088] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:52:18.528096] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:52:18.528103] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:52:18.613663] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:52:18.758593] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:19.323504] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:52:19.402530] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:19.402577] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:52:19.402599] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:52:19.402606] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:52:19.421930] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:19.495962] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:19.496019] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:52:19.515504] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:52:19.515529] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:52:19.536603] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:19.536634] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:20.461867] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:52:21.833793] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:52:21.845055] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f80930d447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f80933a7182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f8093ea1265]))) 0-: received signum (15), shutting down [2016-03-15 14:52:21.856541] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:52:21.860636] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:52:21.861177] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:52:21.861190] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:52:21.861529] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:52:21.861541] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:52:21.861547] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:52:21.861582] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:52:21.861633] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:52:21.861646] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:52:23.504925] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:52:23.508847] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:23.512184] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:23.515213] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:23.515253] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:23.515303] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:23.515310] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:52:23.515879] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:23.515932] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:23.515942] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:52:23.516204] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:23.516241] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:23.516248] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:52:23.519798] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:52:23.520796] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:23.520867] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:52:23.534620] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:23.611959] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:23.612023] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:52:23.628997] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:52:23.629023] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:52:23.727238] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:52:23.748058] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:23.748091] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:52:23.748098] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:52:23.748106] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:52:23.803037] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:23.803067] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:52:23.844517] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:23.844548] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:52:23.844556] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:52:23.844563] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:52:23.845351] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:23.861635] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:23.861678] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:52:23.912959] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:52:23.912993] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:52:23.939897] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:23.939930] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:25.576145] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:52:26.917324] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:52:26.927785] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f081027747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f081054a182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f0811044265]))) 0-: received signum (15), shutting down [2016-03-15 14:52:26.939860] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:52:26.943923] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:52:26.944996] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:52:26.945009] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:52:26.945351] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:52:26.945363] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:52:26.945369] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:52:26.945405] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:52:26.945456] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:52:26.945464] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:52:28.845005] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:52:28.849064] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:28.874890] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:28.877795] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:28.877837] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:28.877887] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:28.877894] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:52:28.878440] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:28.878475] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:28.878483] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:52:28.878697] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:28.878745] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:28.878773] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:52:28.883851] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:52:28.884893] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:28.884944] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:52:28.949501] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:28.949562] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:52:28.969015] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:52:28.969041] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:52:29.009011] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:29.009046] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:52:29.028185] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:52:29.044460] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:29.044493] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:29.044514] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:52:29.044521] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:52:29.044528] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:52:29.150864] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:29.600498] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:29.616029] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:29.616074] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:52:29.652672] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:52:29.652699] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:52:29.671278] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:29.671313] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:30.940528] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:52:32.004114] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:52:32.014681] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f5f3f24547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f5f3f518182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f5f40012265]))) 0-: received signum (15), shutting down [2016-03-15 14:52:32.026283] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:52:32.030240] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:52:32.031177] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:52:32.031196] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:52:32.031704] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:52:32.031721] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:52:32.031730] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:52:32.031781] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:52:32.031858] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:52:32.031872] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:52:33.830599] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:52:33.834457] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:33.837304] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:33.840791] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:33.840853] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:33.840904] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:33.840912] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:52:33.841420] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:33.841457] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:33.841466] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:52:33.841681] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:33.841717] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:33.841725] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:52:33.846252] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:52:33.847141] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:33.847193] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:52:33.865120] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:33.910943] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:52:33.935281] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:33.935339] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:52:33.954318] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:52:33.954352] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:52:33.998248] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:33.998296] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:33.998320] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:52:33.998327] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:52:33.998334] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:52:34.049752] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:52:34.326809] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:34.383707] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:34.383771] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:52:34.438069] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:52:34.438100] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:52:34.485715] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:34.512255] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:35.902579] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:52:37.088675] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:52:37.098948] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f372c67547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f372c948182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f372d442265]))) 0-: received signum (15), shutting down [2016-03-15 14:52:37.110406] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:52:37.114496] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:52:37.115690] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:52:37.115709] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:52:37.116217] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:52:37.116234] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:52:37.116244] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:52:37.116294] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:52:37.116350] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:52:37.116358] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:52:38.834367] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:52:38.850784] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:38.862167] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:38.869906] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:38.869951] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:38.870001] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:38.870009] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:52:38.870577] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:38.870614] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:38.870622] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:52:38.870905] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:38.870942] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:38.870950] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:52:38.881355] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:52:38.882393] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:38.882448] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:52:38.911363] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:38.982942] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:38.983005] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:52:39.034562] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:52:39.034594] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:52:39.069038] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:39.163856] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:39.163894] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:52:39.163902] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:52:39.163909] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:52:39.163953] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:39.163991] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:52:39.183392] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:52:39.183425] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:52:39.204399] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:52:39.221352] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:52:39.299520] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:39.299557] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:52:39.299564] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:52:39.299571] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:52:39.299619] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:39.299639] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:40.936927] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:52:42.173141] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:52:42.184390] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f2dc76ce47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f2dc79a1182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f2dc849b265]))) 0-: received signum (15), shutting down [2016-03-15 14:52:42.196567] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:52:42.199666] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:52:42.200365] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:52:42.200382] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:52:42.200888] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:52:42.200906] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:52:42.200931] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:52:42.200985] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:52:42.201062] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:52:42.201077] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:52:44.042345] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:52:44.049240] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:44.060177] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:44.076034] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:44.076088] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:44.076139] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:44.076146] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:52:44.076721] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:44.076765] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:44.076775] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:52:44.076991] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:44.077065] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:44.077073] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:52:44.086786] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:52:44.087821] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:44.087874] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:52:44.200192] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:52:44.391167] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:44.391238] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:52:44.566905] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:52:44.566936] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:52:44.746466] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:44.746506] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:44.746536] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:52:44.746544] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:52:44.746551] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:52:44.746593] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:52:44.792429] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:44.792466] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:52:44.792473] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:52:44.792480] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:52:44.792996] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:44.808728] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:44.823430] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:44.823482] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:52:44.841564] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:52:44.841587] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:52:44.858801] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:44.858838] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:46.143508] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:52:47.264265] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:52:47.274833] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f880c05a47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f880c32d182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f880ce27265]))) 0-: received signum (15), shutting down [2016-03-15 14:52:47.287465] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:52:47.291969] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:52:47.292654] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:52:47.292672] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:52:47.293244] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:52:47.293262] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:52:47.293272] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:52:47.293324] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:52:47.293401] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:52:47.293426] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:52:49.005785] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:52:49.011706] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:49.020521] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:49.032017] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:49.032062] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:49.032111] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:49.032118] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:52:49.032651] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:49.032686] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:49.032694] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:52:49.032925] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:49.032961] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:49.032968] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:52:49.039161] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:52:49.040119] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:49.040170] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:52:49.193496] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:49.193573] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:52:49.228195] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:52:49.228228] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:52:49.270137] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:52:49.289304] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:49.289336] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:52:49.289344] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:52:49.289351] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:52:49.289414] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:52:49.342824] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:49.342861] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:49.343557] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:49.390330] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:49.390383] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:52:49.485009] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:52:49.485040] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:52:49.546843] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:49.546884] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:50.154333] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:51.095960] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:52:52.348776] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:52:52.359267] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd44bcbf47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd44bf92182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd44ca8c265]))) 0-: received signum (15), shutting down [2016-03-15 14:52:52.370400] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:52:52.374337] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:52:52.375056] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:52:52.375075] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:52:52.375586] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:52:52.375602] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:52:52.375610] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:52:52.375659] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:52:52.375730] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:52:52.375743] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:52:54.127637] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:52:54.132256] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:54.135447] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:54.138486] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:54.138526] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:54.138576] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:54.138592] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:52:54.139169] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:54.139205] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:54.139212] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:52:54.139439] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:54.139482] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:54.139496] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:52:54.147596] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:52:54.148555] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:54.148606] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:52:54.165573] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:54.232536] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:52:54.250928] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:54.250987] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:52:54.270556] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:52:54.270580] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:52:54.306183] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:54.306224] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:54.306247] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:52:54.306253] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:52:54.306260] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:52:54.597592] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:52:55.330894] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:55.391286] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:55.391351] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:52:55.411773] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:52:55.411799] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:52:55.470223] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:52:55.470263] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:56.204247] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:52:57.432146] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:52:57.445356] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f7655cf647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f7655fc9182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f7656ac3265]))) 0-: received signum (15), shutting down [2016-03-15 14:52:57.458694] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:52:57.461008] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:52:57.461510] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:52:57.461522] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:52:57.461858] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:52:57.461868] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:52:57.461874] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:52:57.461909] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:52:57.461959] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:52:57.461967] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:52:59.687187] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:52:59.753793] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:59.764197] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:59.776703] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:52:59.776752] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:59.776803] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:59.776811] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:52:59.777323] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:59.777361] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:59.777369] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:52:59.777592] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:52:59.777629] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:52:59.777636] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:52:59.790630] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:52:59.792116] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:59.792173] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:52:59.825308] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:52:59.872076] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:59.872159] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:52:59.891963] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:52:59.891992] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:52:59.928931] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:52:59.948176] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:52:59.948211] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:52:59.948219] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:52:59.948225] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:53:00.002822] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:00.016438] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:00.016572] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:53:00.037326] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:00.037362] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:53:00.037369] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:53:00.037376] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:53:00.037453] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:00.037487] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:53:00.057976] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:53:00.058006] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:53:00.083338] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:00.083373] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:01.848912] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:53:02.526803] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:53:02.547274] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fe3d180847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fe3d1adb182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fe3d25d5265]))) 0-: received signum (15), shutting down [2016-03-15 14:53:02.559037] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:53:02.563830] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:53:02.564541] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:53:02.564558] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:53:02.565006] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:53:02.565018] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:53:02.565024] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:53:02.565059] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:53:02.565110] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:53:02.565118] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:53:04.602365] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:53:04.615933] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:04.631724] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:04.637476] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:04.637525] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:04.637577] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:04.637585] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:53:04.638101] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:04.638136] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:04.638144] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:53:04.638367] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:04.638404] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:04.638412] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:53:04.650964] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:53:04.651669] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:04.651725] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:53:04.677646] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:04.749927] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:04.749993] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:53:04.768300] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:53:04.768328] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:53:04.787510] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:53:04.811832] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:53:04.835994] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:04.836038] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:53:04.836046] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:53:04.836053] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:53:04.836117] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:04.836159] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:53:04.861443] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:53:04.861470] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:53:04.921949] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:04.921982] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:04.922008] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:04.922027] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:53:04.922033] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:53:04.922040] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:53:04.922080] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:04.922100] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:05.157049] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:06.706925] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:53:07.622347] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:53:07.633073] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f023770047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f02379d3182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f02384cd265]))) 0-: received signum (15), shutting down [2016-03-15 14:53:07.645302] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:53:07.648796] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:53:07.649308] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:53:07.649319] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:53:07.649665] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:53:07.649676] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:53:07.649682] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:53:07.649718] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:53:07.649770] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:53:07.649778] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:53:09.520258] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:53:09.529565] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:09.538553] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:09.548865] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:09.548916] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:09.548966] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:09.548973] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:53:09.549487] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:09.549533] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:09.549543] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:53:09.549765] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:09.549804] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:09.549813] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:53:09.563689] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:53:09.564792] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:09.564848] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:53:09.594058] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:09.652263] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:53:09.671791] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:09.671849] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:53:09.691092] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:53:09.691119] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:53:09.758248] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:09.758286] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:09.758309] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:53:09.758316] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:53:09.758323] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:53:10.333209] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:53:10.392145] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:10.392191] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:53:10.392199] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:53:10.392206] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:53:10.409171] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:10.462285] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:10.462349] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:53:10.564074] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:53:10.564110] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:53:10.613272] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:10.613301] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:11.620303] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:53:12.713535] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:53:12.723912] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f5847c1147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f5847ee4182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f58489de265]))) 0-: received signum (15), shutting down [2016-03-15 14:53:12.735113] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:53:12.738298] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:53:12.738863] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:53:12.738881] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:53:12.739405] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:53:12.739422] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:53:12.739431] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:53:12.739483] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:53:12.739561] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:53:12.739575] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:53:14.578721] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:53:14.582134] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:14.585764] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:14.589258] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:14.589299] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:14.589349] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:14.589357] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:53:14.589933] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:14.589969] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:14.589977] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:53:14.590202] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:14.590248] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:14.590262] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:53:14.596717] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:53:14.597424] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:14.597476] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:53:14.616312] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:14.697854] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:14.697917] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:53:14.729785] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:53:14.729821] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:53:14.771123] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:14.771156] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:53:14.771164] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:53:14.771171] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:53:14.771205] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:53:14.795791] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:14.795815] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:53:14.795822] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:53:14.795829] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:53:14.795867] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:53:14.818709] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:14.833431] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:14.833801] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:14.833843] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:53:14.863393] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:53:14.863419] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:53:14.911484] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:14.911518] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:16.653462] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:53:17.801332] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:53:17.811559] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fc8d42df47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fc8d45b2182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fc8d50ac265]))) 0-: received signum (15), shutting down [2016-03-15 14:53:17.824054] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:53:17.827229] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:53:17.828064] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:53:17.828119] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:53:17.828683] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:53:17.828700] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:53:17.828711] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:53:17.828765] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:53:17.828843] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:53:17.828896] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:53:19.674112] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:53:19.680339] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:19.704915] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:19.708753] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:19.708797] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:19.708850] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:19.708858] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:53:19.709458] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:19.709501] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:19.709510] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:53:19.709734] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:19.709781] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:19.709792] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:53:19.714390] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:53:19.715126] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:19.715181] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:53:19.822231] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:19.822301] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:53:19.842098] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:53:19.842138] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:53:19.867837] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:53:19.901963] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:19.901999] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:53:19.902007] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:53:19.902014] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:53:19.924004] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:20.159599] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:20.610132] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:53:20.644563] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:20.644597] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:53:20.644605] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:53:20.644612] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:53:20.656307] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:20.688468] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:20.688526] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:53:20.794868] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:53:20.794916] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:53:20.842763] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:20.842800] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:21.772020] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:53:22.890061] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:53:22.901570] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd4c561d47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd4c58f0182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd4c63ea265]))) 0-: received signum (15), shutting down [2016-03-15 14:53:22.913351] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:53:22.916313] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:53:22.917006] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:53:22.917034] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:53:22.917541] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:53:22.917559] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:53:22.917570] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:53:22.917717] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:53:22.917796] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:53:22.917811] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:53:25.514981] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:53:25.519678] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:25.528533] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:25.539690] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:25.539736] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:25.539786] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:25.539793] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:53:25.540297] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:25.540335] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:25.540343] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:53:25.540563] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:25.540596] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:25.540603] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:53:25.544947] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:53:25.545722] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:25.545775] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:53:25.565129] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:25.581582] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:25.605171] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:25.605224] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:53:25.666488] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:53:25.666528] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:53:25.749258] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:25.749290] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:53:25.749299] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:53:25.749306] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:53:25.749351] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:53:25.793552] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:25.793614] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:53:25.819849] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:53:25.819876] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:53:25.844664] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:25.844692] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:53:25.868395] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:25.868421] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:53:25.868428] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:53:25.868435] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:53:25.868478] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:27.602202] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:53:27.979002] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:53:27.989343] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fe97333947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fe97360c182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fe974106265]))) 0-: received signum (15), shutting down [2016-03-15 14:53:28.000867] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:53:28.004780] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:53:28.005381] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:53:28.005394] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:53:28.005737] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:53:28.005748] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:53:28.005753] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:53:28.005796] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:53:28.005848] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:53:28.005855] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:53:30.794137] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:53:30.802951] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:30.807200] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:30.813735] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:30.813776] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:30.813827] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:30.813834] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:53:30.814420] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:30.814457] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:30.814465] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:53:30.814756] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:30.814794] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:30.814801] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:53:30.822548] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:53:30.823202] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:30.823287] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:53:30.838120] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:30.862142] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:31.007480] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:31.007548] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:53:31.119933] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:53:31.119964] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:53:31.156873] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:53:31.223427] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:31.223489] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:53:31.223496] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:53:31.223503] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:53:31.297400] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:31.297474] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:53:31.425878] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:53:31.425929] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:53:31.499919] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:31.499962] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:53:31.566049] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:31.566083] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:53:31.566091] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:53:31.566097] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:53:31.566145] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:31.566166] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:32.878966] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:53:33.063061] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:53:33.073443] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fe7a878d47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fe7a8a60182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fe7a955a265]))) 0-: received signum (15), shutting down [2016-03-15 14:53:33.086074] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:53:33.090128] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:53:33.090655] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:53:33.090667] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:53:33.091839] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:53:33.091894] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:53:33.091910] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:53:33.091962] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:53:33.092037] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:53:33.092051] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:53:35.614909] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:53:35.619013] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:35.622546] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:35.626611] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:35.626651] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:35.626701] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:35.626709] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:53:35.627222] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:35.627260] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:35.627268] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:53:35.627480] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:35.627544] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:35.627554] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:53:35.633120] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:53:35.633909] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:35.633995] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:53:35.651919] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:35.667336] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:35.667385] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:53:35.687850] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:53:35.687876] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:53:35.771883] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:35.811832] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:53:35.847250] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:35.847286] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:53:35.847294] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:53:35.847301] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:53:35.904678] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:35.904742] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:53:35.978091] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:53:35.978121] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:53:36.031502] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:53:36.110844] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:36.110876] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:53:36.110884] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:53:36.110890] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:53:36.110937] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:36.110958] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:37.689625] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:53:38.149107] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:53:38.159870] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f64e2fe147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f64e32b4182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f64e3dae265]))) 0-: received signum (15), shutting down [2016-03-15 14:53:38.172110] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:53:38.176019] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:53:38.176738] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:53:38.176756] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:53:38.177265] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:53:38.177281] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:53:38.177290] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:53:38.177342] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:53:38.177413] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:53:38.177425] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:53:40.065227] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:53:40.068468] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:40.089115] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:40.098389] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:40.098435] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:40.098498] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:40.098506] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:53:40.099096] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:40.099132] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:40.099140] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:53:40.099380] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:40.099420] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:40.099429] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:53:40.104492] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:53:40.105541] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:40.105596] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:53:40.121956] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:40.219706] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:53:40.248411] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:40.248471] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:53:40.312714] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:53:40.312751] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:53:40.412361] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:40.412405] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:40.412429] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:53:40.412436] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:53:40.412443] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:53:40.412484] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:40.412531] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:53:40.465170] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:40.498866] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:40.498924] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:53:40.541316] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:53:40.541350] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:53:40.563625] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:40.581656] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:42.161195] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:53:43.233542] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:53:43.244768] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fe2c1d6047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fe2c2033182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fe2c2b2d265]))) 0-: received signum (15), shutting down [2016-03-15 14:53:43.255912] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:53:43.260380] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:53:43.260942] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:53:43.260955] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:53:43.261301] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:53:43.261312] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:53:43.261318] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:53:43.261353] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:53:43.261404] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:53:43.261412] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:53:45.065094] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:53:45.070850] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:45.075981] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:45.092275] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:45.092327] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:45.092381] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:45.092391] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:53:45.093014] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:45.093063] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:45.093078] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:53:45.093317] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:45.093355] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:45.093382] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:53:45.099520] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:53:45.100268] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:45.100320] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:53:45.121780] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:45.158521] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:45.339244] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:45.339320] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:53:45.362720] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:53:45.362755] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:53:45.405795] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:53:45.449033] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:45.449112] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:53:45.475419] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:53:45.475448] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:53:45.497402] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:45.497439] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:53:45.497447] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:53:45.497453] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:53:45.497513] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:45.497536] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:53:45.572785] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:45.572825] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:53:45.572844] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:53:45.572852] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:53:45.572918] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:45.572940] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:47.156178] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:53:48.318955] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:53:48.329785] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f0f532a547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f0f53578182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f0f54072265]))) 0-: received signum (15), shutting down [2016-03-15 14:53:48.342186] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:53:48.345187] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:53:48.345901] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:53:48.345918] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:53:48.346419] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:53:48.346437] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:53:48.346447] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:53:48.346499] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:53:48.346578] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:53:48.346593] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:53:50.058172] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:53:50.062549] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:50.065685] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:50.069005] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:50.069053] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:50.069105] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:50.069113] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:53:50.069683] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:50.069720] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:50.069728] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:53:50.069968] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:50.070007] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:50.070015] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:53:50.074793] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:53:50.075573] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:50.075625] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:53:50.250108] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:50.250176] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:53:50.275134] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:53:50.275183] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:53:50.294924] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:53:50.317704] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:53:50.347407] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:50.347444] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:53:50.347453] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:53:50.347460] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:53:50.347557] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:50.347574] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:50.348205] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:50.617047] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:50.631883] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:50.631932] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:53:50.651585] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:53:50.651618] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:53:50.684487] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:50.707998] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:52.131161] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:53:53.407156] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:53:53.417675] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f0c39b6547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f0c39e38182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f0c3a932265]))) 0-: received signum (15), shutting down [2016-03-15 14:53:53.429426] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:53:53.432943] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:53:53.433469] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:53:53.433482] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:53:53.433819] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:53:53.433830] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:53:53.433836] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:53:53.433873] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:53:53.433924] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:53:53.433951] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:53:55.077317] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:53:55.081140] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:55.084666] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:55.088978] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:53:55.089018] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:55.089067] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:55.089074] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:53:55.089586] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:55.089622] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:55.089629] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:53:55.089851] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:53:55.089888] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:53:55.089896] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:53:55.095024] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:53:55.096116] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:55.096168] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:53:55.194697] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:55.194777] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:53:55.230993] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:53:55.231020] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:53:55.299965] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:53:55.338940] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:55.338985] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:53:55.338993] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:53:55.339000] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:53:55.339046] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:55.341653] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:53:55.530808] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:55.530850] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:53:55.530858] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:53:55.530865] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:53:55.547528] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:55.665556] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:55.665616] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:53:55.700361] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:53:55.700391] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:53:55.769547] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:53:55.772430] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:53:56.166046] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:53:57.151616] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:53:58.495469] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:53:58.509181] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f06d769d47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f06d7970182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f06d846a265]))) 0-: received signum (15), shutting down [2016-03-15 14:53:58.520965] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:53:58.525105] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:53:58.525611] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:53:58.525623] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:53:58.525963] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:53:58.525974] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:53:58.525979] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:53:58.526013] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:53:58.526063] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:53:58.526071] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:54:00.174367] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:54:00.177690] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:00.181020] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:00.184243] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:00.184285] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:00.184335] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:00.184342] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:54:00.184917] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:00.184952] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:00.184959] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:54:00.185215] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:00.185251] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:00.185259] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:54:00.190324] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:54:00.191048] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:00.191102] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:54:00.206966] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:00.288645] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:54:00.307848] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:00.307930] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:54:00.454989] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:54:00.455029] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:54:00.709097] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:00.709143] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:00.709166] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:54:00.709173] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:54:00.709180] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:54:01.342938] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:54:01.396275] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:01.396312] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:54:01.396320] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:54:01.396327] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:54:01.424155] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:01.441394] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:01.441453] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:54:01.481744] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:54:01.481769] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:54:01.524536] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:01.559637] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:02.247298] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:54:03.583619] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:54:03.595425] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f1b2fbfa47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f1b2fecd182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f1b309c7265]))) 0-: received signum (15), shutting down [2016-03-15 14:54:03.607920] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:54:03.612601] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:54:03.613331] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:54:03.613348] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:54:03.613845] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:54:03.613864] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:54:03.613874] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:54:03.613921] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:54:03.613987] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:54:03.613997] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:54:05.449772] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:54:05.454393] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:05.459902] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:05.467368] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:05.467412] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:05.467463] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:05.467470] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:54:05.468044] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:05.468087] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:05.468098] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:54:05.468355] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:05.468388] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:05.468395] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:54:05.472842] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:54:05.473805] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:05.473859] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:54:05.491240] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:05.672394] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:05.672470] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:54:05.775801] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:54:05.775843] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:54:05.835481] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:54:05.903718] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:54:05.955204] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:05.955247] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:54:05.955254] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:54:05.955261] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:54:05.955338] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:05.955358] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:05.955402] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:05.955419] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:54:05.955428] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:54:05.955438] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:54:05.956327] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:06.017544] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:06.017604] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:54:06.065456] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:54:06.065485] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:54:06.122635] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:06.122666] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:07.530190] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:54:08.671875] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:54:08.682348] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f9e638c747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f9e63b9a182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f9e64694265]))) 0-: received signum (15), shutting down [2016-03-15 14:54:08.693813] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:54:08.697859] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:54:08.698564] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:54:08.698582] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:54:08.699130] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:54:08.699149] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:54:08.699160] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:54:08.699212] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:54:08.699295] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:54:08.699310] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:54:10.393835] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:54:10.400451] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:10.404867] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:10.412751] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:10.412793] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:10.412844] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:10.412852] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:54:10.413411] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:10.413461] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:10.413475] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:54:10.413754] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:10.413787] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:10.413795] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:54:10.420007] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:54:10.420746] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:10.420817] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:54:10.595662] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:10.595721] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:54:10.629211] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:54:10.629232] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:54:10.653780] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:10.653806] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:54:10.673518] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:10.673549] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:54:10.673557] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:54:10.673564] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:54:10.673606] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:54:10.697406] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:11.168603] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:11.621035] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:11.689653] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:11.689719] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:54:11.755729] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:54:11.755757] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:54:11.801071] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:11.801104] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:12.476562] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:54:13.761409] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:54:13.771901] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fdaf82bb47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fdaf858e182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fdaf9088265]))) 0-: received signum (15), shutting down [2016-03-15 14:54:13.783490] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:54:13.787075] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:54:13.787572] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:54:13.787584] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:54:13.787921] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:54:13.787932] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:54:13.787937] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:54:13.787974] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:54:13.788025] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:54:13.788032] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:54:15.443663] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:54:15.447689] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:15.452383] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:15.457423] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:15.457465] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:15.457513] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:15.457521] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:54:15.458055] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:15.458092] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:15.458100] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:54:15.458326] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:15.458362] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:15.458371] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:54:15.482910] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:54:15.483778] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:15.483837] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:54:15.500934] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:15.735432] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:15.735507] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:54:15.771089] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:54:15.771124] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:54:15.803599] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:54:15.843171] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:15.843206] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:54:15.843214] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:54:15.843221] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:54:15.874823] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:54:15.985263] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:15.985303] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:16.346343] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:16.369007] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:16.369059] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:54:16.417956] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:54:16.417988] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:54:16.447945] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:16.447978] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:17.539722] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:54:18.850098] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:54:18.860823] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f7badf8747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f7bae25a182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f7baed54265]))) 0-: received signum (15), shutting down [2016-03-15 14:54:18.873143] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:54:18.875464] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:54:18.875981] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:54:18.875993] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:54:18.876340] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:54:18.876352] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:54:18.876358] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:54:18.876393] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:54:18.876445] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:54:18.876453] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:54:20.658726] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:54:20.661988] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:20.686811] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:20.691017] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:20.691061] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:20.691109] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:20.691117] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:54:20.691694] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:20.691731] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:20.691748] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:54:20.692013] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:20.692045] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:20.692052] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:54:20.696553] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:54:20.697646] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:20.697701] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:54:20.718108] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:20.785976] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:20.810914] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:54:20.849673] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:20.849707] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:54:20.849715] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:54:20.849722] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:54:20.849773] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:20.849808] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:54:20.869617] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:54:20.869649] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:54:20.892931] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:20.940802] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:20.940861] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:54:20.969083] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:54:20.969108] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:54:20.990675] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:21.106599] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:54:21.160991] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:21.161028] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:54:21.161035] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:54:21.161042] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:54:21.189658] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:22.752584] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:54:23.937037] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:54:23.948209] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f29b51f447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f29b54c7182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f29b5fc1265]))) 0-: received signum (15), shutting down [2016-03-15 14:54:23.959915] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:54:23.962109] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:54:23.962610] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:54:23.962622] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:54:23.962993] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:54:23.963006] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:54:23.963012] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:54:23.963048] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:54:23.963099] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:54:23.963107] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:54:25.740354] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:54:25.744755] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:25.748545] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:25.754081] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:25.754122] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:25.754173] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:25.754181] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:54:25.754963] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:25.755009] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:25.755020] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:54:25.755225] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:25.755272] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:25.755298] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:54:25.761247] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:54:25.761936] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:25.761988] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:54:26.029614] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:26.029689] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:54:26.123498] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:54:26.123539] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:54:26.237230] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:54:26.264863] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:26.264900] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:54:26.264907] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:54:26.264914] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:54:26.264990] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:26.265647] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:26.623231] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:54:26.689458] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:26.689498] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:54:26.689506] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:54:26.689512] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:54:26.690005] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:26.739991] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:26.740051] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:54:26.793244] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:54:26.793277] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:54:26.861406] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:26.861452] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:27.817698] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:54:29.025087] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:54:29.036437] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f56e434c47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f56e461f182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f56e5119265]))) 0-: received signum (15), shutting down [2016-03-15 14:54:29.048775] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:54:29.051038] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:54:29.051549] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:54:29.051561] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:54:29.051898] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:54:29.051909] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:54:29.051915] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:54:29.051950] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:54:29.052001] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:54:29.052009] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:54:30.710372] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:54:30.714960] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:30.767539] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:30.781494] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:30.781540] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:30.781593] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:30.781602] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:54:30.782187] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:30.782222] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:30.782230] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:54:30.782455] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:30.782518] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:30.782528] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:54:30.788934] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:54:30.789578] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:30.789724] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:54:30.859175] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:30.925106] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:54:30.956328] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:30.956390] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:54:30.997035] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:54:30.997073] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:54:31.072812] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:31.072853] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:31.072876] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:54:31.072883] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:54:31.072890] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:54:31.348726] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:54:31.413894] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:31.413938] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:54:31.413948] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:54:31.413955] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:54:31.461120] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:31.476758] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:31.476804] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:54:31.679629] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:54:31.679660] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:54:31.709415] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:31.709444] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:32.845512] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:54:34.113501] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:54:34.123763] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7faed2dec47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7faed30bf182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7faed3bb9265]))) 0-: received signum (15), shutting down [2016-03-15 14:54:34.135421] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:54:34.139940] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:54:34.140765] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:54:34.140779] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:54:34.141122] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:54:34.141134] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:54:34.141140] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:54:34.141173] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:54:34.141224] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:54:34.141232] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:54:35.856167] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:54:35.861480] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:35.866821] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:35.886820] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:35.886868] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:35.886922] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:35.886930] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:54:35.887502] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:35.887540] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:35.887548] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:54:35.887778] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:35.887813] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:35.887820] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:54:35.892718] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:54:35.893176] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:35.893228] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:54:35.907661] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:35.907714] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:54:35.928039] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:54:35.928072] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:54:35.949663] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:36.006742] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:36.047049] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:54:36.069060] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:36.069096] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:54:36.069104] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:54:36.069111] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:54:36.069161] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:36.069243] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:54:36.091926] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:54:36.091953] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:54:36.132966] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:54:36.155279] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:36.155320] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:36.155339] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:54:36.155345] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:54:36.155352] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:54:37.949033] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:54:39.196645] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:54:39.207259] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fe97ef1147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fe97f1e4182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fe97fcde265]))) 0-: received signum (15), shutting down [2016-03-15 14:54:39.218886] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:54:39.224409] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:54:39.224913] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:54:39.224925] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:54:39.225259] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:54:39.225271] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:54:39.225277] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:54:39.225312] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:54:39.225366] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:54:39.225374] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:54:41.077293] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:54:41.081230] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:41.085063] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:41.090062] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:41.090103] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:41.090151] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:41.090158] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:54:41.090788] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:41.090846] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:41.090862] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:54:41.091131] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:41.091164] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:41.091171] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:54:41.098168] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:54:41.099464] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:41.099519] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:54:41.196243] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:41.196327] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:54:41.233336] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:54:41.233364] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:54:41.261607] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:54:41.291977] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:41.292011] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:54:41.292018] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:54:41.292025] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:54:41.323631] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:54:41.342027] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:41.342051] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:41.342744] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:41.626761] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:41.666634] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:41.666685] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:54:41.687587] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:54:41.687622] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:54:41.710875] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:41.710905] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:43.154225] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:54:44.281054] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:54:44.291502] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f32505e247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f32508b5182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f32513af265]))) 0-: received signum (15), shutting down [2016-03-15 14:54:44.303542] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:54:44.305673] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:54:44.306167] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:54:44.306179] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:54:44.306515] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:54:44.306533] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:54:44.306540] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:54:44.306576] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:54:44.306629] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:54:44.306637] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:54:46.024171] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:54:46.027637] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:46.032021] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:46.036927] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:46.036971] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:46.037023] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:46.037030] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:54:46.037644] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:46.037688] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:46.037699] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:54:46.037938] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:46.037975] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:46.037983] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:54:46.049812] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:54:46.050630] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:46.050688] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:54:46.078920] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:46.078980] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:54:46.117021] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:54:46.117058] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:54:46.386377] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:54:46.454900] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:46.454940] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:54:46.454960] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:54:46.454967] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:54:46.455030] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:54:46.550607] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:46.602882] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:46.602942] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:54:46.630131] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:54:46.630160] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:54:46.661283] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:46.661312] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:47.175057] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:48.106135] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:54:49.370932] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:54:49.382038] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f2c2130347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f2c215d6182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f2c220d0265]))) 0-: received signum (15), shutting down [2016-03-15 14:54:49.393972] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:54:49.398182] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:54:49.398862] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:54:49.398876] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:54:49.399227] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:54:49.399239] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:54:49.399244] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:54:49.399280] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:54:49.399330] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:54:49.399337] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:54:51.154301] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:54:51.158275] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:51.178834] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:51.183237] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:51.183279] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:51.183344] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:51.183352] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:54:51.183949] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:51.183986] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:51.183994] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:54:51.184217] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:51.184262] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:51.184272] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:54:51.189803] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:54:51.190805] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:51.190861] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:54:51.209408] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:51.317201] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:51.317293] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:54:51.342936] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:54:51.342971] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:54:51.407524] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:54:51.465688] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:54:51.569612] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:51.569662] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:54:51.569670] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:54:51.569677] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:54:51.569738] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:51.569761] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:52.353216] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:52.370934] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:52.370984] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:54:52.401610] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:54:52.401635] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:54:52.491818] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:52.491858] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:53.247295] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:54:54.458277] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:54:54.468850] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f73594e947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f73597bc182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f735a2b6265]))) 0-: received signum (15), shutting down [2016-03-15 14:54:54.480268] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:54:54.484342] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:54:54.484910] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:54:54.484923] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:54:54.485260] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:54:54.485272] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:54:54.485278] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:54:54.485312] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:54:54.485363] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:54:54.485371] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:54:56.293878] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:54:56.299277] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:56.304028] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:56.309587] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:54:56.309629] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:56.309680] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:56.309688] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:54:56.310197] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:56.310234] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:56.310242] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:54:56.310461] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:54:56.310497] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:54:56.310523] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:54:56.329023] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:54:56.329672] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:56.329729] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:54:56.362586] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:56.399127] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:56.399190] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:54:56.450548] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:54:56.450586] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:54:56.575870] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:54:56.617469] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:56.617506] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:54:56.617514] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:54:56.617521] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:54:56.617580] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:54:56.644498] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:54:56.660497] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:56.660544] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:54:56.682026] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:54:56.682058] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:54:56.717640] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:54:56.717674] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:54:58.386660] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:54:59.544820] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:54:59.558369] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f8f6b29d47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f8f6b570182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f8f6c06a265]))) 0-: received signum (15), shutting down [2016-03-15 14:54:59.571052] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:54:59.573245] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:54:59.573791] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:54:59.573804] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:54:59.574154] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:54:59.574165] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:54:59.574171] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:54:59.574207] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:54:59.574259] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:54:59.574267] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:55:01.382823] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:55:01.400835] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:01.406700] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:01.423654] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:01.423710] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:01.423764] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:01.423772] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:55:01.424327] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:01.424363] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:01.424370] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:55:01.424598] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:01.424643] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:01.424656] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:55:01.432966] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:55:01.433875] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:01.433930] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:55:01.516453] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:01.743037] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:01.743135] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:55:01.834518] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:55:01.834559] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:55:01.885551] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:55:01.929927] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:55:01.969518] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:01.969603] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:55:02.078539] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:55:02.078579] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:55:02.125099] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:02.125140] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:55:02.125148] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:55:02.125155] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:55:02.125207] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:02.125228] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:02.125252] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:02.125263] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:55:02.125269] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:55:02.125275] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:55:02.125309] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:02.125328] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:02.177816] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:03.490448] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:55:04.640330] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:55:04.651191] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ff2bb8bd47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ff2bbb90182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ff2bc68a265]))) 0-: received signum (15), shutting down [2016-03-15 14:55:04.662223] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:55:04.666308] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:55:04.666973] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:55:04.666987] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:55:04.667357] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:55:04.667368] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:55:04.667374] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:55:04.667409] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:55:04.667461] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:55:04.667469] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:55:06.653251] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:55:06.664733] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:06.670849] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:06.686224] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:06.686270] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:06.686319] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:06.686326] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:55:06.686920] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:06.686956] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:06.686963] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:55:06.687183] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:06.687227] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:06.687240] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:55:06.691294] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:55:06.692020] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:06.692074] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:55:06.706915] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:06.761246] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:55:06.795298] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:06.795358] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:55:06.889602] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:55:06.889631] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:55:07.041680] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:07.041791] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:07.041839] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:55:07.041854] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:55:07.041868] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:55:07.356155] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:55:07.397248] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:07.397282] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:55:07.397290] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:55:07.397297] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:55:07.416854] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:07.441914] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:07.441967] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:55:07.631414] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:55:07.631447] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:55:07.673526] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:07.673558] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:08.747732] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:55:09.728047] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:55:09.739317] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f02447ac47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f0244a7f182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f0245579265]))) 0-: received signum (15), shutting down [2016-03-15 14:55:09.750337] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:55:09.754310] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:55:09.755204] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:55:09.755223] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:55:09.755713] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:55:09.755729] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:55:09.755739] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:55:09.755797] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:55:09.755869] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:55:09.755881] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:55:11.380140] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:55:11.389578] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:11.398342] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:11.411589] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:11.411637] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:11.411688] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:11.411696] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:55:11.412218] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:11.412257] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:11.412265] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:55:11.412487] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:11.412524] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:11.412533] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:55:11.417791] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:55:11.419505] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:11.419557] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:55:11.453183] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:11.453257] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:55:11.527640] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:55:11.527680] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:55:11.589387] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:11.737471] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:55:11.800791] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:55:11.874055] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:11.874092] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:55:11.874100] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:55:11.874107] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:55:11.874189] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:11.874206] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:55:11.874212] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:55:11.874218] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:55:11.875028] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:11.899259] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:11.899304] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:55:11.925993] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:55:11.926012] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:55:11.953888] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:11.953918] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:13.473900] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:55:14.810013] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:55:14.820496] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd0aa0a347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd0aa376182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd0aae70265]))) 0-: received signum (15), shutting down [2016-03-15 14:55:14.832735] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:55:14.835737] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:55:14.836437] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:55:14.836454] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:55:14.836959] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:55:14.836987] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:55:14.836998] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:55:14.837051] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:55:14.837128] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:55:14.837143] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:55:16.582064] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:55:16.589418] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:16.593506] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:16.597110] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:16.597151] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:16.597204] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:16.597211] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:55:16.597802] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:16.597840] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:16.597848] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:55:16.598106] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:16.598139] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:16.598146] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:55:16.602319] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:55:16.603299] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:16.603354] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:55:16.663269] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:55:16.682493] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:16.682546] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:55:16.705314] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:55:16.705350] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:55:16.752158] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:16.752198] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:16.752229] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:55:16.752237] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:55:16.752244] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:55:17.180452] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:17.633159] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:55:17.704870] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:17.704907] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:55:17.704914] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:55:17.704921] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:55:17.705377] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:17.761446] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:17.761503] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:55:17.839944] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:55:17.839982] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:55:17.891007] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:17.891039] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:18.660170] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:55:19.897482] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:55:19.907804] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa65ef7b47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa65f24e182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa65fd48265]))) 0-: received signum (15), shutting down [2016-03-15 14:55:19.920386] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:55:19.923459] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:55:19.924154] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:55:19.924172] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:55:19.924694] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:55:19.924711] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:55:19.924721] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:55:19.924775] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:55:19.924863] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:55:19.924878] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:55:21.785499] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:55:21.789403] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:21.792754] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:21.796319] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:21.796359] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:21.796407] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:21.796414] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:55:21.796969] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:21.797006] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:21.797014] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:55:21.797241] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:21.797279] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:21.797287] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:55:21.801322] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:55:21.801901] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:21.801955] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:55:21.823636] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:21.896164] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:55:21.917553] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:55:21.948628] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:21.948685] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:55:21.969751] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:55:21.969777] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:55:22.005928] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:22.005960] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:22.005993] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:22.006014] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:55:22.006020] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:55:22.006027] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:55:22.359279] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:22.373558] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:22.373607] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:55:22.391426] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:55:22.391453] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:55:22.409308] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:22.769385] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:23.858201] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:55:24.985518] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:55:24.995825] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f0f6ec5f47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f0f6ef32182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f0f6fa2c265]))) 0-: received signum (15), shutting down [2016-03-15 14:55:25.007837] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:55:25.010914] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:55:25.011613] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:55:25.011631] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:55:25.012142] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:55:25.012159] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:55:25.012169] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:55:25.012221] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:55:25.012298] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:55:25.012313] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:55:26.784407] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:55:26.790366] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:26.796622] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:26.821814] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:26.821867] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:26.821933] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:26.821941] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:55:26.822538] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:26.822575] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:26.822583] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:55:26.822853] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:26.822886] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:26.822893] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:55:26.828255] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:55:26.829616] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:26.829673] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:55:26.888942] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:26.901633] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:26.914800] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:26.914854] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:55:26.939678] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:55:26.939703] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:55:26.982987] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:26.983044] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:55:27.020121] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:55:27.020149] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:55:27.043840] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:55:27.080184] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:27.080219] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:55:27.080227] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:55:27.080234] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:55:27.109997] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:27.110027] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:55:27.143323] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:27.143353] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:55:27.143360] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:55:27.143367] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:55:28.884631] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:55:30.072420] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:55:30.083825] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f8b0bace47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f8b0bda1182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f8b0c89b265]))) 0-: received signum (15), shutting down [2016-03-15 14:55:30.095678] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:55:30.098570] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:55:30.099284] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:55:30.099303] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:55:30.099820] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:55:30.099838] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:55:30.099848] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:55:30.099902] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:55:30.099978] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:55:30.099992] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:55:31.950538] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:55:31.964008] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:31.969292] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:31.978620] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:31.978662] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:31.978714] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:31.978721] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:55:31.979349] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:31.979386] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:31.979393] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:55:31.979652] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:31.979688] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:31.979706] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:55:31.985412] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:55:31.986163] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:31.986215] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:55:32.137486] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:32.137568] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:55:32.170741] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:55:32.170773] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:55:32.207333] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:55:32.224999] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:32.225031] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:55:32.225039] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:55:32.225046] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:55:32.225088] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:55:32.284084] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:32.284114] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:32.636735] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:32.652637] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:32.652687] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:55:32.674579] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:55:32.674605] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:55:32.775386] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:32.775421] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:33.089005] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:34.042309] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:55:35.160959] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:55:35.171537] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fdbe95be47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fdbe9891182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fdbea38b265]))) 0-: received signum (15), shutting down [2016-03-15 14:55:35.182842] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:55:35.186850] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:55:35.187516] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:55:35.187532] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:55:35.187979] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:55:35.187992] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:55:35.187998] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:55:35.188035] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:55:35.188085] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:55:35.188094] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:55:36.914860] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:55:36.917866] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:36.921390] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:36.924379] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:36.924419] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:36.924468] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:36.924476] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:55:36.925012] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:36.925047] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:36.925055] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:55:36.925301] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:36.925337] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:36.925345] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:55:36.929358] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:55:36.930127] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:36.930228] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:55:36.942784] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:36.998254] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:55:37.017390] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:37.017443] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:55:37.036298] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:55:37.036316] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:55:37.059122] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:37.059170] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:37.059190] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:55:37.059197] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:55:37.059203] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:55:37.293507] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:55:37.362981] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:37.404579] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:37.404636] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:55:37.461971] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:55:37.462011] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:55:37.561442] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:37.561484] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:38.986132] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:55:40.246110] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:55:40.256261] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f12584c747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f125879a182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f1259294265]))) 0-: received signum (15), shutting down [2016-03-15 14:55:40.268245] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:55:40.271157] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:55:40.271847] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:55:40.271864] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:55:40.272381] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:55:40.272399] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:55:40.272409] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:55:40.272466] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:55:40.272543] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:55:40.272558] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:55:41.927623] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:55:41.930943] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:41.933961] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:41.938107] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:41.938147] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:41.938194] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:41.938202] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:55:41.938751] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:41.938792] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:41.938800] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:55:41.939065] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:41.939097] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:41.939104] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:55:41.942721] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:55:41.943350] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:41.943434] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:55:41.957577] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:41.957640] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:55:41.974340] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:55:41.974361] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:55:41.994455] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:42.063928] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:42.111943] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:55:42.135256] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:42.135321] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:55:42.135352] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:55:42.135365] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:55:42.213537] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:42.213615] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:55:42.268449] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:55:42.268481] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:55:42.325587] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:42.390719] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:55:42.411040] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:42.411066] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:55:42.411074] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:55:42.411081] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:55:43.999323] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:55:45.334431] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:55:45.344993] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fea605d847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fea608ab182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fea613a5265]))) 0-: received signum (15), shutting down [2016-03-15 14:55:45.356079] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:55:45.360496] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:55:45.361159] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:55:45.361172] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:55:45.361514] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:55:45.361526] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:55:45.361531] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:55:45.361572] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:55:45.361623] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:55:45.361631] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:55:46.978482] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:55:46.982345] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:46.985660] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:46.988730] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:46.988769] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:46.988819] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:46.988826] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:55:46.989394] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:46.989429] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:46.989436] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:55:46.989695] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:46.989727] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:46.989734] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:55:46.993547] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:55:46.994888] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:46.994944] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:55:47.186340] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:47.236335] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:47.236403] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:55:47.255224] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:55:47.255241] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:55:47.271932] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:47.271956] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:55:47.291523] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:55:47.310802] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:47.310832] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:55:47.310840] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:55:47.310847] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:55:47.315746] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:47.639590] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:47.691050] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:47.691109] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:55:47.707976] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:55:47.708002] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:55:47.726232] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:47.727014] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:49.050301] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:55:50.428150] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:55:50.439308] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fbde674647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fbde6a19182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fbde7513265]))) 0-: received signum (15), shutting down [2016-03-15 14:55:50.450649] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:55:50.454558] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:55:50.455202] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:55:50.455216] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:55:50.455563] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:55:50.455574] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:55:50.455580] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:55:50.455616] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:55:50.455666] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:55:50.455674] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:55:52.193691] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:55:52.196827] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:52.199609] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:52.202610] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:52.202658] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:52.202709] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:52.202716] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:55:52.203279] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:52.203315] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:52.203322] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:55:52.203542] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:52.203613] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:52.203623] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:55:52.209102] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:55:52.210030] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:52.210085] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:55:52.244381] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:55:52.261644] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:52.261697] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:55:52.280689] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:55:52.280710] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:55:52.316778] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:52.316821] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:52.316843] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:55:52.316850] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:55:52.316857] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:55:52.364807] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:55:52.394936] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:52.394967] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:55:52.394983] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:55:52.394991] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:55:52.411953] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:52.429254] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:52.429302] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:55:52.464447] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:55:52.464493] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:55:52.484769] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:52.484795] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:53.186713] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:54.266105] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:55:55.526630] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:55:55.537019] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f54f173f47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f54f1a12182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f54f250c265]))) 0-: received signum (15), shutting down [2016-03-15 14:55:55.549454] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:55:55.552652] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:55:55.553351] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:55:55.553369] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:55:55.554116] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:55:55.554134] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:55:55.554145] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:55:55.554197] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:55:55.554275] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:55:55.554289] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:55:57.442786] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:55:57.480560] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:57.494304] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:57.501432] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:55:57.501475] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:57.501526] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:57.501533] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:55:57.502077] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:57.502125] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:57.502133] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:55:57.502365] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:55:57.502402] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:55:57.502411] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:55:57.514126] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:55:57.515079] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:57.515139] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:55:57.536114] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:57.572017] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:55:57.612442] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:57.612512] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:55:57.691282] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:55:57.691323] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:55:57.859065] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:57.859115] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:57.859140] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:55:57.859147] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:55:57.859154] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:55:57.859198] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:55:58.366334] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:55:58.403911] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:58.403971] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:55:58.420853] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:55:58.420896] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:55:58.439642] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:55:58.439671] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:55:59.572424] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:56:00.617573] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:56:00.633493] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f8b75f4b47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f8b7621e182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f8b76d18265]))) 0-: received signum (15), shutting down [2016-03-15 14:56:00.645830] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:56:00.648734] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:56:00.649247] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:56:00.649259] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:56:00.649606] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:56:00.649617] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:56:00.649623] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:56:00.649659] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:56:00.649709] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:56:00.649721] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:56:02.330361] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:56:02.334236] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:02.350001] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:02.386669] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:02.386769] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:02.386842] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:02.386853] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:56:02.387471] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:02.387508] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:02.387515] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:56:02.387747] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:02.387785] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:02.387794] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:56:02.394060] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:56:02.394993] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:02.395066] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:56:02.431892] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:02.919233] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:02.919311] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:56:02.968753] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:56:02.968788] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:56:03.021656] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:56:03.195723] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:56:03.231191] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:03.231244] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:56:03.231255] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:56:03.231262] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:56:03.231342] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:03.231360] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:03.231410] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:03.231429] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:56:03.231440] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:56:03.231452] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:56:03.232332] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:03.244620] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:03.244668] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:56:03.263348] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:56:03.263372] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:56:03.289165] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:03.312237] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:04.451611] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:56:05.712941] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:56:05.723926] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f258ceb047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f258d183182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f258dc7d265]))) 0-: received signum (15), shutting down [2016-03-15 14:56:05.735993] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:56:05.740401] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:56:05.740917] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:56:05.740929] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:56:05.741267] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:56:05.741278] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:56:05.741285] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:56:05.741321] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:56:05.741373] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:56:05.741382] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:56:07.440265] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:56:07.453137] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:07.463263] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:07.468518] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:07.468561] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:07.468611] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:07.468619] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:56:07.469194] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:07.469231] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:07.469240] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:56:07.469457] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:07.469494] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:07.469502] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:56:07.483517] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:56:07.484820] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:07.484878] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:56:07.611421] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:07.611485] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:56:07.638021] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:56:07.638054] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:56:07.732408] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:56:07.787534] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:07.787582] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:56:07.787590] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:56:07.787597] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:56:07.787642] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:56:07.849725] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:07.849756] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:08.189453] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:08.643594] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:08.663888] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:08.663934] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:56:08.691963] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:56:08.691988] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:56:08.723794] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:08.734298] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:09.539639] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:56:10.798756] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:56:10.810274] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f45de91347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f45debe6182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f45df6e0265]))) 0-: received signum (15), shutting down [2016-03-15 14:56:10.822202] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:56:10.824664] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:56:10.825180] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:56:10.825192] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:56:10.825541] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:56:10.825552] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:56:10.825558] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:56:10.825594] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:56:10.825646] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:56:10.825654] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:56:12.574144] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:56:12.592554] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:12.598874] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:12.604312] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:12.604355] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:12.604405] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:12.604413] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:56:12.604984] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:12.605021] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:12.605028] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:56:12.605278] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:12.605311] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:12.605319] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:56:12.615142] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:56:12.615867] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:12.615923] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:56:12.638813] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:12.771178] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:56:12.823573] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:12.823658] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:56:12.866017] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:56:12.866051] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:56:12.925684] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:12.925728] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:12.925755] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:56:12.925763] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:56:12.925770] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:56:12.925810] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:12.925832] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:56:13.369355] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:13.385723] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:13.385772] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:56:13.405733] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:56:13.405767] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:56:13.424342] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:13.424373] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:14.671298] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:56:15.887322] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:56:15.897788] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f77d7dc947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f77d809c182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f77d8b96265]))) 0-: received signum (15), shutting down [2016-03-15 14:56:15.909749] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:56:15.912005] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:56:15.912505] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:56:15.912517] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:56:15.912851] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:56:15.912862] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:56:15.912876] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:56:15.912915] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:56:15.912965] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:56:15.912973] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:56:17.670233] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:56:17.674334] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:17.678515] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:17.688885] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:17.688929] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:17.688979] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:17.688986] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:56:17.689559] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:17.689597] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:17.689604] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:56:17.689841] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:17.689875] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:17.689882] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:56:17.694314] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:56:17.695371] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:17.695424] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:56:17.710222] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:17.724141] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:17.796233] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:56:17.814799] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:17.814864] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:56:17.833337] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:56:17.833363] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:56:17.929669] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:17.929736] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:56:17.954529] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:56:17.954550] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:56:17.978519] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:17.978556] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:17.978577] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:56:17.978584] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:56:17.978591] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:56:17.978630] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:56:17.998322] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:17.998351] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:56:17.998358] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:56:17.998365] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:56:17.998408] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:17.998431] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:19.751133] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:56:20.974544] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:56:20.984947] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f55d0e8747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f55d115a182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f55d1c54265]))) 0-: received signum (15), shutting down [2016-03-15 14:56:20.996911] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:56:20.999917] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:56:21.000617] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:56:21.000635] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:56:21.001161] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:56:21.001178] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:56:21.001187] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:56:21.001236] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:56:21.001309] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:56:21.001331] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:56:22.699341] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:56:22.702826] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:22.705933] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:22.709394] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:22.709433] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:22.709484] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:22.709491] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:56:22.710081] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:22.710124] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:22.710133] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:56:22.710396] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:22.710432] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:22.710439] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:56:22.714938] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:56:22.715855] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:22.715907] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:56:22.834398] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:56:22.870267] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:22.870323] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:56:22.888483] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:56:22.888519] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:56:22.973732] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:22.973778] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:22.973801] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:56:22.973808] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:56:22.973815] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:56:22.973869] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:56:23.037474] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:23.191981] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:23.646617] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:23.664299] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:23.664350] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:56:23.682829] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:56:23.682856] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:56:23.706888] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:23.790029] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:24.771104] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:56:26.061216] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:56:26.071483] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f0faebcb47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f0faee9e182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f0faf998265]))) 0-: received signum (15), shutting down [2016-03-15 14:56:26.082666] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:56:26.087155] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:56:26.088177] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:56:26.088195] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:56:26.088553] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:56:26.088565] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:56:26.088571] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:56:26.088607] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:56:26.088658] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:56:26.088667] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:56:27.848061] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:56:27.860991] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:27.864608] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:27.870869] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:27.870909] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:27.870959] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:27.870975] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:56:27.871473] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:27.871511] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:27.871519] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:56:27.871737] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:27.871769] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:27.871776] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:56:27.878213] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:56:27.879323] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:27.879377] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:56:27.912452] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:27.985091] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:56:28.006855] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:28.006915] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:56:28.029305] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:56:28.029340] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:56:28.078869] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:28.078911] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:28.078933] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:56:28.078940] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:56:28.078946] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:56:28.371733] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:56:28.410733] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:28.410777] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:56:28.410784] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:56:28.410804] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:56:28.460958] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:28.496785] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:28.496837] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:56:28.517646] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:56:28.517675] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:56:28.550116] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:28.554502] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:29.934373] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:56:31.144334] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:56:31.156111] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ffbe6db247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ffbe7085182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ffbe7b7f265]))) 0-: received signum (15), shutting down [2016-03-15 14:56:31.168274] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:56:31.172307] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:56:31.173532] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:56:31.173551] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:56:31.174064] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:56:31.174081] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:56:31.174091] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:56:31.174149] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:56:31.174226] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:56:31.174241] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:56:32.991433] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:56:32.995079] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:32.999191] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:33.001996] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:33.002037] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:33.002088] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:33.002096] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:56:33.002751] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:33.002792] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:33.002800] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:56:33.003085] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:33.003119] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:33.003126] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:56:33.007748] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:56:33.008260] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:33.008362] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:56:33.022908] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:33.022970] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:56:33.041639] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:56:33.041657] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:56:33.064737] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:33.122502] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:33.157407] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:56:33.187732] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:33.187762] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:56:33.187771] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:56:33.187778] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:56:33.187825] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:33.187863] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:56:33.219725] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:56:33.219763] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:56:33.389200] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:56:33.479898] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:33.479955] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:33.479975] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:56:33.479981] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:56:33.479988] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:56:35.064664] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:56:36.228255] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:56:36.238565] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f06071d547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f06074a8182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f0607fa2265]))) 0-: received signum (15), shutting down [2016-03-15 14:56:36.249829] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:56:36.254084] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:56:36.254908] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:56:36.254928] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:56:36.255450] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:56:36.255467] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:56:36.255478] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:56:36.255554] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:56:36.255633] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:56:36.255649] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:56:37.976178] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:56:37.979632] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:37.981968] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:37.984841] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:37.984880] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:37.984927] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:37.984934] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:56:37.985433] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:37.985468] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:37.985476] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:56:37.985689] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:37.985725] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:37.985732] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:56:37.991243] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:56:37.992312] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:37.992367] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:56:38.123583] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:38.123647] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:56:38.143466] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:56:38.143505] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:56:38.179185] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:56:38.249761] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:56:38.301133] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:38.301170] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:56:38.301178] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:56:38.301185] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:56:38.301233] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:38.301255] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:38.301911] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:38.649442] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:38.663676] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:38.663725] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:56:38.687249] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:56:38.687276] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:56:38.767783] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:38.767826] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:40.048892] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:56:41.312802] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:56:41.323268] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ff3bdb3b47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ff3bde0e182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ff3be908265]))) 0-: received signum (15), shutting down [2016-03-15 14:56:41.335427] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:56:41.338421] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:56:41.339277] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:56:41.339296] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:56:41.340021] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:56:41.340038] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:56:41.340048] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:56:41.340102] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:56:41.340179] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:56:41.340194] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:56:43.023110] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:56:43.028268] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:43.031867] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:43.035413] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:43.035454] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:43.035505] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:43.035513] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:56:43.036079] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:43.036116] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:43.036124] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:56:43.036348] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:43.036380] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:43.036387] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:56:43.053598] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:56:43.053935] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:43.053991] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:56:43.072196] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:43.072255] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:56:43.108790] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:56:43.108822] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:56:43.217866] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:56:43.252477] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:43.252513] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:56:43.252521] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:56:43.252528] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:56:43.374605] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:56:43.415438] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:43.415468] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:56:43.415475] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:56:43.415482] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:56:43.444276] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:43.468325] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:43.468370] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:56:43.494791] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:56:43.494817] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:56:43.555694] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:43.558341] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:44.195712] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:45.109778] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:56:46.403622] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:56:46.413996] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ffdc6ea547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ffdc7178182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ffdc7c72265]))) 0-: received signum (15), shutting down [2016-03-15 14:56:46.426318] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:56:46.428588] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:56:46.429098] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:56:46.429110] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:56:46.429449] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:56:46.429460] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:56:46.429465] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:56:46.429501] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:56:46.429552] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:56:46.429560] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:56:48.177811] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:56:48.181210] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:48.185246] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:48.189002] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:48.189043] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:48.189095] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:48.189102] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:56:48.189692] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:48.189730] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:48.189738] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:56:48.189963] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:48.190000] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:48.190008] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:56:48.196084] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:56:48.196998] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:48.197052] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:56:48.281680] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:56:48.300935] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:48.300991] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:56:48.323477] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:56:48.323504] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:56:48.370382] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:48.388060] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:48.388095] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:48.388115] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:56:48.388122] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:56:48.388129] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:56:49.226868] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:56:49.376434] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:49.397681] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:49.397732] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:56:49.485458] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:56:49.485493] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:56:49.535966] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:49.535999] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:50.253115] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:56:51.493025] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:56:51.504345] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f47bda8847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f47bdd5b182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f47be855265]))) 0-: received signum (15), shutting down [2016-03-15 14:56:51.516289] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:56:51.518514] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:56:51.519040] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:56:51.519054] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:56:51.519394] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:56:51.519404] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:56:51.519410] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:56:51.519446] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:56:51.519499] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:56:51.519507] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:56:53.391494] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:56:53.395503] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:53.399771] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:53.405151] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:53.405192] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:53.405238] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:53.405246] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:56:53.405847] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:53.405884] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:53.405892] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:56:53.406132] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:53.406164] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:53.406177] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:56:53.412361] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:56:53.412652] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:53.412704] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:56:53.451219] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:53.537945] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:53.538026] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:56:53.568854] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:56:53.568885] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:56:53.658201] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:56:53.705625] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:53.705663] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:56:53.705670] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:56:53.705677] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:56:53.705735] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:56:53.771141] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:53.771168] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:53.772027] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:53.788074] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:53.788117] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:56:53.837561] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:56:53.837598] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:56:53.890108] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:53.890142] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:55.469568] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:56:56.582984] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:56:56.593826] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f9894efb47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f98951ce182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f9895cc8265]))) 0-: received signum (15), shutting down [2016-03-15 14:56:56.605458] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:56:56.608478] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:56:56.609190] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:56:56.609207] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:56:56.609728] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:56:56.609744] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:56:56.609753] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:56:56.609805] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:56:56.609881] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:56:56.609896] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:56:58.398535] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:56:58.403715] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:58.406905] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:58.409656] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:56:58.409694] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:58.409745] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:58.409752] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:56:58.410347] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:58.410389] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:58.410398] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:56:58.410614] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:56:58.410653] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:56:58.410664] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:56:58.419240] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:56:58.419853] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:58.419907] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:56:58.433893] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:56:58.475829] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:56:58.529671] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:58.529736] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:56:58.617033] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:56:58.617079] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:56:58.828715] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:58.828763] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:58.828786] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:56:58.828794] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:56:58.828801] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:56:58.888115] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:58.888187] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:56:58.912563] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:56:58.912589] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:56:58.956953] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:56:58.957002] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:56:58.987648] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:58.987678] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:56:58.987685] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:56:58.987692] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:56:58.987737] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:56:59.198378] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:00.476569] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:57:01.669508] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:57:01.683541] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa7c31d947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa7c34ac182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa7c3fa6265]))) 0-: received signum (15), shutting down [2016-03-15 14:57:01.696542] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:57:01.699928] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:57:01.700650] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:57:01.700667] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:57:01.701201] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:57:01.701218] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:57:01.701229] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:57:01.701283] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:57:01.701361] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:57:01.701375] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:57:03.529593] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:57:03.557859] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:03.561462] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:03.564716] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:03.564760] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:03.564813] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:03.564820] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:57:03.565411] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:03.565448] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:03.565456] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:57:03.565697] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:03.565742] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:03.565754] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:57:03.569646] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:57:03.570576] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:03.570632] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:57:03.590878] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:03.683837] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:57:03.882676] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:03.882768] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:57:03.981283] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:57:03.981326] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:57:04.036220] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:04.036257] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:57:04.079885] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:04.079929] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:57:04.079936] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:57:04.079943] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:57:04.204808] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:04.379202] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:04.431490] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:04.431553] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:57:04.456961] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:57:04.456983] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:57:04.484534] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:04.484570] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:05.626268] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:57:06.763594] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:57:06.774107] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f878502f47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f8785302182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f8785dfc265]))) 0-: received signum (15), shutting down [2016-03-15 14:57:06.786458] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:57:06.791130] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:57:06.791980] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:57:06.791997] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:57:06.792460] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:57:06.792473] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:57:06.792479] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:57:06.792513] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:57:06.792565] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:57:06.792573] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:57:08.576021] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:57:08.666558] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:08.689714] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:08.726663] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:08.726712] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:08.726791] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:08.726803] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:57:08.727395] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:08.727431] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:08.727439] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:57:08.727666] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:08.727702] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:08.727713] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:57:08.771850] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:57:08.772824] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:08.772881] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:57:08.938963] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:08.939027] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:57:08.981625] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:57:08.981672] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:57:09.003318] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:09.110897] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:57:09.132481] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:09.132512] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:57:09.132519] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:57:09.132526] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:57:09.170097] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:57:09.204765] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:09.225312] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:09.225340] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:57:09.225347] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:57:09.225354] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:57:09.225676] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:09.225714] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:57:09.248685] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:57:09.248712] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:57:09.269072] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:09.277650] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:10.828946] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:57:11.848350] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:57:11.860050] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fe1d4c0647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fe1d4ed9182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fe1d59d3265]))) 0-: received signum (15), shutting down [2016-03-15 14:57:11.872295] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:57:11.874532] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:57:11.875058] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:57:11.875071] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:57:11.875419] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:57:11.875430] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:57:11.875436] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:57:11.875471] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:57:11.875521] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:57:11.875528] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:57:13.544889] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:57:13.548648] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:13.552143] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:13.555007] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:13.555047] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:13.555095] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:13.555103] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:57:13.555618] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:13.555657] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:13.555665] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:57:13.555884] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:13.555921] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:13.555929] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:57:13.559651] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:57:13.560728] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:13.560782] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:57:13.626719] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:13.626818] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:57:13.645724] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:57:13.645749] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:57:13.715225] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:13.715260] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:57:13.715268] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:57:13.715276] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:57:13.715320] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:57:13.758355] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:13.883477] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:57:14.201192] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:14.656555] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:14.681615] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:14.681668] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:57:14.704892] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:57:14.704927] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:57:14.735841] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:14.735872] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:15.616015] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:57:16.938607] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:57:16.949003] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fde673de47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fde676b1182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fde681ab265]))) 0-: received signum (15), shutting down [2016-03-15 14:57:16.961287] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:57:16.964510] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:57:16.965215] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:57:16.965232] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:57:16.965743] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:57:16.965771] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:57:16.965782] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:57:16.965835] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:57:16.965912] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:57:16.965925] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:57:18.806255] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:57:18.809314] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:18.812515] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:18.815624] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:18.815665] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:18.815713] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:18.815720] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:57:18.816299] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:18.816336] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:18.816344] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:57:18.816575] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:18.816616] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:18.816628] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:57:18.821572] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:57:18.822235] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:18.822378] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:57:18.839309] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:18.914514] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:18.914577] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:57:18.934667] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:57:18.934693] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:57:18.955202] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:57:18.999984] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:19.000022] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:57:19.000030] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:57:19.000037] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:57:19.000083] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:57:19.025338] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:19.025365] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:19.382453] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:19.406256] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:19.406310] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:57:19.426935] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:57:19.426959] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:57:19.458553] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:19.458585] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:20.878098] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:57:22.025276] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:57:22.035558] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fc884f9247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fc885265182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fc885d5f265]))) 0-: received signum (15), shutting down [2016-03-15 14:57:22.047880] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:57:22.051848] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:57:22.052575] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:57:22.052592] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:57:22.053079] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:57:22.053096] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:57:22.053107] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:57:22.053151] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:57:22.053204] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:57:22.053212] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:57:23.859929] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:57:23.875439] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:23.889508] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:23.899360] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:23.899405] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:23.899454] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:23.899461] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:57:23.900045] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:23.900081] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:23.900088] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:57:23.900351] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:23.900383] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:23.900390] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:57:23.934530] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:57:23.934985] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:23.935086] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:57:23.992496] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:23.992553] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:57:24.044062] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:57:24.044091] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:57:24.106981] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:24.266352] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:24.372298] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:57:24.431224] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:24.431263] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:57:24.431271] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:57:24.431277] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:57:24.431328] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:57:24.495614] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:24.495648] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:57:24.495655] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:57:24.495662] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:57:24.495710] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:24.495751] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:57:24.572124] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:57:24.572169] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:57:24.630804] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:24.630837] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:25.990483] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:57:27.110695] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:57:27.121252] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fcf947aa47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fcf94a7d182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fcf95577265]))) 0-: received signum (15), shutting down [2016-03-15 14:57:27.132387] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:57:27.136326] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:57:27.136833] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:57:27.136846] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:57:27.137178] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:57:27.137189] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:57:27.137195] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:57:27.137229] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:57:27.137279] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:57:27.137286] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:57:28.931150] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:57:28.943188] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:28.946368] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:28.952780] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:28.952822] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:28.952871] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:28.952889] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:57:28.953465] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:28.953502] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:28.953510] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:57:28.953756] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:28.953788] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:28.953795] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:57:28.963972] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:57:28.965160] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:28.965213] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:57:29.073308] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:57:29.129331] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:29.129395] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:57:29.148262] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:57:29.148297] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:57:29.235738] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:29.235784] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:29.235807] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:57:29.235814] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:57:29.235821] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:57:29.236535] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:29.658876] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:57:29.719654] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:29.719687] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:57:29.719694] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:57:29.719712] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:57:29.799786] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:29.815555] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:29.815599] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:57:29.835088] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:57:29.835113] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:57:29.857581] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:29.857613] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:31.021294] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:57:32.197736] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:57:32.209228] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f510697847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f5106c4b182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f5107745265]))) 0-: received signum (15), shutting down [2016-03-15 14:57:32.221076] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:57:32.224230] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:57:32.224940] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:57:32.224958] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:57:32.225464] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:57:32.225481] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:57:32.225491] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:57:32.225545] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:57:32.225628] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:57:32.225643] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:57:34.116711] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:57:34.127939] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:34.134228] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:34.140507] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:34.140550] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:34.140601] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:34.140608] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:57:34.141149] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:34.141184] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:34.141202] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:57:34.141432] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:34.141476] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:34.141485] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:57:34.158978] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:57:34.160024] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:34.160080] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:57:34.276430] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:57:34.336759] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:34.336835] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:57:34.415250] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:57:34.415290] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:57:34.591489] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:34.591536] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:34.591561] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:57:34.591571] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:57:34.591578] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:57:34.591632] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:57:34.781847] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:34.781886] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:57:34.781894] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:57:34.781902] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:57:34.782528] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:34.822796] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:34.822863] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:57:34.849922] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:57:34.849947] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:57:34.887857] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:34.887892] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:35.205024] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:36.215215] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:57:37.287485] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:57:37.297847] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f2b68cf547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f2b68fc8182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f2b69ac2265]))) 0-: received signum (15), shutting down [2016-03-15 14:57:37.309721] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:57:37.313166] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:57:37.313728] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:57:37.313741] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:57:37.314138] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:57:37.314154] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:57:37.314164] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:57:37.314205] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:57:37.314256] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:57:37.314264] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:57:39.222212] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:57:39.287558] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:39.291548] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:39.306565] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:39.306620] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:39.306674] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:39.306682] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:57:39.307382] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:39.307419] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:39.307427] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:57:39.307690] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:39.307722] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:39.307729] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:57:39.312707] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:57:39.313539] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:39.313637] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:57:39.340630] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:39.340693] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:57:39.398386] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:57:39.398421] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:57:39.432195] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:39.521439] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:39.574246] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:57:39.632845] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:39.632885] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:57:39.632892] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:57:39.632899] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:57:39.716047] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:39.716115] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:57:39.774633] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:57:39.774672] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:57:39.838582] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:57:39.878656] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:39.878687] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:57:39.878694] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:57:39.878701] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:57:39.878804] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:39.878827] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:41.369336] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:57:42.375173] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:57:42.385699] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f35bfa4d47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f35bfd20182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f35c081a265]))) 0-: received signum (15), shutting down [2016-03-15 14:57:42.397175] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:57:42.400736] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:57:42.401239] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:57:42.401251] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:57:42.401584] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:57:42.401595] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:57:42.401601] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:57:42.401636] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:57:42.401686] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:57:42.401694] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:57:44.119678] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:57:44.123480] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:44.126773] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:44.171437] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:44.171491] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:44.171540] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:44.171547] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:57:44.172113] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:44.172150] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:44.172158] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:57:44.172377] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:44.172414] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:44.172423] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:57:44.176888] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:57:44.177129] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:44.177189] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:57:44.191680] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:44.191726] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:57:44.212245] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:57:44.212263] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:57:44.373305] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:57:44.392757] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:44.392787] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:57:44.392795] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:57:44.392803] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:57:44.392843] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:57:44.421019] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:44.662302] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:44.680496] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:44.680546] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:57:44.702237] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:57:44.702259] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:57:44.720723] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:44.730375] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:46.232802] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:57:47.462909] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:57:47.473686] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f1766e8f47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f1767162182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f1767c5c265]))) 0-: received signum (15), shutting down [2016-03-15 14:57:47.485622] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:57:47.489207] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:57:47.489710] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:57:47.489722] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:57:47.490055] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:57:47.490066] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:57:47.490072] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:57:47.490108] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:57:47.490158] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:57:47.490166] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:57:49.243297] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:57:49.250915] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:49.273388] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:49.300711] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:49.300761] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:49.300816] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:49.300824] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:57:49.301331] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:49.301369] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:49.301377] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:57:49.301598] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:49.301641] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:49.301654] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:57:49.363879] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:57:49.364951] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:49.365010] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:57:49.398348] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:49.398399] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:57:49.433009] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:57:49.433038] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:57:49.540846] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:57:49.583678] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:49.583720] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:57:49.583729] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:57:49.583737] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:57:49.666835] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:57:49.771297] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:49.771340] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:57:49.771348] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:57:49.771354] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:57:49.771416] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:49.865589] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:49.865654] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:57:49.884622] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:57:49.884649] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:57:49.913954] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:49.913985] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:50.207725] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:51.420604] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:57:52.550511] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:57:52.562217] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fafa7aab47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fafa7d7e182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fafa8878265]))) 0-: received signum (15), shutting down [2016-03-15 14:57:52.574761] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:57:52.579485] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:57:52.580303] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:57:52.580317] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:57:52.580657] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:57:52.580676] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:57:52.580682] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:57:52.580718] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:57:52.580769] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:57:52.580777] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:57:54.426016] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:57:54.429544] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:54.433333] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:54.436166] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:54.436259] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:54.436313] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:54.436320] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:57:54.436911] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:54.436949] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:54.436957] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:57:54.437182] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:54.437217] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:54.437224] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:57:54.441301] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:57:54.441560] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:54.441617] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:57:54.456273] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:54.553242] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:54.553327] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:57:54.572605] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:57:54.572632] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:57:54.594797] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:57:54.692094] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:54.692140] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:57:54.692148] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:57:54.692155] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:57:54.692213] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:54.710591] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:54.710614] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:57:55.389001] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:55.434556] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:55.434612] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:57:55.462252] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:57:55.462278] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:57:55.491161] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:55.494159] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:56.498633] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:57:57.635904] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:57:57.646763] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fbddeffc47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fbddf2cf182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fbddfdc9265]))) 0-: received signum (15), shutting down [2016-03-15 14:57:57.659040] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:57:57.663298] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:57:57.664208] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:57:57.664226] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:57:57.664827] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:57:57.664844] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:57:57.664855] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:57:57.664909] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:57:57.664987] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:57:57.665032] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:57:59.468800] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:57:59.473546] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:59.479340] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:59.485800] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:57:59.485841] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:59.485893] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:59.485901] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:57:59.486468] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:59.486505] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:59.486512] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:57:59.486799] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:57:59.486839] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:57:59.486846] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:57:59.491288] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:57:59.492030] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:59.492082] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:57:59.512642] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:59.685051] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:59.685120] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:57:59.712916] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:57:59.712949] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:57:59.732846] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:57:59.756325] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:59.756357] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:57:59.756365] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:57:59.756372] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:57:59.756414] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:57:59.779116] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:59.779149] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:59.779985] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:57:59.797992] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:57:59.798038] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:57:59.821785] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:57:59.821807] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:57:59.848174] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:57:59.848203] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:01.548674] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:58:02.721326] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:58:02.734853] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f3677c4d47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f3677f20182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f3678a1a265]))) 0-: received signum (15), shutting down [2016-03-15 14:58:02.746839] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:58:02.749851] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:58:02.750551] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:58:02.750569] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:58:02.751248] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:58:02.751267] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:58:02.751277] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:58:02.751330] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:58:02.751412] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:58:02.751427] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:58:04.641524] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:58:04.659448] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:04.667854] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:04.681890] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:04.681934] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:04.681985] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:04.681993] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:58:04.682575] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:04.682612] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:04.682620] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:58:04.682911] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:04.682945] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:04.682952] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:58:04.692281] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:58:04.693414] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:04.693541] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:58:04.788115] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:58:04.811184] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:04.811242] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:58:04.833584] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:58:04.833622] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:58:04.872761] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:04.872804] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:04.872825] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:58:04.872832] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:58:04.872839] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:58:05.210401] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:05.285009] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:58:05.666259] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:05.764263] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:05.764325] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:58:05.813283] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:58:05.813313] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:58:05.865609] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:05.865654] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:06.758431] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:58:07.814959] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:58:07.825571] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ffd790d347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ffd793a6182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ffd79ea0265]))) 0-: received signum (15), shutting down [2016-03-15 14:58:07.837514] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:58:07.839702] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:58:07.840210] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:58:07.840222] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:58:07.840552] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:58:07.840564] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:58:07.840569] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:58:07.840608] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:58:07.840658] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:58:07.840666] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:58:09.531698] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:58:09.535194] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:09.542823] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:09.545965] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:09.546007] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:09.546059] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:09.546066] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:58:09.546567] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:09.546604] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:09.546612] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:58:09.546862] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:09.546896] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:09.546903] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:58:09.551127] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:58:09.551931] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:09.551989] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:58:09.580922] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:09.815100] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:58:09.852331] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:09.852396] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:58:09.902312] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:58:09.902353] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:58:09.950167] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:09.950214] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:09.950237] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:58:09.950244] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:58:09.950251] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:58:09.950292] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:09.950307] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:58:10.391990] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:10.511846] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:10.511911] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:58:10.530148] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:58:10.530173] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:58:10.550876] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:10.585141] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:11.608057] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:58:12.900346] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:58:12.911689] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ff711f0647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ff7121d9182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ff712cd3265]))) 0-: received signum (15), shutting down [2016-03-15 14:58:12.923731] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:58:12.927671] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:58:12.928260] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:58:12.928273] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:58:12.928612] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:58:12.928623] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:58:12.928629] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:58:12.928665] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:58:12.928714] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:58:12.928722] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:58:14.815511] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:58:14.820709] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:14.828264] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:14.857833] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:14.857885] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:14.857935] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:14.857943] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:58:14.858493] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:14.858531] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:14.858539] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:58:14.858786] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:14.858821] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:14.858832] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:58:14.875216] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:58:14.875760] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:14.875816] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:58:14.912048] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:14.954516] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:15.073893] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:15.073975] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:58:15.103170] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:58:15.103198] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:58:15.122080] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:58:15.173312] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:15.173378] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:58:15.252395] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:58:15.252426] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:58:15.275120] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:15.275152] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:58:15.275159] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:58:15.275166] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:58:15.275211] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:15.275233] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:58:15.292523] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:15.292547] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:58:15.292554] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:58:15.292560] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:58:15.292603] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:15.292619] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:16.931345] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:58:17.984512] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:58:17.994910] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa5ef28647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa5ef559182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa5f0053265]))) 0-: received signum (15), shutting down [2016-03-15 14:58:18.006283] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:58:18.008802] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:58:18.009314] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:58:18.009334] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:58:18.010487] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:58:18.010503] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:58:18.010513] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:58:18.010562] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:58:18.010639] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:58:18.010651] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:58:19.782258] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:58:19.786426] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:19.791063] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:19.793900] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:19.793939] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:19.793988] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:19.793996] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:58:19.794528] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:19.794563] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:19.794571] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:58:19.794814] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:19.794861] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:19.794872] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:58:19.799195] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:58:19.799824] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:19.799877] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:58:19.888388] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:19.888451] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:58:19.919845] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:58:19.919873] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:58:19.937871] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:58:19.954707] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:19.954745] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:58:19.954753] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:58:19.954760] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:58:19.954801] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:58:20.015053] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:20.015087] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:20.213025] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:20.669239] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:20.686380] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:20.686426] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:58:20.718546] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:58:20.718573] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:58:20.755687] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:20.771080] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:21.855075] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:58:23.071043] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:58:23.081748] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa1e505247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa1e5325182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa1e5e1f265]))) 0-: received signum (15), shutting down [2016-03-15 14:58:23.093277] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:58:23.095405] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:58:23.095893] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:58:23.095905] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:58:23.096237] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:58:23.096248] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:58:23.096254] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:58:23.096290] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:58:23.096343] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:58:23.096352] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:58:24.748822] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:58:24.753024] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:24.756098] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:24.759752] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:24.759793] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:24.759842] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:24.759849] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:58:24.760413] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:24.760448] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:24.760456] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:58:24.760700] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:24.760738] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:24.760746] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:58:24.781252] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:58:24.781917] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:24.782028] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:58:24.800599] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:24.853148] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:58:24.872007] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:24.872057] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:58:24.894031] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:58:24.894059] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:58:24.948644] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:24.948686] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:24.948708] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:58:24.948715] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:58:24.948730] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:58:25.394421] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:58:25.426125] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:25.426158] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:58:25.426165] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:58:25.426173] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:58:25.458288] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:25.473443] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:25.473500] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:58:25.492850] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:58:25.492881] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:58:25.518590] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:25.518621] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:26.838285] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:58:28.156818] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:58:28.167307] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f6e5cba647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f6e5ce79182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f6e5d973265]))) 0-: received signum (15), shutting down [2016-03-15 14:58:28.178949] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:58:28.182849] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:58:28.183472] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:58:28.183486] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:58:28.183822] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:58:28.183833] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:58:28.183839] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:58:28.183875] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:58:28.183924] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:58:28.183931] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:58:29.897165] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:58:29.900403] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:29.903794] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:29.907477] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:29.907517] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:29.907590] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:29.907600] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:58:29.908123] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:29.908158] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:29.908165] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:58:29.908384] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:29.908428] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:29.908441] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:58:29.912269] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:58:29.913323] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:29.913375] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:58:29.931958] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:29.945458] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:29.963698] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:29.963751] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:58:29.985615] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:58:29.985642] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:58:30.049039] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:30.049081] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:58:30.049089] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:58:30.049097] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:58:30.049149] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:58:30.154317] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:30.154383] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:58:30.180100] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:58:30.180129] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:58:30.205587] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:30.205611] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:58:30.224023] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:30.224050] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:58:30.224058] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:58:30.224065] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:58:30.224110] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:31.969652] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:58:33.241932] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:58:33.253328] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f86d831847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f86d85eb182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f86d90e5265]))) 0-: received signum (15), shutting down [2016-03-15 14:58:33.266270] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:58:33.268632] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:58:33.269136] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:58:33.269147] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:58:33.269488] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:58:33.269499] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:58:33.269504] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:58:33.269539] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:58:33.269590] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:58:33.269598] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:58:35.025251] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:58:35.029421] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:35.032505] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:35.037302] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:35.037343] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:35.037393] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:35.037401] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:58:35.037950] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:35.037985] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:35.037992] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:58:35.038212] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:35.038248] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:35.038260] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:58:35.044324] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:58:35.045140] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:35.045192] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:58:35.202928] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:35.202996] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:58:35.251971] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:58:35.252005] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:58:35.320513] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:58:35.389180] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:35.389229] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:58:35.389236] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:58:35.389243] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:58:35.389307] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:58:35.446802] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:35.446830] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:35.447550] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:35.672118] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:35.688359] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:35.688405] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:58:35.710543] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:58:35.710561] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:58:35.729336] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:35.771094] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:37.101301] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:58:38.335197] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:58:38.345847] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fc090c3047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fc090f03182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fc0919fd265]))) 0-: received signum (15), shutting down [2016-03-15 14:58:38.357020] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:58:38.361051] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:58:38.361586] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:58:38.361598] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:58:38.362002] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:58:38.362014] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:58:38.362020] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:58:38.362054] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:58:38.362106] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:58:38.362114] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:58:40.128200] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:58:40.146830] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:40.156990] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:40.166030] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:40.166077] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:40.166126] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:40.166134] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:58:40.166720] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:40.166779] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:40.166788] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:58:40.167030] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:40.167071] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:40.167080] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:58:40.174768] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:58:40.175872] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:40.175927] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:58:40.250691] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:58:40.397777] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:40.481520] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:40.481580] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:58:40.524720] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:58:40.524757] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:58:40.545434] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:40.545502] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:58:40.599934] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:58:40.599970] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:58:40.618726] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:40.618783] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:40.823566] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:58:41.216757] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:41.335399] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:41.335452] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:58:41.335465] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:58:41.335472] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:58:41.478141] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:41.478197] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:42.232228] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:58:43.418240] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:58:43.430063] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f9afcadb47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f9afcdae182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f9afd8a8265]))) 0-: received signum (15), shutting down [2016-03-15 14:58:43.441283] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:58:43.445317] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:58:43.446035] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:58:43.446052] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:58:43.446578] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:58:43.446595] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:58:43.446606] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:58:43.446660] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:58:43.446871] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:58:43.446889] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:58:45.233914] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:58:45.237908] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:45.241451] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:45.251525] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:45.251567] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:45.251618] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:45.251625] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:58:45.252163] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:45.252199] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:45.252207] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:58:45.252425] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:45.252492] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:45.252501] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:58:45.264314] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:58:45.265229] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:45.265307] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:58:45.413939] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:45.492468] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:58:45.511645] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:45.511673] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:58:45.511681] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:58:45.511688] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:58:45.511729] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:58:45.548467] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:45.548565] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:58:45.569972] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:58:45.570015] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:58:45.591713] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:45.591742] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:46.399131] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:46.459148] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:46.459205] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:58:46.486759] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:58:46.486778] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:58:46.583923] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:46.583950] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:47.320403] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:58:48.503716] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:58:48.514103] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ff177efd47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ff1781d0182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ff178cca265]))) 0-: received signum (15), shutting down [2016-03-15 14:58:48.525848] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:58:48.529597] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:58:48.530099] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:58:48.530118] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:58:48.530455] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:58:48.530466] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:58:48.530472] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:58:48.530508] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:58:48.530558] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:58:48.530566] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:58:50.223224] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:58:50.226899] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:50.230302] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:50.233361] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:50.233401] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:50.233450] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:50.233457] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:58:50.234056] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:50.234094] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:50.234102] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:58:50.234333] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:50.234372] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:50.234381] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:58:50.239734] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:58:50.240432] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:50.240486] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:58:50.257166] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:50.411999] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:50.412089] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:58:50.456792] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:58:50.456828] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:58:50.570895] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:58:50.599774] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:50.599807] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:58:50.599815] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:58:50.599823] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:58:50.599881] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:58:50.620849] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:50.620872] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:50.675136] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:50.690034] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:50.690080] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:58:50.719807] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:58:50.719835] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:58:50.749254] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:50.749286] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:52.296624] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:58:53.588289] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:58:53.599792] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f9294d9147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f9295064182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f9295b5e265]))) 0-: received signum (15), shutting down [2016-03-15 14:58:53.611201] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:58:53.615744] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:58:53.616435] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:58:53.616453] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:58:53.616946] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:58:53.616962] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:58:53.616969] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:58:53.617005] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:58:53.617060] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:58:53.617069] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:58:55.327398] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:58:55.333661] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:55.340940] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:55.344844] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:58:55.344885] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:55.344934] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:55.344941] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:58:55.345505] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:55.345548] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:55.345558] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:58:55.345786] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:58:55.345852] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:58:55.345861] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:58:55.350272] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:58:55.351139] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:55.351192] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:58:55.426824] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:55.426892] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:58:55.454963] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:58:55.454997] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:58:55.549973] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:55.550009] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:58:55.550017] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:58:55.550024] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:58:55.550074] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:58:55.614122] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:55.614164] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:58:55.636868] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:55.636894] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:58:55.636900] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:58:55.636908] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:58:55.656849] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:55.701642] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:55.701705] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:58:55.777248] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:58:55.777285] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:58:55.797016] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:58:55.797044] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:58:56.219466] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:58:57.407473] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:58:58.670852] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:58:58.681974] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f35c830147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f35c85d4182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f35c90ce265]))) 0-: received signum (15), shutting down [2016-03-15 14:58:58.694408] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:58:58.698315] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:58:58.699316] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:58:58.699335] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:58:58.700124] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:58:58.700141] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:58:58.700151] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:58:58.700196] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:58:58.700250] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:58:58.700258] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:59:00.346377] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:59:00.350314] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:00.353936] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:00.357060] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:00.357100] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:00.357148] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:00.357156] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:59:00.357726] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:00.357768] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:00.357778] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:59:00.358030] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:00.358062] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:00.358069] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:59:00.362769] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:59:00.363412] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:00.363463] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:59:00.394026] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:00.451834] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:59:00.473664] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:00.473719] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:59:00.519969] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:59:00.520007] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:59:00.563300] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:00.563341] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:00.563362] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:59:00.563369] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:59:00.563375] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:59:00.786745] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:59:01.402180] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:01.434226] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:01.434279] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:59:01.464385] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:59:01.464415] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:59:01.573826] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:01.573869] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:02.419268] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:59:03.758697] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:59:03.771145] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fe191a0547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fe191cd8182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fe1927d2265]))) 0-: received signum (15), shutting down [2016-03-15 14:59:03.782793] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:59:03.786954] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:59:03.787684] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:59:03.787702] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:59:03.788230] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:59:03.788247] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:59:03.788258] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:59:03.788311] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:59:03.788388] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:59:03.788402] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:59:05.484703] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:59:05.489001] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:05.492491] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:05.495863] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:05.495904] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:05.495956] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:05.495964] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:59:05.496543] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:05.496585] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:05.496595] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:59:05.496845] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:05.496880] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:05.496898] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:59:05.501695] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:59:05.502177] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:05.502274] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:59:05.520713] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:05.520778] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:59:05.542260] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:59:05.542311] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:59:05.561544] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:05.691707] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:59:05.711570] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:59:05.741945] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:05.741982] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:59:05.741989] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:59:05.741996] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:59:05.742083] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:05.742099] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:59:05.742111] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:59:05.742118] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:59:05.742839] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:05.758548] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:05.758594] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:59:05.861772] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:59:05.861914] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:59:05.886113] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:05.886148] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:07.558563] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:59:08.845986] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:59:08.856543] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f076e6d047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f076e9a3182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f076f49d265]))) 0-: received signum (15), shutting down [2016-03-15 14:59:08.868591] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:59:08.870900] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:59:08.871478] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:59:08.871491] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:59:08.871834] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:59:08.871845] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:59:08.871851] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:59:08.871887] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:59:08.871938] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:59:08.871946] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:59:10.834912] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:59:10.868812] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:10.915827] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:10.952500] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:10.952552] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:10.952607] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:10.952615] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:59:10.953228] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:10.953266] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:10.953274] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:59:10.953506] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:10.953546] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:10.953554] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:59:10.984643] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:59:10.985790] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:10.985850] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:59:11.279628] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:59:11.318396] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:59:11.392132] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:11.392194] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:59:11.481060] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:59:11.481098] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:59:11.554800] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:11.554833] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:11.554858] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:11.554882] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:59:11.554889] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:59:11.554896] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:59:11.555643] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:11.679317] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:11.698873] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:11.698938] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:59:11.723077] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:59:11.723106] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:59:11.747569] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:11.779621] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:13.041003] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:59:13.933339] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:59:13.944757] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f777bd8947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f777c05c182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f777cb56265]))) 0-: received signum (15), shutting down [2016-03-15 14:59:13.956208] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:59:13.960072] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:59:13.960672] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:59:13.960685] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:59:13.961024] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:59:13.961036] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:59:13.961042] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:59:13.961077] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:59:13.961127] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:59:13.961135] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:59:15.745416] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:59:15.795623] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:15.808228] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:15.814806] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:15.814850] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:15.814902] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:15.814910] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:59:15.815487] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:15.815523] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:15.815531] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:59:15.815752] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:15.815796] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:15.815810] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:59:15.830682] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:59:15.831126] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:15.831183] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:59:15.868781] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:16.066186] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:59:16.128504] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:16.128571] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:59:16.193650] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:59:16.193680] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:59:16.272422] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:16.272466] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:16.272491] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:59:16.272498] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:59:16.272505] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:59:16.272547] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:16.272568] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:59:16.404905] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:16.435793] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:16.435851] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:59:16.476053] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:59:16.476081] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:59:16.514229] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:16.514255] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:17.887230] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:59:19.018922] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:59:19.029484] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f949f33847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f949f60b182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f94a0105265]))) 0-: received signum (15), shutting down [2016-03-15 14:59:19.041518] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:59:19.044546] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:59:19.045245] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:59:19.045263] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:59:19.045790] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:59:19.045816] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:59:19.045828] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:59:19.045881] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:59:19.045956] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:59:19.045970] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:59:20.926478] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:59:20.935131] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:20.941856] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:20.946455] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:20.946497] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:20.946546] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:20.946554] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:59:20.947185] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:20.947224] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:20.947232] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:59:20.947465] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:20.947502] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:20.947509] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:59:20.954939] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:59:20.955675] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:20.955728] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:59:20.983673] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:21.009736] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:21.122103] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:21.122167] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:59:21.196887] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:59:21.196925] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:59:21.233148] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:59:21.267326] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:59:21.296413] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:21.296450] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:59:21.296458] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:59:21.296465] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:59:21.296521] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:21.296561] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:59:21.332263] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:59:21.332293] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:59:21.357693] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:21.357722] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:21.357748] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:21.357764] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:59:21.357770] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:59:21.357777] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:59:21.357821] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:21.357836] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:23.010725] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:59:24.107102] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:59:24.117831] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fb9dc5d547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fb9dc8a8182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fb9dd3a2265]))) 0-: received signum (15), shutting down [2016-03-15 14:59:24.128819] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:59:24.132787] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:59:24.133304] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:59:24.133317] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:59:24.133658] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:59:24.133670] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:59:24.133683] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:59:24.133718] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:59:24.133771] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:59:24.133779] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:59:26.072245] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:59:26.104771] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:26.113269] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:26.124199] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:26.124245] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:26.124297] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:26.124305] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:59:26.124886] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:26.124930] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:26.124940] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:59:26.125207] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:26.125240] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:26.125247] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:59:26.134702] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:59:26.135753] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:26.135809] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:59:26.268171] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:26.268234] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:59:26.293302] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:59:26.293329] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:59:26.312000] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:59:26.336841] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:59:26.352346] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:26.352378] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:59:26.352394] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:59:26.352401] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:59:26.352475] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:26.352492] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:26.353152] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:26.681985] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:26.699875] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:26.699923] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:59:26.791525] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:59:26.791563] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:59:26.812001] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:26.812041] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:28.191338] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:59:29.189780] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:59:29.200823] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa3f8dd447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa3f90a7182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa3f9ba1265]))) 0-: received signum (15), shutting down [2016-03-15 14:59:29.213029] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:59:29.217017] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:59:29.218251] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:59:29.218269] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:59:29.218830] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:59:29.218848] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:59:29.218858] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:59:29.218923] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:59:29.219000] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:59:29.219014] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:59:30.997108] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:59:31.005807] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:31.033710] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:31.037433] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:31.037489] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:31.037539] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:31.037547] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:59:31.038139] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:31.038176] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:31.038184] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:59:31.038436] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:31.038469] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:31.038476] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:59:31.042037] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:59:31.043114] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:31.043170] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:59:31.057617] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:31.282750] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:31.282814] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:59:31.302900] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:59:31.302925] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:59:31.362224] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:31.362252] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:59:31.407822] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:31.426710] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:59:31.456858] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:31.456893] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:59:31.456901] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:59:31.456907] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:59:31.456959] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:31.456984] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:31.457019] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:59:31.480284] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:59:31.480311] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:59:31.521152] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:31.521192] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:33.098368] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:59:34.272626] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:59:34.283913] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f76cfe1447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f76d00e7182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f76d0be1265]))) 0-: received signum (15), shutting down [2016-03-15 14:59:34.295080] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:59:34.299306] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:59:34.300009] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:59:34.300027] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:59:34.300551] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:59:34.300568] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:59:34.300578] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:59:34.300631] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:59:34.300706] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:59:34.300720] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:59:36.339458] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:59:36.348425] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:36.371436] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:36.393847] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:36.393898] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:36.393951] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:36.393958] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:59:36.394476] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:36.394514] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:36.394522] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:59:36.394766] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:36.394819] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:36.394832] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:59:36.402292] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:59:36.402901] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:36.402973] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:59:36.427344] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:36.516279] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:36.579403] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:36.579466] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:59:36.597977] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:59:36.597996] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:59:36.617058] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:59:36.645053] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:36.645087] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:59:36.645095] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:59:36.645102] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:59:36.645143] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:59:36.667050] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:36.667077] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:59:36.667084] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:59:36.667090] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:59:36.667138] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:36.667163] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:36.667204] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:59:36.690340] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:59:36.690375] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:59:36.711867] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:36.711899] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:36.741754] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:38.458808] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:59:39.357555] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:59:39.368060] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f24928f847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f2492bcb182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f24936c5265]))) 0-: received signum (15), shutting down [2016-03-15 14:59:39.381057] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:59:39.385054] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:59:39.385653] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:59:39.385670] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:59:39.386010] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:59:39.386021] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:59:39.386027] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:59:39.386062] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:59:39.386115] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:59:39.386123] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:59:41.054297] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:59:41.066341] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:41.086974] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:41.094914] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:41.094959] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:41.095014] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:41.095022] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:59:41.095607] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:41.095642] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:41.095649] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:59:41.095907] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:41.095939] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:41.095946] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:59:41.108667] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:59:41.109450] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:41.109502] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:59:41.270459] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:41.270526] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:59:41.307642] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:59:41.307666] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:59:41.333533] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:59:41.353631] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:41.353664] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:59:41.353671] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:59:41.353678] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:59:41.353720] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:59:41.390413] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:41.390451] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:41.391075] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:41.684820] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:41.744118] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:41.744175] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:59:41.820019] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:59:41.820048] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:59:41.861359] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:41.861391] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:43.165569] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:59:44.442886] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:59:44.453279] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f700d80f47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f700dae2182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f700e5dc265]))) 0-: received signum (15), shutting down [2016-03-15 14:59:44.465937] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:59:44.469885] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:59:44.470605] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:59:44.470623] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:59:44.471166] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:59:44.471184] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:59:44.471194] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:59:44.471288] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:59:44.471365] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:59:44.471376] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:59:46.372235] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:59:46.379984] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:46.384439] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:46.387543] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:46.387583] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:46.387630] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:46.387637] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:59:46.388199] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:46.388242] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:46.388252] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:59:46.388507] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:46.388539] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:46.388546] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:59:46.392551] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:59:46.393556] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:46.393610] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:59:46.464894] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:59:46.497227] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:46.497265] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:59:46.497274] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:59:46.497281] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:59:46.537280] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:46.537342] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:59:46.642779] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:59:46.642834] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:59:46.724947] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:59:46.839703] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:46.839832] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:59:46.839950] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:59:46.839969] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:59:46.840105] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:46.948708] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:47.007325] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:47.007669] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:47.007717] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:59:47.030056] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:59:47.030084] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:59:47.054193] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:47.081946] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:47.228243] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:48.449727] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:59:49.529261] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:59:49.540040] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fb85354f47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fb853822182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fb85431c265]))) 0-: received signum (15), shutting down [2016-03-15 14:59:49.552159] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:59:49.554583] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:59:49.555117] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:59:49.555131] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:59:49.555473] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:59:49.555484] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:59:49.555490] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:59:49.555526] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:59:49.555577] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:59:49.555585] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:59:51.436576] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:59:51.448829] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:51.459649] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:51.472718] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:51.472773] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:51.472832] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:51.472840] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:59:51.473377] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:51.473412] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:51.473420] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:59:51.473641] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:51.473677] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:51.473685] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:59:51.535431] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:59:51.536493] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:51.536549] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:59:51.607391] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:51.772642] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:51.772718] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:59:51.795629] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:59:51.795657] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:59:51.819575] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:59:51.848055] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:51.848089] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:59:51.848097] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:59:51.848104] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:59:51.848159] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:51.848176] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:51.848198] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:59:52.411671] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:52.429117] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:52.429165] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:59:52.450684] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:59:52.450704] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:59:52.474156] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:52.476770] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:53.592582] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:59:54.617452] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:59:54.628919] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f727fbaf47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f727fe82182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f728097c265]))) 0-: received signum (15), shutting down [2016-03-15 14:59:54.640171] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:59:54.644091] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:59:54.644599] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:59:54.644618] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:59:54.644960] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:59:54.644971] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:59:54.644977] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:59:54.645012] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:59:54.645062] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:59:54.645070] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:59:56.401129] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:59:56.406995] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:56.410523] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:56.413653] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:59:56.413693] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:56.413740] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:56.413747] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:59:56.414329] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:56.414367] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:56.414375] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:59:56.414597] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:59:56.414641] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:59:56.414657] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:59:56.421037] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:59:56.421664] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:56.421719] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:59:56.436617] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:56.436674] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:59:56.456613] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:59:56.456645] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:59:56.520116] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:56.600700] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:59:56.621466] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:56.621499] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:59:56.621506] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:59:56.621513] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:59:56.621558] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:59:56.687654] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:59:56.710354] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:56.710407] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:59:56.739140] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:59:56.739167] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:59:56.760084] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:59:56.760113] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:59:58.477190] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:59:59.701376] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:59:59.711901] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f0ae836447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f0ae8637182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f0ae9131265]))) 0-: received signum (15), shutting down [2016-03-15 14:59:59.723869] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:59:59.728401] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:59:59.729082] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:59:59.729096] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:59:59.729440] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:59:59.729451] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:59:59.729458] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:59:59.729492] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:59:59.729544] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:59:59.729552] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:00:01.495293] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:00:01.518331] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:01.522061] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:01.531078] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:01.531129] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:01.531182] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:01.531189] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:00:01.531775] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:01.531814] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:01.531821] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:00:01.532041] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:01.532085] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:01.532097] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:00:01.536778] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:00:01.538037] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:01.538093] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:00:01.664656] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:00:01.752650] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:01.752724] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:00:02.125373] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:00:02.125424] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:00:02.392097] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:02.392134] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:00:02.424781] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:02.424822] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:00:02.424830] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:00:02.424837] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:00:02.450979] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:02.451870] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:02.689041] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:02.705493] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:02.705552] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:00:02.724809] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:00:02.724839] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:00:02.748103] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:02.748799] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:03.594865] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:00:04.785664] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:00:04.800082] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f9f7d5e847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f9f7d8bb182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f9f7e3b5265]))) 0-: received signum (15), shutting down [2016-03-15 15:00:04.811153] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:00:04.815152] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:00:04.815752] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:00:04.815766] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:00:04.816109] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:00:04.816121] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:00:04.816127] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:00:04.816162] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:00:04.816213] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:00:04.816221] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:00:06.465443] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:00:06.468698] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:06.473088] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:06.477164] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:06.477205] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:06.477254] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:06.477262] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:00:06.477855] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:06.477906] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:06.477919] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:00:06.478193] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:06.478240] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:06.478253] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:00:06.484541] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:00:06.485541] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:06.485665] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:00:06.505540] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:06.544421] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:00:06.568525] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:06.568587] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:00:06.591191] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:00:06.591219] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:00:06.666153] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:06.666195] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:06.666219] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:00:06.666226] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:00:06.666233] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:00:07.414119] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:00:07.484121] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:07.484160] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:00:07.484169] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:00:07.484175] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:00:07.526625] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:07.575364] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:07.575441] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:00:07.633067] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:00:07.633099] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:00:07.696879] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:07.696911] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:08.542117] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:00:09.873011] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:00:09.883972] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f68e6a9547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f68e6d68182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f68e7862265]))) 0-: received signum (15), shutting down [2016-03-15 15:00:09.896324] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:00:09.900418] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:00:09.900932] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:00:09.900944] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:00:09.901282] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:00:09.901293] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:00:09.901298] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:00:09.901332] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:00:09.901383] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:00:09.901391] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:00:11.680592] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:00:11.685821] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:11.691580] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:11.694782] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:11.694824] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:11.694876] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:11.694884] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:00:11.695457] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:11.695493] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:11.695500] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:00:11.695747] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:11.695784] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:11.695791] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:00:11.700225] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:00:11.700957] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:11.701055] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:00:11.720347] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:11.752740] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:11.752801] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:00:11.921669] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:00:11.921706] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:00:12.041945] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:12.121188] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:00:12.142691] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:12.142725] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:00:12.142758] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:00:12.142772] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:00:12.142829] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:00:12.190608] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:12.190668] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:00:12.230535] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:00:12.230565] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:00:12.279648] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:12.320204] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:12.320247] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:00:12.320254] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:00:12.320275] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:00:12.342933] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:13.757030] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:00:14.957956] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:00:14.969293] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa1ef26347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa1ef536182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa1f0030265]))) 0-: received signum (15), shutting down [2016-03-15 15:00:14.981249] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:00:14.984933] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:00:14.985447] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:00:14.985458] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:00:14.985798] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:00:14.985809] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:00:14.985815] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:00:14.985851] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:00:14.985902] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:00:14.985910] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:00:16.805758] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:00:16.809625] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:16.816854] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:16.830791] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:16.830841] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:16.830894] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:16.830901] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:00:16.831443] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:16.831480] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:16.831487] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:00:16.831714] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:16.831750] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:16.831757] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:00:16.841370] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:00:16.842435] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:16.842489] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:00:16.928447] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:16.928521] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:00:16.956537] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:00:16.956574] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:00:17.013348] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:00:17.113326] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:17.113373] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:00:17.113381] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:00:17.113391] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:00:17.242882] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:17.242926] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:17.242941] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:00:17.308002] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:17.691993] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:17.740313] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:17.740390] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:00:17.811978] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:00:17.812015] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:00:17.868740] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:17.868774] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:18.899231] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:00:20.046271] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:00:20.057206] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fad367ad47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fad36a80182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fad3757a265]))) 0-: received signum (15), shutting down [2016-03-15 15:00:20.070161] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:00:20.073961] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:00:20.074469] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:00:20.074481] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:00:20.075101] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:00:20.075121] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:00:20.075131] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:00:20.075182] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:00:20.075325] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:00:20.075340] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:00:21.821366] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:00:21.824442] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:21.829394] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:21.832932] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:21.832973] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:21.833024] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:21.833032] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:00:21.833609] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:21.833647] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:21.833655] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:00:21.833905] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:21.833942] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:21.833949] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:00:21.841418] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:00:21.842375] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:21.842428] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:00:21.861592] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:21.925816] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:00:21.990415] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:21.990489] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:00:22.045071] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:00:22.045109] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:00:22.096723] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:22.096766] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:22.096788] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:00:22.096795] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:00:22.096801] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:00:22.417131] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:00:22.474501] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:22.474541] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:00:22.474549] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:00:22.474556] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:00:22.494651] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:22.537921] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:22.537983] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:00:22.567497] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:00:22.567526] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:00:22.591198] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:22.591230] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:23.898682] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:00:25.134056] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:00:25.145069] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f6d7c7fd47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f6d7cad0182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f6d7d5ca265]))) 0-: received signum (15), shutting down [2016-03-15 15:00:25.156908] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:00:25.161137] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:00:25.161681] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:00:25.161693] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:00:25.162034] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:00:25.162045] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:00:25.162051] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:00:25.162085] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:00:25.162136] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:00:25.162144] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:00:27.055675] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:00:27.060491] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:27.065555] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:27.078658] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:27.078702] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:27.078779] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:27.078791] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:00:27.079358] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:27.079401] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:27.079411] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:00:27.079668] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:27.079701] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:27.079708] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:00:27.093631] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:00:27.094380] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:27.094452] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:00:27.130781] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:27.130845] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:00:27.157814] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:00:27.157848] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:00:27.178780] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:27.252754] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:27.274312] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:00:27.296988] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:27.297022] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:00:27.297029] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:00:27.297036] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:00:27.396760] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:27.396827] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:00:27.461683] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:00:27.461714] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:00:27.506858] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:00:27.556034] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:27.556073] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:00:27.556080] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:00:27.556087] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:00:27.556138] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:27.556160] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:29.150331] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:00:30.217788] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:00:30.228677] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f193ff9447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f1940267182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f1940d61265]))) 0-: received signum (15), shutting down [2016-03-15 15:00:30.240620] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:00:30.242813] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:00:30.243317] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:00:30.243329] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:00:30.243705] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:00:30.243716] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:00:30.243730] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:00:30.243766] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:00:30.243817] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:00:30.243825] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:00:31.933485] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:00:31.942061] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:31.958935] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:31.963429] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:31.963471] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:31.963520] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:31.963527] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:00:31.964106] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:31.964149] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:31.964159] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:00:31.964417] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:31.964449] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:31.964456] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:00:31.975598] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:00:31.977144] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:31.977185] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:00:32.045348] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:32.045425] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:00:32.096165] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:00:32.096202] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:00:32.201407] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:32.201447] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:00:32.201455] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:00:32.201462] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:00:32.231241] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:00:32.263272] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:32.263943] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:32.414631] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:00:32.694896] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:32.714942] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:32.714994] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:00:32.740541] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:00:32.740568] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:00:32.758705] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:32.758731] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:34.032113] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:00:35.305098] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:00:35.316335] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f5518b6d47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f5518e40182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f551993a265]))) 0-: received signum (15), shutting down [2016-03-15 15:00:35.329298] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:00:35.332622] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:00:35.333143] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:00:35.333161] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:00:35.333503] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:00:35.333514] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:00:35.333520] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:00:35.333555] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:00:35.333607] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:00:35.333615] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:00:37.196573] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:00:37.202011] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:37.208180] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:37.223153] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:37.223232] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:37.223299] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:37.223308] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:00:37.223829] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:37.223867] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:37.223874] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:00:37.224092] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:37.224127] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:37.224134] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:00:37.235429] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:00:37.236947] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:37.236995] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:00:37.373638] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:37.373705] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:00:37.394203] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:00:37.394224] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:00:37.420162] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:00:37.441753] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:37.441785] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:00:37.441792] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:00:37.441799] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:00:37.508278] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:00:37.538879] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:37.538908] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:37.539642] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:37.583319] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:37.583386] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:00:37.605152] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:00:37.605179] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:00:37.643300] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:37.643329] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:38.237462] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:39.292062] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:00:40.392806] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:00:40.403953] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f78dc82147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f78dcaf4182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f78dd5ee265]))) 0-: received signum (15), shutting down [2016-03-15 15:00:40.415607] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:00:40.419614] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:00:40.420323] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:00:40.420340] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:00:40.421051] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:00:40.421069] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:00:40.421080] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:00:40.421914] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:00:40.421993] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:00:40.422007] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:00:42.887298] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:00:42.890649] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:42.897755] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:42.902160] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:42.902203] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:42.902255] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:42.902263] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:00:42.902803] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:42.902843] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:42.902851] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:00:42.903108] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:42.903146] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:42.903166] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:00:42.908112] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:00:42.908622] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:42.908722] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:00:42.928402] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:42.976600] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:42.976662] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:00:42.995882] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:00:42.995901] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:00:43.064842] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:43.101193] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:00:43.123463] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:00:43.167946] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:43.167980] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:00:43.167988] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:00:43.167995] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:00:43.168038] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:43.168064] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:43.168077] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:00:43.168083] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:00:43.168089] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:00:43.168126] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:43.168165] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:00:43.185263] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:00:43.185286] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:00:43.206766] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:43.206795] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:44.963950] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:00:45.476235] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:00:45.487135] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f3c1af6f47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f3c1b242182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f3c1bd3c265]))) 0-: received signum (15), shutting down [2016-03-15 15:00:45.499165] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:00:45.503233] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:00:45.503949] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:00:45.503966] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:00:45.504469] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:00:45.504485] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:00:45.504494] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:00:45.504540] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:00:45.504610] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:00:45.504623] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:00:47.099702] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:00:47.106379] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:47.117773] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:47.123655] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:47.123699] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:47.123750] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:47.123757] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:00:47.124321] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:47.124356] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:47.124364] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:00:47.124590] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:47.124632] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:47.124645] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:00:47.146719] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:00:47.147852] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:47.147908] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:00:47.160741] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:47.160787] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:00:47.212074] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:00:47.212109] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:00:47.417340] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:00:47.438075] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:47.438113] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:00:47.438121] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:00:47.438128] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:00:47.438166] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:00:47.458681] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:47.697831] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:47.745114] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:47.745170] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:00:47.806036] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:00:47.806068] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:00:47.850369] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:47.850405] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:49.203599] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:00:50.559845] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:00:50.570343] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa2b063447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa2b0907182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa2b1401265]))) 0-: received signum (15), shutting down [2016-03-15 15:00:50.581921] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:00:50.585333] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:00:50.585837] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:00:50.585849] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:00:50.586184] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:00:50.586195] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:00:50.586200] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:00:50.586235] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:00:50.586285] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:00:50.586293] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:00:52.306042] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:00:52.309438] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:52.312480] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:52.315867] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:52.315912] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:52.315965] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:52.315972] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:00:52.316560] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:52.316595] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:52.316603] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:00:52.316863] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:52.316896] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:52.316903] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:00:52.326554] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:00:52.326861] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:52.326921] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:00:52.344718] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:52.344784] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:00:52.365085] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:00:52.365124] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:00:52.471527] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:00:52.489544] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:52.489575] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:00:52.489583] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:00:52.489590] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:00:52.489632] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:00:52.542630] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:52.558535] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:52.558585] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:00:52.590939] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:00:52.590969] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:00:52.609835] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:52.609860] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:53.240079] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:54.382541] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:00:55.648434] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:00:55.659325] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fbfc6fa647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fbfc7279182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fbfc7d73265]))) 0-: received signum (15), shutting down [2016-03-15 15:00:55.670599] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:00:55.674587] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:00:55.675223] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:00:55.675238] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:00:55.675571] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:00:55.675583] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:00:55.675588] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:00:55.675623] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:00:55.675673] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:00:55.675688] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:00:57.436581] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:00:57.441102] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:57.445256] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:57.449560] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:00:57.449601] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:57.449651] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:57.449659] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:00:57.450245] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:57.450280] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:57.450288] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:00:57.450540] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:00:57.450578] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:00:57.450585] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:00:57.455006] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:00:57.455941] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:57.455994] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:00:57.466938] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:57.501633] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:57.501687] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:00:57.575990] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:00:57.576018] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:00:57.631017] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:57.631051] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:00:57.631058] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:00:57.631066] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:00:57.665393] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:00:57.796271] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:58.090631] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:00:58.424697] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:00:58.437765] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:58.437813] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:00:58.454960] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:00:58.454998] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:00:58.475789] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:00:58.475813] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:00:59.511795] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:01:00.732672] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:01:00.743977] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f349eaaf47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f349ed82182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f349f87c265]))) 0-: received signum (15), shutting down [2016-03-15 15:01:00.755425] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:01:00.759982] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:01:00.760709] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:01:00.760727] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:01:00.761249] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:01:00.761262] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:01:00.761268] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:01:00.761306] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:01:00.761358] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:01:00.761366] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:01:02.415583] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:01:02.441637] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:02.445139] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:02.451015] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:02.451059] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:02.451110] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:02.451118] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:01:02.451703] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:02.451751] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:02.451760] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:01:02.452023] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:02.452055] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:02.452062] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:01:02.456998] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:01:02.457289] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:02.457343] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:01:02.472525] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:02.551963] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:02.552036] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:01:02.574110] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:01:02.574147] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:01:02.593212] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:01:02.610016] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:02.610049] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:01:02.610056] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:01:02.610064] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:01:02.610108] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:02.610132] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:01:02.702398] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:02.703224] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:02.739734] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:02.739793] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:01:02.759019] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:01:02.759046] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:01:02.793616] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:02.793660] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:04.513513] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:01:05.818577] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:01:05.831908] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fe9f4b3847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fe9f4e0b182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fe9f5905265]))) 0-: received signum (15), shutting down [2016-03-15 15:01:05.843580] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:01:05.845815] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:01:05.846319] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:01:05.846331] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:01:05.846670] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:01:05.846680] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:01:05.846686] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:01:05.846721] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:01:05.846827] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:01:05.846843] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:01:07.475608] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:01:07.478597] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:07.481536] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:07.484632] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:07.484670] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:07.484721] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:07.484728] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:01:07.485196] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:07.485239] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:07.485251] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:01:07.485506] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:07.485549] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:07.485562] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:01:07.490539] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:01:07.491554] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:07.491610] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:01:07.577210] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:07.577276] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:01:07.618056] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:01:07.618084] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:01:07.679124] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:07.679159] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:01:07.679166] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:01:07.679173] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:01:07.679224] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:01:07.699488] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:01:07.743831] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:08.242804] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:08.701986] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:08.720113] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:08.720164] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:01:08.738620] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:01:08.738646] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:01:08.756407] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:08.775658] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:09.546926] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:01:10.911894] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:01:10.922647] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f6956b9847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f6956e6b182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f6957965265]))) 0-: received signum (15), shutting down [2016-03-15 15:01:10.934252] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:01:10.938171] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:01:10.938679] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:01:10.938691] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:01:10.939560] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:01:10.939579] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:01:10.939589] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:01:10.939640] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:01:10.939719] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:01:10.939734] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:01:12.465735] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:01:12.468833] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:12.471781] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:12.488285] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:12.488331] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:12.488380] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:12.488387] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:01:12.488955] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:12.488992] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:12.489000] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:01:12.489217] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:12.489253] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:12.489263] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:01:12.493594] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:01:12.494382] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:12.494437] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:01:12.508712] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:12.596496] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:12.596569] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:01:12.617284] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:01:12.617309] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:01:12.634582] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:01:12.653121] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:12.653158] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:01:12.675100] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:12.675131] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:01:12.675138] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:01:12.675145] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:01:12.675186] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:13.427673] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:13.442452] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:13.442500] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:01:13.460198] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:01:13.460236] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:01:13.475678] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:13.531924] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:14.550482] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:01:15.997622] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:01:16.009019] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f758c52e47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f758c801182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f758d2fb265]))) 0-: received signum (15), shutting down [2016-03-15 15:01:16.020750] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:01:16.024731] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:01:16.025332] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:01:16.025346] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:01:16.025690] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:01:16.025708] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:01:16.025715] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:01:16.025751] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:01:16.025800] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:01:16.025808] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:01:17.655464] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:01:17.658287] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:17.661103] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:17.664032] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:17.664070] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:17.664120] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:17.664127] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:01:17.664701] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:17.664743] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:17.664753] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:01:17.665012] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:17.665044] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:17.665051] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:01:17.669172] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:01:17.669727] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:17.669778] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:01:17.682585] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:17.760903] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:17.760965] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:01:17.777559] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:01:17.777592] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:01:17.807692] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:01:17.826314] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:17.826353] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:01:17.826361] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:01:17.826368] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:01:17.826409] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:01:17.850522] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:17.850597] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:17.850614] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:01:17.850625] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:01:17.850632] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:01:17.850964] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:17.872640] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:17.872966] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:17.873003] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:01:17.890184] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:01:17.890220] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:01:17.911305] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:18.026686] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:19.725634] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:01:21.082885] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:01:21.093366] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f439d4f547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f439d7c8182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f439e2c2265]))) 0-: received signum (15), shutting down [2016-03-15 15:01:21.104480] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:01:21.108380] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:01:21.108959] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:01:21.108972] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:01:21.109310] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:01:21.109321] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:01:21.109327] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:01:21.109370] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:01:21.109421] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:01:21.109429] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:01:22.783246] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:01:22.786224] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:22.788996] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:22.793011] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:22.793051] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:22.793128] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:22.793141] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:01:22.793697] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:22.793738] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:22.793748] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:01:22.793990] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:22.794028] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:22.794035] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:01:22.797638] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:01:22.798676] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:22.798753] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:01:22.951111] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:22.951191] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:01:22.977250] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:01:22.977269] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:01:22.994019] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:22.994043] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:01:23.010513] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:01:23.029409] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:23.029448] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:01:23.029456] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:01:23.029463] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:01:23.034845] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:23.245524] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:23.704865] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:23.720101] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:23.720149] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:01:23.752894] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:01:23.752926] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:01:23.768635] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:23.768664] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:24.853900] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:01:26.166873] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:01:26.177423] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f57a08e147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f57a0bb4182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f57a16ae265]))) 0-: received signum (15), shutting down [2016-03-15 15:01:26.188999] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:01:26.191211] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:01:26.191702] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:01:26.191713] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:01:26.192053] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:01:26.192064] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:01:26.192070] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:01:26.192105] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:01:26.192155] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:01:26.192163] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:01:27.909407] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:01:27.912724] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:27.915961] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:27.918989] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:27.919038] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:27.919088] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:27.919095] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:01:27.919621] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:27.919657] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:27.919665] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:01:27.919883] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:27.919924] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:27.919936] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:01:27.930320] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:01:27.931241] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:27.931296] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:01:27.945743] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:27.999482] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:27.999550] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:01:28.016513] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:01:28.016546] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:01:28.052437] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:28.052473] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:01:28.052481] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:01:28.052488] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:01:28.052534] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:01:28.122345] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:01:28.139837] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:28.430725] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:28.447907] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:28.447954] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:01:28.512889] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:01:28.512916] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:01:28.554785] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:28.554819] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:29.986671] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:01:31.251788] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:01:31.262488] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f2752cd047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f2752fa3182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f2753a9d265]))) 0-: received signum (15), shutting down [2016-03-15 15:01:31.273961] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:01:31.278038] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:01:31.278579] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:01:31.278591] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:01:31.279444] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:01:31.279462] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:01:31.279471] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:01:31.279521] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:01:31.279600] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:01:31.279614] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:01:32.947876] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:01:32.950860] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:32.953937] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:32.956758] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:32.956797] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:32.956846] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:32.956854] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:01:32.957432] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:32.957469] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:32.957477] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:01:32.957740] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:32.957776] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:32.957784] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:01:32.961972] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:01:32.962584] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:32.962634] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:01:33.009613] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:33.023412] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:33.038673] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:33.038722] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:01:33.103130] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:01:33.103168] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:01:33.170356] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:33.170391] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:01:33.170399] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:01:33.170406] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:01:33.170450] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:01:33.191428] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:01:33.208618] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:33.208672] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:01:33.224992] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:01:33.225023] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:01:33.281148] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:33.281189] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:33.281223] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:33.281258] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:01:33.281266] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:01:33.281273] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:01:35.017519] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:01:36.334179] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:01:36.345526] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f26c3e6247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f26c4135182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f26c4c2f265]))) 0-: received signum (15), shutting down [2016-03-15 15:01:36.357312] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:01:36.360351] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:01:36.361062] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:01:36.361080] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:01:36.361575] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:01:36.361591] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:01:36.361601] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:01:36.361733] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:01:36.361812] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:01:36.361827] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:01:38.072303] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:01:38.075346] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:38.078377] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:38.081389] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:38.081429] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:38.081477] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:38.081484] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:01:38.082025] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:38.082061] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:38.082068] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:01:38.082280] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:38.082316] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:38.082326] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:01:38.086519] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:01:38.087932] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:38.087986] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:01:38.155230] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:38.155295] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:01:38.243984] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:01:38.244018] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:01:38.287698] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:38.287732] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:01:38.287739] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:01:38.287747] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:01:38.311458] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:01:38.340659] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:38.341391] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:38.707129] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:01:38.754987] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:38.755024] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:01:38.755031] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:01:38.755038] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:01:38.789460] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:38.808714] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:38.808758] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:01:38.828738] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:01:38.828762] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:01:38.873162] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:38.898074] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:40.142794] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:01:41.427910] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:01:41.439219] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f2d1f94547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f2d1fc18182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f2d20712265]))) 0-: received signum (15), shutting down [2016-03-15 15:01:41.451249] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:01:41.453472] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:01:41.453976] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:01:41.453987] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:01:41.454328] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:01:41.454338] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:01:41.454344] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:01:41.454380] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:01:41.454430] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:01:41.454438] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:01:43.152187] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:01:43.157102] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:43.161478] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:43.166167] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:43.166207] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:43.166258] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:43.166266] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:01:43.166773] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:43.166813] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:43.166821] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:01:43.167044] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:43.167076] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:43.167114] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:01:43.170703] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:01:43.171404] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:43.171467] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:01:43.264152] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:43.264222] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:01:43.281792] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:01:43.281827] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:01:43.344270] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:01:43.360671] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:43.360710] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:01:43.413915] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:43.413956] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:01:43.413964] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:01:43.413971] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:01:43.423051] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:43.433495] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:43.455352] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:43.455398] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:01:43.503932] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:01:43.503959] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:01:43.531032] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:43.531063] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:44.249315] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:45.227074] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:01:46.519687] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:01:46.530258] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f6fa80fb47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f6fa83ce182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f6fa8ec8265]))) 0-: received signum (15), shutting down [2016-03-15 15:01:46.542716] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:01:46.545876] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:01:46.546589] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:01:46.546607] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:01:46.547302] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:01:46.547322] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:01:46.547332] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:01:46.547386] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:01:46.547468] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:01:46.547483] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:01:48.188601] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:01:48.191284] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:48.194450] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:48.198345] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:48.198384] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:48.198435] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:48.198443] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:01:48.199011] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:48.199049] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:48.199057] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:01:48.199279] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:48.199322] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:48.199335] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:01:48.203077] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:01:48.204131] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:48.204185] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:01:48.237054] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:48.307293] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:48.307372] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:01:48.327917] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:01:48.327960] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:01:48.344018] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:01:48.360110] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:48.360140] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:01:48.377012] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:48.377041] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:01:48.377048] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:01:48.377055] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:01:48.384101] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:49.434956] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:49.448985] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:49.449034] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:01:49.465709] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:01:49.465741] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:01:49.521571] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:49.535155] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:50.258971] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:01:51.607238] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:01:51.618115] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f93b444447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f93b4717182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f93b5211265]))) 0-: received signum (15), shutting down [2016-03-15 15:01:51.629937] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:01:51.633743] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:01:51.634251] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:01:51.634262] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:01:51.634598] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:01:51.634610] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:01:51.634615] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:01:51.634650] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:01:51.634707] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:01:51.634715] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:01:53.339038] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:01:53.342632] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:53.347294] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:53.350019] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:53.350061] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:53.350109] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:53.350117] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:01:53.350603] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:53.350641] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:53.350649] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:01:53.350901] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:53.350948] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:53.350961] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:01:53.355131] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:01:53.356354] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:53.356407] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:01:53.381092] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:53.416302] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:53.416362] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:01:53.436789] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:01:53.436815] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:01:53.453480] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:01:53.556958] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:53.557001] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:53.557025] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:01:53.557041] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:01:53.557048] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:01:53.710116] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:01:53.808436] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:53.808472] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:01:53.808480] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:01:53.808487] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:01:53.824667] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:53.840717] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:53.840760] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:01:53.859742] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:01:53.859767] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:01:53.877800] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:53.877826] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:55.411236] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:01:56.694498] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:01:56.705886] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f0eb4bfc47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f0eb4ecf182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f0eb59c9265]))) 0-: received signum (15), shutting down [2016-03-15 15:01:56.717416] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:01:56.721009] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:01:56.721515] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:01:56.721526] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:01:56.721864] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:01:56.721875] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:01:56.721881] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:01:56.721917] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:01:56.721967] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:01:56.721975] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:01:58.381018] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:01:58.384650] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:58.388623] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:58.391620] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:01:58.391659] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:58.391709] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:58.391716] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:01:58.392282] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:58.392317] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:58.392325] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:01:58.392544] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:01:58.392621] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:01:58.392635] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:01:58.397180] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:01:58.398572] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:58.398626] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:01:58.491075] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:01:58.510203] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:58.510257] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:01:58.526378] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:01:58.526396] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:01:58.563722] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:58.563763] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:58.563786] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:01:58.563793] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:01:58.563800] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:01:58.563836] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:01:58.604632] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:58.604662] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:01:58.604670] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:01:58.604677] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:01:58.605060] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:01:58.620317] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:58.620362] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:01:58.638084] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:01:58.638120] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:01:58.656069] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:01:58.656482] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:01:59.252147] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:00.453742] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:02:01.784417] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:02:01.795327] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f456a6b547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f456a988182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f456b482265]))) 0-: received signum (15), shutting down [2016-03-15 15:02:01.808391] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:02:01.812460] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:02:01.812984] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:02:01.812996] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:02:01.813364] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:02:01.813376] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:02:01.813381] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:02:01.813417] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:02:01.813468] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:02:01.813477] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:02:03.519445] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:02:03.522376] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:03.525287] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:03.528104] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:03.528157] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:03.528207] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:03.528214] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:02:03.528800] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:03.528836] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:03.528844] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:02:03.529068] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:03.529105] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:03.529112] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:02:03.532919] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:02:03.533680] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:03.533733] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:02:03.546995] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:03.648993] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:03.649085] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:02:03.667015] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:02:03.667048] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:02:03.698689] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:02:03.716487] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:02:03.752017] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:03.752055] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:02:03.752064] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:02:03.752071] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:02:03.752122] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:03.752145] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:04.437559] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:04.452453] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:04.452500] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:02:04.475186] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:02:04.475215] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:02:04.493796] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:04.497703] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:05.589525] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:02:06.871942] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:02:06.884680] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa1b760147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa1b78d4182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa1b83ce265]))) 0-: received signum (15), shutting down [2016-03-15 15:02:06.896843] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:02:06.899112] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:02:06.899625] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:02:06.899637] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:02:06.899975] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:02:06.899986] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:02:06.899991] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:02:06.900028] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:02:06.900078] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:02:06.900086] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:02:08.568666] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:02:08.571645] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:08.574453] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:08.577080] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:08.577121] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:08.577175] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:08.577183] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:02:08.577762] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:08.577805] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:08.577815] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:02:08.578057] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:08.578094] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:08.578113] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:02:08.582250] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:02:08.583097] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:08.583152] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:02:08.596170] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:08.654607] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:08.654669] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:02:08.692954] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:02:08.692989] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:02:08.713159] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:02:08.752602] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:08.752631] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:02:08.772163] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:08.772197] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:02:08.772207] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:02:08.772215] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:02:08.796640] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:08.796673] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:02:08.796680] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:02:08.796705] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:02:08.797512] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:08.815690] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:08.815735] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:02:08.838753] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:02:08.838779] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:02:08.857428] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:08.879933] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:10.639388] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:02:11.961197] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:02:11.971738] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f52b395847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f52b3c2b182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f52b4725265]))) 0-: received signum (15), shutting down [2016-03-15 15:02:11.983446] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:02:11.985612] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:02:11.986123] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:02:11.986135] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:02:11.986479] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:02:11.986491] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:02:11.986497] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:02:11.986532] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:02:11.986584] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:02:11.986592] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:02:13.554012] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:02:13.557256] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:13.560500] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:13.563180] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:13.563221] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:13.563271] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:13.563278] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:02:13.563852] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:13.563888] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:13.563895] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:02:13.564115] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:13.564159] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:13.564171] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:02:13.567696] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:02:13.569245] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:13.569301] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:02:13.640819] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:13.640886] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:02:13.656508] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:02:13.656543] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:02:13.678143] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:13.678168] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:02:13.694781] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:02:13.711467] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:13.711495] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:02:13.711502] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:02:13.711509] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:02:13.711550] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:14.254579] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:14.714704] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:14.744615] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:14.744668] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:02:14.813349] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:02:14.813377] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:02:14.830066] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:14.830102] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:15.624275] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:02:17.048425] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:02:17.059417] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fdfb4dcf47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fdfb50a2182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fdfb5b9c265]))) 0-: received signum (15), shutting down [2016-03-15 15:02:17.071069] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:02:17.075010] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:02:17.075693] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:02:17.075709] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:02:17.076070] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:02:17.076081] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:02:17.076087] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:02:17.076123] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:02:17.076173] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:02:17.076181] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:02:18.668668] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:02:18.671364] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:18.674050] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:18.677190] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:18.677230] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:18.677281] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:18.677288] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:02:18.677914] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:18.677949] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:18.677956] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:02:18.678188] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:18.678232] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:18.678243] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:02:18.681549] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:02:18.682734] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:18.682793] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:02:18.697330] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:18.778915] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:18.778975] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:02:18.795475] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:02:18.795509] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:02:18.846537] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:18.846566] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:02:18.866147] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:02:18.934251] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:18.934300] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:02:18.934308] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:02:18.934316] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:02:18.946106] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:19.440535] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:19.458704] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:19.458757] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:02:19.476245] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:02:19.476271] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:02:19.493742] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:19.497615] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:20.738196] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:02:22.133916] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:02:22.144528] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd86514c47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd86541f182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd865f19265]))) 0-: received signum (15), shutting down [2016-03-15 15:02:22.156020] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:02:22.160495] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:02:22.161199] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:02:22.161226] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:02:22.161686] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:02:22.161699] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:02:22.161704] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:02:22.161739] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:02:22.161791] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:02:22.161799] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:02:23.734106] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:02:23.737320] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:23.740175] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:23.743308] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:23.743346] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:23.743395] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:23.743402] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:02:23.743949] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:23.743985] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:23.743993] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:02:23.744247] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:23.744281] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:23.744288] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:02:23.748316] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:02:23.749445] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:23.749498] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:02:23.834568] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:23.850669] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:23.865341] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:23.865392] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:02:23.882363] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:02:23.882381] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:02:23.922884] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:23.922916] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:02:23.922924] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:02:23.922931] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:02:23.922974] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:02:23.957831] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:23.957898] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:02:23.973951] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:02:23.973984] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:02:23.990528] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:23.990555] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:02:24.007236] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:24.007264] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:02:24.007272] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:02:24.007279] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:02:24.007329] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:24.007346] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:25.805458] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:02:27.215991] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:02:27.226637] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f8c7b50347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f8c7b7d6182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f8c7c2d0265]))) 0-: received signum (15), shutting down [2016-03-15 15:02:27.238172] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:02:27.242305] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:02:27.243249] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:02:27.243268] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:02:27.243767] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:02:27.243784] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:02:27.243803] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:02:27.243855] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:02:27.243931] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:02:27.243945] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:02:28.891547] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:02:28.896807] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:28.899844] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:28.903552] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:28.903592] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:28.903646] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:28.903654] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:02:28.904223] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:28.904258] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:28.904266] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:02:28.904519] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:28.904555] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:28.904562] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:02:28.908984] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:02:28.909887] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:28.909941] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:02:28.979146] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:28.979219] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:02:28.994886] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:02:28.994918] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:02:29.027132] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:02:29.045460] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:29.045499] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:02:29.070291] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:29.070324] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:02:29.070332] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:02:29.070339] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:02:29.087280] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:29.257205] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:29.717895] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:29.739197] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:29.739252] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:02:29.761703] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:02:29.761725] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:02:29.780100] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:29.802633] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:30.966338] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:02:32.301193] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:02:32.312021] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f2aeed5b47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f2aef02e182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f2aefb28265]))) 0-: received signum (15), shutting down [2016-03-15 15:02:32.324431] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:02:32.326837] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:02:32.327345] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:02:32.327356] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:02:32.327696] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:02:32.327708] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:02:32.327714] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:02:32.327749] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:02:32.327800] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:02:32.327808] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:02:33.980655] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:02:33.983951] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:33.988615] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:33.992167] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:33.992209] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:33.992261] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:33.992269] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:02:33.992769] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:33.992807] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:33.992815] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:02:33.993039] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:33.993076] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:33.993088] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:02:33.997209] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:02:33.997919] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:33.997970] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:02:34.090856] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:34.090929] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:02:34.136681] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:02:34.136719] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:02:34.175972] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:02:34.212574] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:34.212612] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:02:34.212620] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:02:34.212627] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:02:34.212678] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:02:34.234464] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:34.234495] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:34.443414] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:34.458911] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:34.458958] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:02:34.480652] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:02:34.480681] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:02:34.499322] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:34.499350] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:35.258318] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:36.054632] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:02:37.388830] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:02:37.399897] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f26fdfdd47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f26fe2b0182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f26fedaa265]))) 0-: received signum (15), shutting down [2016-03-15 15:02:37.412247] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:02:37.416527] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:02:37.417100] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:02:37.417113] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:02:37.417446] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:02:37.417457] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:02:37.417463] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:02:37.417502] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:02:37.417554] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:02:37.417562] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:02:39.062113] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:02:39.065049] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:39.139710] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:39.143079] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:39.143121] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:39.143173] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:39.143180] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:02:39.143784] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:39.143826] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:39.143836] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:02:39.144087] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:39.144130] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:39.144137] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:02:39.148649] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:02:39.149188] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:39.149273] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:02:39.182061] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:39.182114] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:02:39.225297] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:02:39.225332] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:02:39.257623] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:39.319178] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:39.335325] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:02:39.370527] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:39.370558] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:02:39.370565] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:02:39.370572] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:02:39.452421] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:39.452501] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:02:39.474636] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:02:39.474663] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:02:39.496259] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:02:39.590027] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:39.590059] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:02:39.590076] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:02:39.590083] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:02:39.590133] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:39.590148] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:41.205812] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:02:42.474305] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:02:42.485464] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fee0a0fb47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fee0a3ce182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fee0aec8265]))) 0-: received signum (15), shutting down [2016-03-15 15:02:42.497025] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:02:42.501482] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:02:42.502003] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:02:42.502015] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:02:42.502494] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:02:42.502506] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:02:42.502512] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:02:42.502550] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:02:42.502600] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:02:42.502608] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:02:44.123913] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:02:44.128665] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:44.134396] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:44.140551] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:44.140592] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:44.140642] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:44.140649] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:02:44.141232] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:44.141275] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:44.141285] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:02:44.141544] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:44.141576] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:44.141583] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:02:44.145767] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:02:44.146632] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:44.146686] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:02:44.314157] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:44.314227] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:02:44.369867] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:02:44.369900] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:02:44.433620] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:02:44.506761] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:44.506798] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:02:44.506806] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:02:44.506813] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:02:44.506859] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:02:44.547131] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:44.547163] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:44.547762] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:44.720691] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:44.745139] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:44.745194] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:02:44.806316] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:02:44.806346] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:02:44.861113] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:44.861144] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:46.204014] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:02:47.559641] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:02:47.570919] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ff81b3cc47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ff81b69f182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ff81c199265]))) 0-: received signum (15), shutting down [2016-03-15 15:02:47.582314] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:02:47.584845] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:02:47.585352] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:02:47.585364] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:02:47.585774] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:02:47.585786] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:02:47.585792] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:02:47.585828] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:02:47.585879] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:02:47.585887] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:02:49.287648] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:02:49.291331] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:49.295045] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:49.298284] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:49.298322] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:49.298370] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:49.298377] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:02:49.298970] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:49.299007] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:49.299014] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:02:49.299306] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:49.299345] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:49.299353] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:02:49.304757] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:02:49.305038] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:49.305093] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:02:49.319296] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:49.319349] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:02:49.343821] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:02:49.343854] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:02:49.431010] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:02:49.485187] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:49.485225] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:02:49.485232] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:02:49.485240] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:02:49.485285] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:02:49.519407] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:49.544868] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:49.544928] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:02:49.631362] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:02:49.631400] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:02:49.660982] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:49.661012] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:50.260795] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:51.360003] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:02:52.647801] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:02:52.658872] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f4986f6147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f4987234182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f4987d2e265]))) 0-: received signum (15), shutting down [2016-03-15 15:02:52.670550] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:02:52.674406] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:02:52.675377] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:02:52.675397] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:02:52.675896] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:02:52.675914] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:02:52.675932] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:02:52.675983] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:02:52.676059] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:02:52.676072] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:02:54.259478] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:02:54.264560] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:54.268626] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:54.275102] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:54.275146] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:54.275195] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:54.275202] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:02:54.275685] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:54.275723] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:54.275730] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:02:54.275944] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:54.275987] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:54.276000] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:02:54.280253] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:02:54.281150] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:54.281211] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:02:54.296625] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:54.408361] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:54.408428] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:02:54.429693] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:02:54.429729] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:02:54.467805] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:02:54.485542] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:54.485571] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:02:54.485586] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:02:54.485594] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:02:54.485634] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:02:54.505660] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:55.447341] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:55.461839] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:55.461890] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:02:55.480437] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:02:55.480465] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:02:55.503620] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:55.545803] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:56.336604] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:02:57.732704] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:02:57.743824] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f1298f4147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f1299214182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f1299d0e265]))) 0-: received signum (15), shutting down [2016-03-15 15:02:57.755532] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:02:57.758847] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:02:57.759352] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:02:57.759364] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:02:57.759703] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:02:57.759714] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:02:57.759720] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:02:57.759758] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:02:57.759809] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:02:57.759817] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:02:59.452398] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:02:59.455671] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:59.458773] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:59.461833] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:02:59.461873] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:59.461931] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:59.461939] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:02:59.462558] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:59.462599] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:59.462609] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:02:59.462898] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:02:59.462931] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:02:59.462938] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:02:59.468321] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:02:59.468976] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:59.469033] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:02:59.482241] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:59.580927] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:59.580993] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:02:59.597954] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:02:59.597972] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:02:59.617331] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:59.617363] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:02:59.635132] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:02:59.655236] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:59.655267] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:02:59.655274] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:02:59.655281] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:02:59.663567] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:59.723676] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:02:59.736923] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:02:59.736968] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:02:59.753393] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:02:59.753425] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:02:59.787545] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:02:59.787579] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:01.524914] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:03:02.819822] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:03:02.830282] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f26a2b6747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f26a2e3a182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f26a3934265]))) 0-: received signum (15), shutting down [2016-03-15 15:03:02.842654] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:03:02.845711] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:03:02.846409] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:03:02.846426] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:03:02.847093] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:03:02.847112] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:03:02.847122] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:03:02.847275] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:03:02.847355] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:03:02.847369] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:03:04.548567] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:03:04.551981] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:04.554945] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:04.557826] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:04.557868] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:04.557921] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:04.557929] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:03:04.558492] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:04.558535] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:04.558545] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:03:04.558821] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:04.558857] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:04.558870] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:03:04.562574] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:03:04.563883] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:04.563939] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:03:04.641838] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:04.641903] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:03:04.659801] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:03:04.659834] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:03:04.698093] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:04.698135] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:03:04.698143] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:03:04.698149] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:03:04.698203] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:03:04.739789] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:03:04.758742] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:05.263743] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:05.725423] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:05.758904] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:05.758960] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:03:05.776581] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:03:05.776610] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:03:05.797286] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:05.825243] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:06.620707] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:03:07.907185] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:03:07.919552] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ff4c245c47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ff4c272f182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ff4c3229265]))) 0-: received signum (15), shutting down [2016-03-15 15:03:07.931720] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:03:07.936266] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:03:07.936808] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:03:07.936821] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:03:07.937161] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:03:07.937172] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:03:07.937177] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:03:07.937213] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:03:07.937263] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:03:07.937271] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:03:09.487342] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:03:09.490859] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:09.493855] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:09.496258] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:09.496298] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:09.496348] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:09.496355] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:03:09.497046] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:09.497089] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:09.497102] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:03:09.497422] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:09.497456] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:09.497463] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:03:09.501150] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:03:09.501892] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:09.501955] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:03:09.513166] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:09.607789] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:03:09.696628] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:03:09.722368] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:09.722448] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:03:09.749264] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:03:09.749300] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:03:09.768996] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:09.769024] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:09.769048] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:09.769068] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:03:09.769075] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:03:09.769082] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:03:10.450265] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:10.467389] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:10.467446] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:03:10.486212] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:03:10.486230] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:03:10.504558] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:10.532284] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:11.558118] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:03:12.994560] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:03:13.005251] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f062925c47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f062952f182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f062a029265]))) 0-: received signum (15), shutting down [2016-03-15 15:03:13.016654] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:03:13.020208] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:03:13.020722] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:03:13.020734] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:03:13.021078] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:03:13.021090] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:03:13.021095] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:03:13.021131] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:03:13.021181] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:03:13.021189] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:03:14.698787] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:03:14.701734] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:14.704946] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:14.707812] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:14.707853] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:14.707904] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:14.707911] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:03:14.708470] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:14.708511] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:14.708520] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:03:14.708738] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:14.708778] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:14.708790] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:03:14.714301] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:03:14.715006] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:14.715062] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:03:14.730349] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:14.730416] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:03:14.749567] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:03:14.749596] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:03:14.769124] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:14.803265] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:03:14.837498] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:14.837545] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:03:14.837553] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:03:14.837561] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:03:14.886490] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:03:14.905249] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:14.962441] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:14.962476] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:03:14.962485] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:03:14.962492] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:03:14.962880] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:14.962927] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:03:14.982118] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:03:14.982137] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:03:15.024718] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:15.024751] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:16.771461] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:03:18.080228] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:03:18.091763] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fe2a6f0747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fe2a71da182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fe2a7cd4265]))) 0-: received signum (15), shutting down [2016-03-15 15:03:18.103679] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:03:18.105985] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:03:18.106524] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:03:18.106536] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:03:18.107039] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:03:18.107052] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:03:18.107058] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:03:18.107101] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:03:18.107204] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:03:18.107225] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:03:19.743083] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:03:19.746302] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:19.750105] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:19.754069] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:19.754109] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:19.754156] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:19.754164] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:03:19.754723] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:19.754781] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:19.754790] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:03:19.755020] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:19.755054] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:19.755061] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:03:19.759524] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:03:19.760546] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:19.760599] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:03:19.825590] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:19.825654] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:03:19.846530] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:03:19.846568] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:03:19.910639] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:19.910678] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:03:19.910686] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:03:19.910693] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:03:19.945014] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:03:20.043286] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:20.266117] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:20.727211] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:03:20.749596] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:20.749628] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:03:20.749635] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:03:20.749642] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:03:20.766428] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:20.783304] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:20.783353] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:03:20.815547] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:03:20.815573] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:03:20.833514] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:20.839052] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:21.815525] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:03:23.169265] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:03:23.179912] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fe37632e47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fe376601182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fe3770fb265]))) 0-: received signum (15), shutting down [2016-03-15 15:03:23.191418] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:03:23.195376] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:03:23.195981] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:03:23.195995] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:03:23.196338] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:03:23.196349] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:03:23.196355] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:03:23.196390] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:03:23.196441] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:03:23.196456] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:03:24.876635] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:03:24.879676] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:24.882491] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:24.885206] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:24.885247] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:24.885295] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:24.885303] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:03:24.885896] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:24.885932] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:24.885939] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:03:24.886210] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:24.886243] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:24.886250] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:03:24.891030] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:03:24.892050] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:24.892100] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:03:24.904191] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:25.175654] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:25.175735] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:03:25.191693] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:03:25.191713] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:03:25.207272] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:25.207325] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:03:25.251584] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:25.251618] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:03:25.251626] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:03:25.251643] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:03:25.251694] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:03:25.268832] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:25.453406] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:25.512523] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:25.512586] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:03:25.529626] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:03:25.529652] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:03:25.546587] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:25.550972] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:26.947059] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:03:28.253209] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:03:28.264512] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f496f1b847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f496f48b182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f496ff85265]))) 0-: received signum (15), shutting down [2016-03-15 15:03:28.276092] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:03:28.280056] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:03:28.280670] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:03:28.280684] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:03:28.281032] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:03:28.281043] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:03:28.281049] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:03:28.281085] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:03:28.281136] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:03:28.281144] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:03:30.073851] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:03:30.078677] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:30.083677] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:30.090266] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:30.090336] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:30.090402] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:30.090412] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:03:30.091038] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:30.091074] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:30.091082] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:03:30.091304] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:30.091367] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:30.091380] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:03:30.095620] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:03:30.096677] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:30.096731] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:03:30.111312] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:30.126724] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:30.203352] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:30.203418] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:03:30.245749] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:03:30.245784] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:03:30.268514] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:03:30.285149] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:30.285179] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:03:30.285187] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:03:30.285193] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:03:30.315841] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:30.315896] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:03:30.335536] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:03:30.335568] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:03:30.359753] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:30.359793] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:03:30.381516] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:30.381543] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:03:30.381551] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:03:30.381557] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:03:30.381605] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:30.381626] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:32.151895] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:03:33.338247] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:03:33.348673] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fe17f4af47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fe17f782182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fe18027c265]))) 0-: received signum (15), shutting down [2016-03-15 15:03:33.361078] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:03:33.364560] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:03:33.365282] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:03:33.365299] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:03:33.365823] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:03:33.365839] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:03:33.365849] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:03:33.365901] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:03:33.365977] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:03:33.365990] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:03:35.060112] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:03:35.063054] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:35.066823] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:35.069933] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:35.069972] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:35.070023] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:35.070031] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:03:35.070551] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:35.070586] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:35.070594] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:03:35.070849] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:35.070895] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:35.070907] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:03:35.074702] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:03:35.075798] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:35.075853] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:03:35.153776] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:03:35.170365] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:35.170418] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:03:35.201014] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:03:35.201054] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:03:35.218587] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:35.218630] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:35.218652] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:03:35.218659] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:03:35.218666] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:03:35.268857] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:35.281814] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:03:35.730776] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:35.747206] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:35.747254] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:03:35.786197] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:03:35.786220] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:03:35.807604] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:35.827145] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:37.130842] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:03:38.422711] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:03:38.434404] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f611360f47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f61138e2182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f61143dc265]))) 0-: received signum (15), shutting down [2016-03-15 15:03:38.446834] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:03:38.451373] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:03:38.452322] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:03:38.452340] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:03:38.452963] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:03:38.452976] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:03:38.452981] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:03:38.453018] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:03:38.453072] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:03:38.453080] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:03:40.074397] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:03:40.079704] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:40.082725] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:40.088703] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:40.088745] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:40.088799] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:40.088806] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:03:40.089360] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:40.089395] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:40.089402] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:03:40.089628] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:40.089666] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:40.089678] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:03:40.095117] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:03:40.095522] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:40.095579] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:03:40.161948] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:40.162009] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:03:40.180853] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:03:40.180888] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:03:40.204946] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:03:40.247752] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:03:40.267290] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:40.267322] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:03:40.267329] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:03:40.267336] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:03:40.267379] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:40.456210] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:40.482866] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:40.482917] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:03:40.502797] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:03:40.502822] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:03:40.519682] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:40.535165] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:41.269799] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:42.151424] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:03:43.508525] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:03:43.519504] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f90daf6f47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f90db242182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f90dbd3c265]))) 0-: received signum (15), shutting down [2016-03-15 15:03:43.531440] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:03:43.535453] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:03:43.536068] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:03:43.536082] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:03:43.537458] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:03:43.537475] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:03:43.537484] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:03:43.537529] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:03:43.537601] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:03:43.537615] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:03:45.189055] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:03:45.192625] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:45.195906] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:45.201202] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:45.201242] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:45.201289] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:45.201297] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:03:45.201795] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:45.201831] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:45.201839] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:03:45.202053] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:45.202094] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:45.202107] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:03:45.205695] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:03:45.206761] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:45.206818] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:03:45.223306] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:45.294554] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:45.294621] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:03:45.313980] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:03:45.314023] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:03:45.336126] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:03:45.356976] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:45.357009] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:03:45.357016] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:03:45.357023] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:03:45.411293] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:46.456764] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:03:46.496208] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:46.496243] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:03:46.496250] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:03:46.496257] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:03:46.527034] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:46.543739] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:46.543785] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:03:46.565628] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:03:46.565657] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:03:46.585834] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:46.624324] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:47.261733] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:03:48.593834] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:03:48.604509] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f756d1e647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f756d4b9182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f756dfb3265]))) 0-: received signum (15), shutting down [2016-03-15 15:03:48.616169] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:03:48.621362] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:03:48.622071] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:03:48.622088] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:03:48.623195] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:03:48.623213] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:03:48.623223] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:03:48.623270] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:03:48.623324] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:03:48.623332] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:03:50.373586] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:03:50.377099] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:50.380569] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:50.383810] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:50.383850] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:50.383900] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:50.383907] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:03:50.384440] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:50.384476] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:50.384483] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:03:50.384710] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:50.384755] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:50.384768] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:03:50.389571] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:03:50.390542] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:50.390595] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:03:50.412076] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:50.471657] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:50.471729] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:03:50.513387] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:03:50.513418] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:03:50.554749] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:03:50.582936] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:50.582975] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:03:50.582982] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:03:50.582989] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:03:50.618768] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:50.757933] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:03:50.870817] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:50.870864] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:03:50.870872] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:03:50.870879] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:03:51.229343] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:51.243404] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:51.243455] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:03:51.261694] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:03:51.261721] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:03:51.282220] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:51.949684] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:52.447320] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:03:53.678070] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:03:53.689010] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f5b9d06847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f5b9d33b182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f5b9de35265]))) 0-: received signum (15), shutting down [2016-03-15 15:03:53.700511] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:03:53.703874] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:03:53.704377] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:03:53.704389] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:03:53.704731] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:03:53.704742] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:03:53.704748] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:03:53.704790] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:03:53.704844] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:03:53.704853] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:03:55.339106] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:03:55.348731] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:55.356124] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:55.364115] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:03:55.364160] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:55.364209] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:55.364217] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:03:55.364793] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:55.364827] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:55.364834] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:03:55.365092] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:03:55.365124] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:03:55.365131] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:03:55.371935] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:03:55.372745] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:55.372798] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:03:55.453142] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:55.453213] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:03:55.582423] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:03:55.582460] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:03:55.610642] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:03:55.632943] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:55.632981] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:03:55.632989] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:03:55.633005] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:03:55.661391] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:55.661422] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:03:55.725373] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:55.725412] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:03:55.725420] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:03:55.725427] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:03:55.725924] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:55.750610] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:55.750659] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:03:55.771071] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:03:55.771097] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:03:55.788141] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:03:55.870633] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:03:56.272489] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:03:57.429186] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:03:58.764271] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:03:58.775227] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f7d1494847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f7d14c1b182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f7d15715265]))) 0-: received signum (15), shutting down [2016-03-15 15:03:58.786537] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:03:58.789411] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:03:58.790132] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:03:58.790150] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:03:58.790662] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:03:58.790679] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:03:58.790689] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:03:58.790776] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:03:58.790858] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:03:58.790872] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:04:00.620181] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:04:00.624467] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:00.715346] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:00.720154] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:00.720200] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:00.720252] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:00.720260] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:04:00.720808] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:00.720845] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:00.720853] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:04:00.721070] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:00.721115] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:00.721127] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:04:00.726408] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:04:00.727368] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:00.727423] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:04:00.846055] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:00.960651] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:04:01.066269] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:01.066346] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:04:01.165688] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:04:01.165741] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:04:01.611337] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:01.611386] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:01.611411] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:04:01.611419] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:04:01.611426] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:04:01.611484] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:04:01.632777] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:01.632809] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:04:01.632821] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:04:01.632829] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:04:01.633518] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:01.730992] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:01.731055] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:04:01.855644] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:04:01.855668] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:04:01.878684] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:01.896069] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:02.783038] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:04:03.855219] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:04:03.865947] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fb0b4b7947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fb0b4e4c182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fb0b5946265]))) 0-: received signum (15), shutting down [2016-03-15 15:04:03.878263] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:04:03.882323] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:04:03.883291] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:04:03.883310] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:04:03.883818] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:04:03.883835] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:04:03.883845] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:04:03.883896] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:04:03.883974] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:04:03.883987] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:04:05.513883] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:04:05.518568] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:05.523497] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:05.528646] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:05.528689] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:05.528737] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:05.528745] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:04:05.529298] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:05.529335] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:05.529342] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:04:05.529564] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:05.529609] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:05.529620] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:04:05.535033] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:04:05.536068] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:05.536122] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:04:05.551797] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:05.636179] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:05.636264] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:04:05.654642] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:04:05.654660] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:04:05.716938] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:04:05.740092] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:05.740136] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:04:05.740151] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:04:05.740163] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:04:05.740218] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:05.740264] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:04:05.805088] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:05.805150] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:04:05.805157] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:04:05.805164] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:04:05.829004] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:05.843265] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:05.843313] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:04:05.861079] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:04:05.861114] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:04:05.879008] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:05.895610] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:07.592531] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:04:08.941282] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:04:08.953387] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f2ce5f1847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f2ce61eb182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f2ce6ce5265]))) 0-: received signum (15), shutting down [2016-03-15 15:04:08.964801] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:04:08.968760] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:04:08.969360] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:04:08.969374] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:04:08.969717] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:04:08.969728] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:04:08.969733] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:04:08.969769] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:04:08.969818] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:04:08.969826] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:04:10.675015] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:04:10.679867] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:10.683165] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:10.686099] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:10.686138] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:10.686211] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:10.686224] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:04:10.686848] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:10.686888] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:10.686897] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:04:10.687127] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:10.687167] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:10.687176] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:04:10.690940] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:04:10.692644] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:10.692693] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:04:10.835356] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:10.835429] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:04:10.860528] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:04:10.860549] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:04:10.904920] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:04:10.967418] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:10.967466] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:04:10.967473] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:04:10.967480] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:04:10.967537] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:04:10.985068] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:10.985092] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:11.275170] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:11.737756] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:11.800792] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:11.800858] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:04:11.816294] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:04:11.816319] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:04:11.845068] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:11.849915] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:12.747639] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:04:14.026813] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:04:14.037436] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fbd8806647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fbd88339182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fbd88e33265]))) 0-: received signum (15), shutting down [2016-03-15 15:04:14.049503] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:04:14.052565] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:04:14.053532] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:04:14.053550] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:04:14.054054] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:04:14.054071] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:04:14.054082] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:04:14.054135] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:04:14.054213] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:04:14.054227] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:04:15.622593] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:04:15.626019] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:15.629041] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:15.631947] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:15.631987] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:15.632039] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:15.632046] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:04:15.632598] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:15.632634] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:15.632642] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:04:15.632870] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:15.632915] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:15.632927] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:04:15.636695] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:04:15.637589] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:15.637642] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:04:15.650600] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:15.719134] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:04:15.736064] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:15.736117] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:04:15.757854] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:04:15.757880] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:04:15.819123] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:15.819171] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:15.819195] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:04:15.819202] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:04:15.819209] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:04:15.819249] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:04:15.899009] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:16.463264] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:16.476038] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:16.476085] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:04:16.523399] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:04:16.523428] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:04:16.541116] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:16.541145] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:17.693420] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:04:19.116471] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:04:19.127510] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f6540ecd47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f65411a0182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f6541c9a265]))) 0-: received signum (15), shutting down [2016-03-15 15:04:19.139468] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:04:19.144387] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:04:19.145033] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:04:19.145046] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:04:19.145384] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:04:19.145395] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:04:19.145401] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:04:19.145435] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:04:19.145486] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:04:19.145495] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:04:21.156090] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:04:21.159380] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:21.162549] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:21.165666] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:21.165706] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:21.165756] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:21.165763] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:04:21.166321] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:21.166356] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:21.166364] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:04:21.166611] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:21.166648] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:21.166656] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:04:21.171400] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:04:21.172114] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:21.172230] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:04:21.183818] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:21.183864] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:04:21.223013] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:04:21.223046] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:04:21.242620] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:21.379289] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:21.393227] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:04:21.483640] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:21.483676] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:04:21.483683] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:04:21.483690] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:04:21.530005] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:21.530068] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:04:21.550517] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:04:21.550538] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:04:21.568168] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:04:21.585706] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:21.585734] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:04:21.585741] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:04:21.585748] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:04:21.585806] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:21.585827] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:23.228199] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:04:24.202490] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:04:24.213135] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f5babfa747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f5bac27a182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f5bacd74265]))) 0-: received signum (15), shutting down [2016-03-15 15:04:24.225284] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:04:24.229109] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:04:24.229759] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:04:24.229772] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:04:24.230126] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:04:24.230142] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:04:24.230151] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:04:24.230199] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:04:24.230281] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:04:24.230295] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:04:26.008394] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:04:26.011876] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:26.015110] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:26.018470] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:26.018511] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:26.018561] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:26.018568] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:04:26.019194] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:26.019232] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:26.019239] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:04:26.019505] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:26.019537] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:26.019544] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:04:26.024306] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:04:26.025752] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:26.025804] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:04:26.111604] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:26.111672] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:04:26.130588] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:04:26.130615] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:04:26.149111] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:26.224217] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:04:26.242499] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:04:26.263156] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:26.263193] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:26.263214] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:04:26.263221] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:04:26.263228] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:04:26.277895] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:26.740627] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:26.755935] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:26.755988] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:04:26.778987] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:04:26.779015] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:04:26.807986] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:26.808017] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:28.081520] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:04:29.287190] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:04:29.298091] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fc9b527f47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fc9b5552182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fc9b604c265]))) 0-: received signum (15), shutting down [2016-03-15 15:04:29.310067] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:04:29.312329] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:04:29.312826] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:04:29.312837] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:04:29.313190] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:04:29.313202] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:04:29.313208] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:04:29.313253] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:04:29.313303] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:04:29.313311] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:04:30.967338] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:04:30.971069] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:30.974181] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:30.976909] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:30.976950] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:30.976998] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:30.977006] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:04:30.977548] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:30.977584] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:30.977592] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:04:30.977811] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:30.977847] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:30.977859] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:04:30.981873] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:04:30.983110] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:30.983165] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:04:31.020356] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:31.020420] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:04:31.035177] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:04:31.035212] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:04:31.086526] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:04:31.113831] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:31.113871] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:04:31.113878] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:04:31.113895] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:04:31.114070] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:04:31.149798] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:31.466309] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:31.482129] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:31.482179] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:04:31.498964] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:04:31.498992] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:04:31.519426] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:31.568791] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:32.278951] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:33.039219] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:04:34.375434] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:04:34.385999] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f57fccdf47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f57fcfb2182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f57fdaac265]))) 0-: received signum (15), shutting down [2016-03-15 15:04:34.397077] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:04:34.400995] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:04:34.401590] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:04:34.401604] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:04:34.401946] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:04:34.401957] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:04:34.401963] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:04:34.401998] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:04:34.402050] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:04:34.402057] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:04:35.993198] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:04:35.996380] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:36.000834] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:36.005034] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:36.005075] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:36.005138] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:36.005145] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:04:36.005722] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:36.005758] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:36.005765] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:04:36.005992] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:36.006036] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:36.006047] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:04:36.010379] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:04:36.011508] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:36.011565] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:04:36.024235] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:36.024284] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:04:36.038854] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:04:36.038873] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:04:36.054402] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:36.088132] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:36.107177] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:04:36.151172] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:36.151208] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:04:36.151215] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:04:36.151223] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:04:36.172839] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:36.172908] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:04:36.213495] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:04:36.213543] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:04:36.238653] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:04:36.262830] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:36.262861] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:04:36.262868] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:04:36.262875] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:04:36.262949] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:36.262966] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:38.066911] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:04:39.459592] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:04:39.470831] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd24f79a47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd24fa6d182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd250567265]))) 0-: received signum (15), shutting down [2016-03-15 15:04:39.482177] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:04:39.486237] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:04:39.487132] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:04:39.487152] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:04:39.487649] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:04:39.487667] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:04:39.487677] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:04:39.487730] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:04:39.487807] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:04:39.487821] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:04:41.051870] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:04:41.055281] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:41.058781] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:41.061704] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:41.061744] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:41.061796] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:41.061804] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:04:41.062374] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:41.062410] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:41.062418] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:04:41.062658] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:41.062695] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:41.062703] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:04:41.067329] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:04:41.068054] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:41.068105] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:04:41.168495] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:41.168562] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:04:41.194575] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:04:41.194593] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:04:41.236081] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:04:41.254764] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:41.254799] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:04:41.254808] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:04:41.254815] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:04:41.254856] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:04:41.291405] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:41.291541] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:41.292238] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:41.743730] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:41.761108] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:41.761162] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:04:41.784713] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:04:41.784746] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:04:41.821431] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:41.821475] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:43.124344] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:04:44.545091] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:04:44.556395] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f7a2deb447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f7a2e187182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f7a2ec81265]))) 0-: received signum (15), shutting down [2016-03-15 15:04:44.569198] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:04:44.571539] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:04:44.572053] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:04:44.572065] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:04:44.572407] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:04:44.572419] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:04:44.572424] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:04:44.572461] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:04:44.572511] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:04:44.572519] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:04:46.545524] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:04:46.550075] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:46.553201] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:46.557102] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:46.557144] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:46.557197] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:46.557205] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:04:46.557781] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:46.557824] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:46.557833] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:04:46.558082] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:46.558119] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:46.558127] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:04:46.562601] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:04:46.562968] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:46.563026] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:04:46.597842] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:46.597895] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:04:46.617127] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:04:46.617154] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:04:46.636598] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:46.663380] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:46.663431] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:04:46.681931] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:04:46.681956] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:04:46.724005] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:46.724041] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:04:46.724048] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:04:46.724055] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:04:46.744251] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:04:46.763600] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:04:46.785578] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:46.785620] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:46.785640] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:04:46.785646] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:04:46.785653] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:04:47.281502] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:48.619320] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:04:49.633031] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:04:49.643323] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f66a704447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f66a7317182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f66a7e11265]))) 0-: received signum (15), shutting down [2016-03-15 15:04:49.655623] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:04:49.660055] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:04:49.660590] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:04:49.660602] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:04:49.660938] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:04:49.660949] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:04:49.660955] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:04:49.660990] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:04:49.661041] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:04:49.661049] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:04:52.685935] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:04:52.690791] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:52.694864] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:52.697804] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:52.697842] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:52.697890] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:52.697898] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:04:52.698468] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:52.698504] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:52.698512] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:04:52.698788] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:52.698822] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:52.698829] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:04:52.716794] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:04:52.717632] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:52.717688] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:04:52.733604] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:54.582921] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:54.594785] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:54.594842] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:04:54.608959] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:04:54.608984] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:04:54.703971] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:54.704019] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:04:54.704032] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:04:54.704044] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:04:54.704124] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:04:54.724839] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:04:54.746295] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:54.746377] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:04:54.765455] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:04:54.765482] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:04:54.790070] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:54.790095] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:54.790119] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:54.790136] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:04:54.790143] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:04:54.790149] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:04:54.790501] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:04:54.790988] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:04:54.801903] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f98fab1d47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f98fadf0182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f98fb8ea265]))) 0-: received signum (15), shutting down [2016-03-15 15:04:54.813564] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:04:54.817072] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:04:54.817574] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:04:54.817586] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:04:54.817924] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:04:54.817935] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:04:54.817941] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:04:54.817976] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:04:54.818027] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:04:54.818034] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:04:56.350721] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:04:56.354564] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:56.357837] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:56.360938] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:04:56.360978] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:56.361026] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:56.361033] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:04:56.361614] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:56.361662] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:56.361675] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:04:56.361930] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:04:56.361963] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:04:56.361970] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:04:56.365997] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:56.366041] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:04:56.376693] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:56.433004] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:56.433074] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:04:56.448455] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:04:56.448493] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:04:56.463154] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:56.463179] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:04:56.479696] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:56.479728] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:04:56.479735] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:04:56.479742] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:04:56.497173] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:04:56.516077] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:56.746657] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:04:56.762086] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:56.762172] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:04:56.782039] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:04:56.782068] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:04:56.800691] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:04:56.800727] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:04:56.848038] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:04:58.903756] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:04:59.878570] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:04:59.889764] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f533508647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f5335359182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f5335e53265]))) 0-: received signum (15), shutting down [2016-03-15 15:04:59.901817] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:04:59.906003] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:04:59.906719] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:04:59.906979] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:04:59.907594] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:04:59.907611] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:04:59.907622] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:04:59.907676] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:04:59.907762] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:04:59.907776] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:05:01.627435] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:05:01.631503] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:01.635950] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:01.640276] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:01.640323] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:01.640376] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:01.640383] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:05:01.640963] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:01.641001] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:01.641009] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:05:01.641224] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:01.641261] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:01.641269] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:05:01.645224] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:05:01.646193] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:01.646249] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:05:01.730584] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:01.730674] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:05:01.755592] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:05:01.755623] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:05:01.775574] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:01.775607] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:05:01.799933] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:05:01.823308] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:01.823344] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:05:01.823362] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:05:01.823369] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:05:01.823417] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:02.284874] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:02.747813] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:02.769627] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:02.769680] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:05:02.789476] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:05:02.789504] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:05:02.829521] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:02.829551] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:03.702760] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:05:04.963953] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:05:04.974821] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fae61e0147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fae620d4182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fae62bce265]))) 0-: received signum (15), shutting down [2016-03-15 15:05:04.987100] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:05:04.991483] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:05:04.991987] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:05:04.991999] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:05:04.992343] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:05:04.992353] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:05:04.992359] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:05:04.992394] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:05:04.992445] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:05:04.992453] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:05:06.659022] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:05:06.663076] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:06.666702] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:06.669820] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:06.669862] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:06.669911] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:06.669930] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:05:06.670524] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:06.670568] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:06.670578] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:05:06.670867] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:06.670901] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:06.670908] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:05:06.675942] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:05:06.676663] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:06.676739] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:05:06.696973] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:06.774309] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:06.774372] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:05:06.815465] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:05:06.815499] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:05:06.849136] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:06.849166] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:05:06.849174] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:05:06.849181] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:05:06.849222] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:05:06.866087] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:05:06.900661] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:07.473200] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:07.502987] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:07.503042] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:05:07.523220] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:05:07.523246] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:05:07.579359] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:07.579398] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:08.732038] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:05:10.051307] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:05:10.063777] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7faa1195d47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7faa11c30182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7faa1272a265]))) 0-: received signum (15), shutting down [2016-03-15 15:05:10.076339] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:05:10.080639] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:05:10.081192] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:05:10.081204] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:05:10.081541] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:05:10.081551] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:05:10.081557] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:05:10.081592] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:05:10.081643] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:05:10.081650] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:05:11.687294] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:05:11.691320] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:11.695854] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:11.700581] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:11.700621] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:11.700669] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:11.700677] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:05:11.701275] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:11.701321] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:11.701330] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:05:11.701598] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:11.701630] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:11.701637] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:05:11.705681] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:05:11.706381] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:11.706437] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:05:11.727335] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:11.759638] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:05:11.834457] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:11.834514] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:05:11.856999] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:05:11.857021] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:05:11.878495] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:05:11.901268] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:11.901304] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:05:11.901312] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:05:11.901319] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:05:11.901365] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:11.901378] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:05:11.901389] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:05:11.901398] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:05:11.901432] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:11.901849] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:11.919263] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:11.919622] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:11.919663] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:05:11.939340] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:05:11.939373] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:05:11.958525] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:12.050731] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:13.763485] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:05:15.138111] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:05:15.148615] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f0de40d247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f0de43a5182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f0de4e9f265]))) 0-: received signum (15), shutting down [2016-03-15 15:05:15.160142] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:05:15.164208] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:05:15.164911] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:05:15.164928] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:05:15.165419] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:05:15.165435] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:05:15.165446] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:05:15.165496] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:05:15.165571] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:05:15.165585] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:05:16.714084] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:05:16.718234] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:16.721581] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:16.724680] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:16.724722] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:16.724770] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:16.724777] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:05:16.725298] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:16.725333] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:16.725341] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:05:16.725559] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:16.725596] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:16.725605] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:05:16.730449] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:05:16.731503] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:16.731558] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:05:16.829444] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:16.829512] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:05:16.847983] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:05:16.848010] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:05:16.885152] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:16.885182] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:05:16.928996] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:05:16.959375] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:16.959412] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:16.959434] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:05:16.959440] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:05:16.959447] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:05:17.286696] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:17.750594] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:17.776291] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:17.776343] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:05:17.799100] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:05:17.799131] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:05:17.820773] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:17.820803] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:18.787466] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:05:20.220745] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:05:20.232222] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fe82a78c47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fe82aa5f182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fe82b559265]))) 0-: received signum (15), shutting down [2016-03-15 15:05:20.244603] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:05:20.248509] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:05:20.249020] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:05:20.249031] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:05:20.249365] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:05:20.249376] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:05:20.249381] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:05:20.249416] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:05:20.249466] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:05:20.249474] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:05:21.807185] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:05:21.811537] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:21.815016] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:21.818797] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:21.818853] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:21.818906] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:21.818914] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:05:21.819472] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:21.819519] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:21.819529] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:05:21.819779] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:21.819815] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:21.819822] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:05:21.823913] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:05:21.824811] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:21.824864] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:05:21.854182] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:21.885942] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:21.886002] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:05:21.905213] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:05:21.905240] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:05:21.928971] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:05:21.950445] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:21.950476] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:05:21.950483] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:05:21.950490] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:05:21.950532] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:22.475599] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:05:22.528278] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:22.528325] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:05:22.528332] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:05:22.528339] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:05:22.555411] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:22.570211] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:22.570267] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:05:22.600570] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:05:22.600602] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:05:22.642690] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:22.645910] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:23.880817] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:05:25.307912] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:05:25.318531] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f2c8684747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f2c86b1a182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f2c87614265]))) 0-: received signum (15), shutting down [2016-03-15 15:05:25.330639] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:05:25.333090] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:05:25.333608] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:05:25.333623] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:05:25.333968] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:05:25.333979] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:05:25.333984] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:05:25.334020] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:05:25.334070] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:05:25.334078] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:05:26.992881] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:05:26.998393] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:27.001833] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:27.004921] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:27.004960] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:27.005010] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:27.005017] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:05:27.005610] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:27.005648] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:27.005656] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:05:27.005925] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:27.005958] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:27.005965] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:05:27.011976] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:05:27.012576] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:27.012632] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:05:27.029553] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:27.102083] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:27.145993] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:27.146074] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:05:27.190320] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:05:27.190358] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:05:27.270927] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:05:27.333744] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:27.333782] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:05:27.333790] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:05:27.333797] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:05:27.333842] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:05:27.362036] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:27.362063] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:05:27.362070] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:05:27.362077] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:05:27.362120] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:27.362145] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:27.362175] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:05:27.382552] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:05:27.382569] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:05:27.407451] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:27.407487] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:27.407501] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:29.069744] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:05:30.395741] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:05:30.406222] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f7ff308847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f7ff335b182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f7ff3e55265]))) 0-: received signum (15), shutting down [2016-03-15 15:05:30.418302] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:05:30.421771] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:05:30.422480] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:05:30.422497] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:05:30.423137] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:05:30.423156] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:05:30.423166] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:05:30.423220] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:05:30.423344] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:05:30.423359] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:05:32.078133] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:05:32.082122] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:32.087246] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:32.103572] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:32.103628] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:32.103681] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:32.103689] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:05:32.104293] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:32.104330] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:32.104338] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:05:32.104557] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:32.104633] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:32.104642] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:05:32.120507] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:05:32.121677] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:32.121737] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:05:32.269123] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:32.269188] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:05:32.293602] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:05:32.293623] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:05:32.315904] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:05:32.337935] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:32.337962] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:05:32.393369] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:32.393409] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:05:32.393416] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:05:32.393423] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:05:32.393459] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:32.393990] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:32.753807] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:32.767857] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:32.767910] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:05:32.787247] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:05:32.787273] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:05:32.807168] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:32.807200] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:34.178269] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:05:35.486212] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:05:35.497172] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f751bce547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f751bfb8182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f751cab2265]))) 0-: received signum (15), shutting down [2016-03-15 15:05:35.508824] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:05:35.512201] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:05:35.512905] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:05:35.512923] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:05:35.513446] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:05:35.513463] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:05:35.513473] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:05:35.513522] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:05:35.513596] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:05:35.513619] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:05:37.155461] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:05:37.158903] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:37.162243] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:37.165670] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:37.165729] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:37.165799] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:37.165811] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:05:37.166397] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:37.166435] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:37.166443] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:05:37.166665] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:37.166703] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:37.166716] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:05:37.170923] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:05:37.171768] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:37.171827] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:05:37.219925] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:37.220001] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:05:37.241442] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:05:37.241470] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:05:37.261245] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:05:37.281906] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:37.281940] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:05:37.281947] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:05:37.281954] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:05:37.312950] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:37.478736] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:05:37.537053] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:37.537101] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:05:37.537109] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:05:37.537116] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:05:37.609410] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:37.627477] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:37.627526] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:05:37.648348] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:05:37.648379] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:05:37.670119] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:38.168702] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:38.290506] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:39.228619] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:05:40.573610] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:05:40.584743] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f8de8b1647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f8de8de9182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f8de98e3265]))) 0-: received signum (15), shutting down [2016-03-15 15:05:40.595921] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:05:40.600464] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:05:40.601164] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:05:40.601181] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:05:40.601663] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:05:40.601675] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:05:40.601681] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:05:40.601716] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:05:40.601768] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:05:40.601776] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:05:42.180217] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:05:42.184400] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:42.187596] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:42.190629] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:42.190670] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:42.190721] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:42.190728] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:05:42.191281] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:42.191319] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:42.191327] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:05:42.191551] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:42.191587] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:42.191596] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:05:42.195404] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:05:42.196400] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:42.196455] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:05:42.211657] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:42.314337] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:42.314423] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:05:42.330711] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:05:42.330737] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:05:42.349863] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:05:42.367959] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:42.367992] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:05:42.368000] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:05:42.368007] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:05:42.388190] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:43.479616] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:05:43.552157] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:43.552205] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:05:43.552213] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:05:43.552220] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:05:43.634562] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:43.649123] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:43.649171] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:05:43.666796] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:05:43.666825] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:05:43.685762] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:43.805689] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:44.251276] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:05:45.659361] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:05:45.670382] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f886b1c447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f886b497182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f886bf91265]))) 0-: received signum (15), shutting down [2016-03-15 15:05:45.682114] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:05:45.685042] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:05:45.685761] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:05:45.685779] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:05:45.686297] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:05:45.686313] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:05:45.686323] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:05:45.686374] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:05:45.686449] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:05:45.686462] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:05:47.267559] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:05:47.272293] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:47.275515] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:47.280593] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:47.280632] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:47.280693] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:47.280701] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:05:47.281177] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:47.281215] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:47.281223] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:05:47.281436] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:47.281471] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:47.281482] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:05:47.284906] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:05:47.285978] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:47.286031] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:05:47.366822] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:47.366889] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:05:47.387420] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:05:47.387454] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:05:47.408660] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:05:47.454228] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:47.469797] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:47.469832] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:05:47.469839] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:05:47.469846] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:05:47.469885] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:47.755924] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:05:47.793956] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:47.793998] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:05:47.794017] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:05:47.794025] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:05:47.818702] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:47.845521] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:47.845572] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:05:47.861741] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:05:47.861777] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:05:47.879096] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:47.879130] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:49.342291] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:05:50.743498] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:05:50.756355] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f1ddc0b447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f1ddc387182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f1ddce81265]))) 0-: received signum (15), shutting down [2016-03-15 15:05:50.768151] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:05:50.772297] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:05:50.772845] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:05:50.772858] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:05:50.773208] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:05:50.773220] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:05:50.773226] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:05:50.773261] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:05:50.773312] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:05:50.773320] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:05:52.368280] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:05:52.374199] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:52.377730] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:52.380374] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:52.380417] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:52.380468] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:52.380476] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:05:52.381050] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:52.381085] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:52.381104] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:05:52.381329] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:52.381366] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:52.381373] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:05:52.385276] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:05:52.434090] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:05:52.493542] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:05:52.521955] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:52.521990] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:05:52.521998] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:05:52.522005] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:05:52.522322] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:05:52.630863] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:52.647854] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:52.647901] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:05:52.665248] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:05:52.665274] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:05:52.682080] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:52.747117] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:53.293013] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:53.757818] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:53.772889] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:53.772935] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:05:53.794263] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:05:53.794298] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:05:53.812781] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:53.858610] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:54.441659] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:05:55.830556] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:05:55.841280] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f1cdae8a47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f1cdb15d182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f1cdbc57265]))) 0-: received signum (15), shutting down [2016-03-15 15:05:55.853849] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:05:55.856042] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:05:55.858291] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:05:55.858309] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:05:55.858819] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:05:55.858837] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:05:55.858847] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:05:55.858898] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:05:55.858971] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:05:55.858985] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:05:57.461859] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:05:57.464976] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:57.467968] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:57.471099] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:05:57.471138] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:57.471189] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:57.471196] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:05:57.471758] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:57.471800] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:57.471808] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:05:57.472039] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:05:57.472084] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:05:57.472096] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:05:57.476648] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:05:57.477517] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:57.477570] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:05:57.510882] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:57.572472] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:05:57.592014] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:57.592069] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:05:57.611218] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:05:57.611245] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:05:57.649518] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:57.649567] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:57.649588] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:05:57.649595] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:05:57.649602] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:05:58.482560] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:05:58.519325] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:58.519366] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:05:58.519374] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:05:58.519381] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:05:58.551288] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:05:58.566949] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:58.567000] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:05:58.591435] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:05:58.591469] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:05:58.609616] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:05:58.609653] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:05:59.533492] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:06:00.917733] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:06:00.928820] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7feb127ed47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7feb12ac0182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7feb135ba265]))) 0-: received signum (15), shutting down [2016-03-15 15:06:00.940567] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:06:00.943631] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:06:00.944339] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:06:00.944356] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:06:00.944870] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:06:00.944887] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:06:00.944897] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:06:00.944950] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:06:00.945027] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:06:00.945041] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:06:02.549033] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:06:02.553216] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:02.556551] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:02.559580] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:02.559621] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:02.559673] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:02.559680] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:06:02.560289] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:02.560332] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:02.560341] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:06:02.560570] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:02.560601] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:02.560613] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:06:02.566121] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:06:02.567245] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:02.567305] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:06:02.587055] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:02.659975] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:02.660039] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:06:02.682478] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:06:02.682513] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:06:02.739491] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:06:02.761043] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:02.761077] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:06:02.761085] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:06:02.761092] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:06:02.761134] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:02.761156] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:06:02.781022] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:02.781854] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:02.795742] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:02.795789] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:06:02.815122] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:06:02.815148] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:06:02.832654] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:02.836986] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:04.624150] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:06:06.006105] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:06:06.016924] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f3d06d9d47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f3d07070182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f3d07b6a265]))) 0-: received signum (15), shutting down [2016-03-15 15:06:06.028938] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:06:06.032244] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:06:06.033579] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:06:06.033598] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:06:06.034310] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:06:06.034328] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:06:06.034339] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:06:06.034516] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:06:06.035352] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:06:06.035371] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:06:07.635036] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:06:07.638920] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:07.642340] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:07.645594] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:07.645635] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:07.645686] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:07.645693] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:06:07.646201] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:07.646236] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:07.646243] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:06:07.646457] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:07.646500] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:07.646513] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:06:07.650689] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:06:07.651930] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:07.651984] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:06:07.779327] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:07.779389] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:06:07.806693] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:06:07.806734] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:06:07.827207] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:07.827235] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:06:07.849534] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:06:07.873862] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:07.873896] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:06:07.873903] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:06:07.873910] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:06:07.877262] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:08.295757] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:08.760692] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:08.776448] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:08.776495] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:06:08.793612] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:06:08.793639] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:06:08.811238] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:08.811267] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:09.707154] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:06:11.090464] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:06:11.102770] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fb95a3d947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fb95a6ac182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fb95b1a6265]))) 0-: received signum (15), shutting down [2016-03-15 15:06:11.114398] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:06:11.118380] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:06:11.119397] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:06:11.119417] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:06:11.119917] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:06:11.119934] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:06:11.119944] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:06:11.120009] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:06:11.120085] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:06:11.120099] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:06:12.745669] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:06:12.750326] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:12.753980] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:12.757313] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:12.757377] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:12.757429] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:12.757436] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:06:12.757983] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:12.758019] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:12.758026] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:06:12.758508] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:12.758543] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:12.758550] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:06:12.762216] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:06:12.763150] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:12.763211] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:06:12.775382] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:12.828006] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:06:12.873554] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:12.873613] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:06:12.894365] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:06:12.894384] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:06:12.933299] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:12.933344] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:12.933375] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:06:12.933382] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:06:12.933389] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:06:13.128543] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:06:13.486217] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:13.526822] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:13.526876] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:06:13.555585] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:06:13.555611] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:06:13.609924] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:13.609959] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:14.818157] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:06:16.176259] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:06:16.187781] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f939b36347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f939b636182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f939c130265]))) 0-: received signum (15), shutting down [2016-03-15 15:06:16.198903] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:06:16.202700] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:06:16.204075] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:06:16.204094] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:06:16.204480] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:06:16.204492] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:06:16.204497] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:06:16.204534] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:06:16.204585] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:06:16.204593] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:06:17.941927] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:06:17.959307] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:17.964638] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:17.967831] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:17.967872] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:17.967922] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:17.967939] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:06:17.968520] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:17.968558] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:17.968566] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:06:17.968799] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:17.968842] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:17.968851] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:06:17.972780] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:06:17.973756] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:17.973812] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:06:17.999731] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:17.999793] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:06:18.029709] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:06:18.029743] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:06:18.048723] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:18.135422] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:18.158818] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:06:18.188868] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:18.188900] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:06:18.188907] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:06:18.188914] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:06:18.207978] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:18.208028] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:06:18.238772] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:06:18.238815] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:06:18.300744] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:06:18.321995] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:18.322118] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:18.322140] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:06:18.322147] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:06:18.322154] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:06:20.029202] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:06:21.260995] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:06:21.272216] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f039835047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f0398623182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f039911d265]))) 0-: received signum (15), shutting down [2016-03-15 15:06:21.283470] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:06:21.287358] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:06:21.287953] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:06:21.287967] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:06:21.288311] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:06:21.288323] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:06:21.288329] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:06:21.288364] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:06:21.288415] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:06:21.288426] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:06:22.885435] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:06:22.913979] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:22.918512] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:22.922203] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:22.922246] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:22.922297] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:22.922305] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:06:22.922864] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:22.922901] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:22.922909] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:06:22.923164] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:22.923200] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:22.923218] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:06:22.928314] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:06:22.929098] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:22.929147] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:06:23.021196] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:23.021263] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:06:23.041632] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:06:23.041665] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:06:23.061673] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:06:23.081394] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:23.081436] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:06:23.081443] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:06:23.081450] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:06:23.081509] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:06:23.104585] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:23.104609] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:23.298347] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:23.763655] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:23.779217] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:23.779264] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:06:23.797159] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:06:23.797195] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:06:23.818381] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:23.818415] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:24.984627] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:06:26.345926] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:06:26.356978] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f917be8f47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f917c162182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f917cc5c265]))) 0-: received signum (15), shutting down [2016-03-15 15:06:26.368857] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:06:26.371856] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:06:26.372565] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:06:26.372582] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:06:26.373085] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:06:26.373101] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:06:26.373111] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:06:26.373162] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:06:26.373236] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:06:26.373251] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:06:27.911656] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:06:27.914469] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:27.917617] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:27.920357] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:27.920397] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:27.920445] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:27.920453] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:06:27.921030] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:27.921072] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:27.921081] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:06:27.921345] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:27.921378] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:27.921385] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:06:27.925564] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:06:27.926178] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:27.926230] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:06:28.007635] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:28.007707] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:06:28.025625] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:06:28.025658] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:06:28.042461] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:06:28.077652] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:28.077687] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:06:28.077694] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:06:28.077701] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:06:28.077743] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:06:28.118516] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:28.118549] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:28.489089] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:28.504564] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:28.504617] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:06:28.529100] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:06:28.529127] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:06:28.547130] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:28.568140] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:29.299325] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:29.981341] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:06:31.435236] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:06:31.446203] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa4b70d347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa4b73a6182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa4b7ea0265]))) 0-: received signum (15), shutting down [2016-03-15 15:06:31.458393] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:06:31.462406] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:06:31.463408] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:06:31.463428] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:06:31.464199] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:06:31.464217] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:06:31.464226] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:06:31.464277] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:06:31.464354] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:06:31.464368] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:06:33.038835] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:06:33.043034] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:33.046840] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:33.050132] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:33.050172] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:33.050223] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:33.050231] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:06:33.050725] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:33.050783] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:33.050792] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:06:33.051009] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:33.051045] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:33.051056] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:06:33.054929] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:06:33.055798] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:33.055850] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:06:33.070290] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:33.091433] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:33.120856] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:33.120924] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:06:33.136530] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:06:33.136564] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:06:33.186121] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:33.186190] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:06:33.204738] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:06:33.204771] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:06:33.221437] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:06:33.237024] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:33.237054] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:06:33.237062] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:06:33.237069] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:06:33.237113] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:33.405367] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:06:33.481516] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:33.481560] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:06:33.481568] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:06:33.481575] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:06:33.499660] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:33.499705] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:35.111820] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:06:36.519660] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:06:36.530228] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f1e00f6647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f1e01239182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f1e01d33265]))) 0-: received signum (15), shutting down [2016-03-15 15:06:36.542403] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:06:36.546301] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:06:36.547265] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:06:36.547294] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:06:36.547804] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:06:36.547821] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:06:36.547830] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:06:36.547881] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:06:36.547956] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:06:36.547970] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:06:38.162961] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:06:38.166684] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:38.169832] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:38.172634] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:38.172675] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:38.172725] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:38.172732] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:06:38.173220] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:38.173257] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:38.173265] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:06:38.173486] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:38.173522] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:38.173530] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:06:38.177127] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:06:38.177926] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:38.177978] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:06:38.253007] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:38.253084] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:06:38.269458] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:06:38.269476] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:06:38.301844] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:38.301873] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:06:38.330603] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:06:38.345241] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:38.345287] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:38.345309] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:06:38.345317] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:06:38.345324] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:06:38.345997] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:38.766839] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:38.810327] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:38.810406] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:06:38.835097] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:06:38.835135] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:06:38.852939] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:38.852967] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:40.233210] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:06:41.604986] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:06:41.616409] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f1e6320747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f1e634da182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f1e63fd4265]))) 0-: received signum (15), shutting down [2016-03-15 15:06:41.627773] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:06:41.631802] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:06:41.632512] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:06:41.632529] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:06:41.633040] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:06:41.633058] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:06:41.633068] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:06:41.633118] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:06:41.633197] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:06:41.633211] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:06:43.172661] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:06:43.184207] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:43.187175] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:43.189816] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:43.189857] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:43.189908] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:43.189916] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:06:43.190490] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:43.190531] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:43.190541] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:06:43.190817] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:43.190851] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:43.190858] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:06:43.194571] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:06:43.195475] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:43.195531] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:06:43.241499] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:06:43.289362] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:43.289432] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:06:43.306182] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:06:43.306201] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:06:43.339728] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:43.339780] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:43.339800] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:06:43.339807] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:06:43.339814] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:06:43.491329] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:06:43.537087] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:43.537129] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:06:43.537137] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:06:43.537144] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:06:43.560485] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:43.574653] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:43.574701] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:06:43.592751] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:06:43.592783] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:06:43.608339] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:43.650521] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:44.302102] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:45.250400] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:06:46.690189] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:06:46.700910] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f602815647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f6028429182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f6028f23265]))) 0-: received signum (15), shutting down [2016-03-15 15:06:46.712452] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:06:46.715927] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:06:46.716432] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:06:46.716443] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:06:46.716782] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:06:46.716794] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:06:46.716799] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:06:46.716838] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:06:46.716889] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:06:46.716897] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:06:48.278978] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:06:48.282437] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:48.286523] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:48.289307] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:48.289347] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:48.289396] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:48.289404] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:06:48.289969] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:48.290007] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:48.290014] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:06:48.290238] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:48.290270] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:48.290276] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:06:48.294192] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:06:48.295323] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:48.295377] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:06:48.465877] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:48.622622] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:06:48.641066] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:48.641127] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:06:48.657361] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:06:48.657389] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:06:48.676228] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:06:48.718234] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:48.718288] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:48.718314] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:06:48.718321] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:06:48.718328] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:06:48.718388] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:49.493236] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:49.520322] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:49.520379] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:06:49.558042] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:06:49.558066] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:06:49.586320] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:49.636229] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:50.351586] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:06:51.779903] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:06:51.790957] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f659768b47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f659795e182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f6598458265]))) 0-: received signum (15), shutting down [2016-03-15 15:06:51.803281] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:06:51.807205] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:06:51.807869] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:06:51.807883] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:06:51.808230] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:06:51.808242] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:06:51.808248] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:06:51.808284] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:06:51.808334] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:06:51.808342] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:06:53.341896] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:06:53.345686] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:53.348835] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:53.351798] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:53.351837] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:53.351883] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:53.351891] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:06:53.352426] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:53.352463] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:53.352470] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:06:53.352684] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:53.352760] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:53.352768] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:06:53.356520] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:06:53.357251] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:53.357301] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:06:53.372374] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:53.450836] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:53.450902] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:06:53.476153] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:06:53.476179] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:06:53.518224] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:06:53.561920] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:53.561960] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:06:53.561967] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:06:53.561974] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:06:53.562035] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:06:53.580169] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:53.580197] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:53.769732] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:53.782650] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:53.782698] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:06:53.799520] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:06:53.799545] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:06:53.829392] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:53.829423] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:55.412819] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:06:56.865906] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:06:56.876438] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f974825f47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f9748532182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f974902c265]))) 0-: received signum (15), shutting down [2016-03-15 15:06:56.888423] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:06:56.891413] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:06:56.892123] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:06:56.892141] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:06:56.892650] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:06:56.892668] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:06:56.892678] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:06:56.892732] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:06:56.892995] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:06:56.893011] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:06:58.455159] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:06:58.458000] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:58.460839] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:58.463777] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:06:58.463817] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:58.463867] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:58.463874] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:06:58.464416] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:58.464452] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:58.464459] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:06:58.464677] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:06:58.464717] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:06:58.464728] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:06:58.476973] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:06:58.509722] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:06:58.564455] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:06:58.581030] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:58.581061] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:06:58.581069] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:06:58.581076] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:06:58.581117] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:06:58.625980] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:58.643071] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:58.643118] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:06:58.661366] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:06:58.661399] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:06:58.679891] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:58.679921] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:59.304830] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:59.770777] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:06:59.811401] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:06:59.811465] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:06:59.841374] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:06:59.841401] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:06:59.856674] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:06:59.877860] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:00.533423] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:07:01.957901] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:07:01.969335] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa3710f547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa3713c8182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa371ec2265]))) 0-: received signum (15), shutting down [2016-03-15 15:07:01.982040] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:07:01.985931] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:07:01.986443] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:07:01.986455] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:07:01.987922] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:07:01.987937] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:07:01.987943] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:07:01.987983] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:07:01.988035] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:07:01.988044] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:07:03.617285] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:07:03.620876] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:03.626841] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:03.630111] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:03.630165] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:03.630217] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:03.630224] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:07:03.630801] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:03.630842] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:03.630849] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:07:03.631074] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:03.631111] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:03.631119] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:07:03.637135] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:07:03.637990] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:03.638044] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:07:03.651746] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:03.881895] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:03.881971] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:07:03.931759] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:07:03.931799] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:07:03.948692] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:03.948721] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:07:03.948729] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:07:03.948736] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:07:03.948781] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:07:03.965409] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:07:03.992568] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:03.992605] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:04.495990] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:04.509685] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:04.509735] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:07:04.531549] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:07:04.531571] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:07:04.552317] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:04.646269] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:05.694150] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:07:07.048836] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:07:07.059531] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fafcaa8747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fafcad5a182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fafcb854265]))) 0-: received signum (15), shutting down [2016-03-15 15:07:07.071202] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:07:07.074635] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:07:07.075835] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:07:07.075850] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:07:07.076189] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:07:07.076200] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:07:07.076214] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:07:07.076250] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:07:07.076300] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:07:07.076309] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:07:08.641301] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:07:08.644221] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:08.647160] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:08.651132] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:08.651175] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:08.651225] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:08.651232] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:07:08.651795] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:08.651832] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:08.651840] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:07:08.652061] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:08.652124] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:08.652137] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:07:08.656290] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:07:08.657361] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:08.657449] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:07:08.682878] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:08.768322] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:08.768387] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:07:08.793725] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:07:08.793752] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:07:08.829808] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:07:08.846915] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:08.846943] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:07:08.863431] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:08.863466] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:07:08.863474] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:07:08.863481] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:07:08.863526] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:08.863539] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:07:08.863545] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:07:08.863551] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:07:08.864125] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:08.876303] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:08.876721] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:08.876757] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:07:08.893991] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:07:08.894011] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:07:08.910871] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:08.912198] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:10.712617] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:07:12.135965] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:07:12.149034] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fc2cebdd47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fc2ceeb0182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fc2cf9aa265]))) 0-: received signum (15), shutting down [2016-03-15 15:07:12.162232] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:07:12.165568] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:07:12.166077] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:07:12.166088] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:07:12.166439] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:07:12.166450] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:07:12.166456] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:07:12.166492] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:07:12.166550] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:07:12.166558] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:07:13.762476] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:07:13.774800] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:13.779224] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:13.782304] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:13.782346] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:13.782398] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:13.782405] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:07:13.783038] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:13.783076] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:13.783085] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:07:13.783306] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:13.783352] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:13.783364] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:07:13.787103] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:07:13.788317] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:13.788369] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:07:13.874606] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:13.874668] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:07:13.890672] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:07:13.890689] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:07:13.955982] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:07:13.973150] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:13.973186] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:07:13.973194] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:07:13.973201] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:07:13.973253] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:13.973276] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:13.973292] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:07:14.307573] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:14.773853] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:14.789163] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:14.789210] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:07:14.814402] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:07:14.814435] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:07:14.834256] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:14.899909] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:15.842829] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:07:17.225051] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:07:17.235802] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd078e1b47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd0790ee182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd079be8265]))) 0-: received signum (15), shutting down [2016-03-15 15:07:17.248020] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:07:17.250151] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:07:17.250655] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:07:17.250667] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:07:17.251038] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:07:17.251050] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:07:17.251056] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:07:17.251092] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:07:17.251143] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:07:17.251151] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:07:18.820936] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:07:18.824005] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:18.826968] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:18.834630] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:18.834669] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:18.834730] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:18.834756] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:07:18.835267] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:18.835305] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:18.835313] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:07:18.835536] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:18.835573] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:18.835585] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:07:18.840361] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:07:18.841129] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:18.841181] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:07:18.857242] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:18.906098] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:07:18.934890] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:18.934950] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:07:18.969225] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:07:18.969253] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:07:19.021693] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:19.021734] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:19.021756] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:07:19.021763] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:07:19.021770] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:07:19.137957] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:07:19.498762] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:19.515548] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:19.515606] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:07:19.537454] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:07:19.537481] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:07:19.558908] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:19.632387] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:20.897462] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:07:22.314404] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:07:22.326233] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fe3c486947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fe3c4b3c182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fe3c5636265]))) 0-: received signum (15), shutting down [2016-03-15 15:07:22.338252] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:07:22.341410] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:07:22.341909] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:07:22.341921] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:07:22.342258] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:07:22.342269] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:07:22.342274] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:07:22.342309] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:07:22.342360] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:07:22.342368] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:07:23.957421] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:07:23.960799] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:23.964444] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:23.967463] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:23.967502] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:23.967548] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:23.967555] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:07:23.968082] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:23.968119] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:23.968126] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:07:23.968352] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:23.968390] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:23.968399] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:07:23.972071] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:07:23.972785] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:23.972835] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:07:23.985220] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:24.035087] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:24.047453] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:07:24.065548] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:24.065576] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:07:24.065584] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:07:24.065591] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:07:24.082301] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:24.082357] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:07:24.102209] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:07:24.102247] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:07:24.141168] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:24.141218] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:24.141272] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:07:24.166853] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:07:24.166879] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:07:24.182672] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:07:24.200481] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:24.200506] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:07:24.200513] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:07:24.200520] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:07:24.200576] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:24.200596] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:24.200609] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:26.029599] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:07:27.401878] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:07:27.412525] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f5fdda8c47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f5fddd5f182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f5fde859265]))) 0-: received signum (15), shutting down [2016-03-15 15:07:27.424610] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:07:27.426849] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:07:27.427349] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:07:27.427361] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:07:27.427698] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:07:27.427710] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:07:27.427716] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:07:27.427752] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:07:27.427803] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:07:27.427811] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:07:28.992309] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:07:28.995389] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:28.998344] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:29.001207] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:29.001247] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:29.001297] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:29.001304] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:07:29.001853] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:29.001888] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:29.001896] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:07:29.002132] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:29.002170] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:29.002178] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:07:29.006489] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:07:29.007724] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:29.007780] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:07:29.067264] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:29.067335] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:07:29.083744] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:07:29.083778] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:07:29.151579] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:29.151616] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:07:29.151623] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:07:29.151630] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:07:29.151676] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:07:29.169723] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:07:29.187018] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:29.310331] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:29.776811] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:29.823194] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:29.823252] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:07:29.863145] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:07:29.863167] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:07:29.879970] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:29.880001] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:31.063642] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:07:32.489457] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:07:32.500180] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fb06c6a547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fb06c978182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fb06d472265]))) 0-: received signum (15), shutting down [2016-03-15 15:07:32.512163] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:07:32.516132] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:07:32.516730] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:07:32.516743] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:07:32.517082] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:07:32.517093] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:07:32.517098] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:07:32.517133] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:07:32.517188] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:07:32.517196] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:07:34.043876] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:07:34.047408] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:34.050379] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:34.078484] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:34.078534] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:34.078590] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:34.078598] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:07:34.079224] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:34.079262] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:34.079270] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:07:34.079502] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:34.079543] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:34.079556] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:07:34.083906] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:07:34.084931] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:34.084987] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:07:34.098877] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:34.098925] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:07:34.119151] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:07:34.119177] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:07:34.238025] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:07:34.255121] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:34.255152] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:07:34.255160] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:07:34.255167] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:07:34.255210] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:07:34.501672] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:34.516911] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:34.516958] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:07:34.579438] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:07:34.579475] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:07:34.599067] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:34.622176] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:35.311387] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:36.140105] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:07:37.574502] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:07:37.585480] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f7402cfe47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f7402fd1182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f7403acb265]))) 0-: received signum (15), shutting down [2016-03-15 15:07:37.597489] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:07:37.600487] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:07:37.601179] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:07:37.601197] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:07:37.601713] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:07:37.601730] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:07:37.601741] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:07:37.601800] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:07:37.601887] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:07:37.601902] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:07:39.181351] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:07:39.184301] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:39.187147] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:39.190183] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:39.190223] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:39.190269] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:39.190277] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:07:39.190793] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:39.190827] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:39.190835] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:07:39.191049] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:39.191091] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:39.191103] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:07:39.195512] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:07:39.196517] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:39.196570] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:07:39.209625] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:39.281608] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:07:39.297528] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:39.297583] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:07:39.313466] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:07:39.313492] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:07:39.336374] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:39.336417] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:39.336439] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:07:39.336454] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:07:39.336461] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:07:39.336501] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:07:39.386536] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:40.502914] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:40.543387] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:40.543442] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:07:40.562141] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:07:40.562161] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:07:40.578739] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:40.646957] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:41.251442] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:07:42.663640] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:07:42.674466] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f33fb4e047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f33fb7b3182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f33fc2ad265]))) 0-: received signum (15), shutting down [2016-03-15 15:07:42.685628] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:07:42.688976] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:07:42.689479] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:07:42.689491] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:07:42.689824] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:07:42.689835] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:07:42.689844] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:07:42.689879] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:07:42.689929] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:07:42.689936] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:07:44.237801] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:07:44.241714] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:44.244642] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:44.247493] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:44.247532] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:44.247587] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:44.247594] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:07:44.248144] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:44.248179] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:44.248187] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:07:44.248408] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:44.248445] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:44.248457] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:07:44.255383] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:07:44.256100] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:44.256155] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:07:44.362650] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:44.362723] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:07:44.378475] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:07:44.378496] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:07:44.410872] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:44.410909] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:07:44.436579] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:07:44.452100] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:44.452178] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:44.452205] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:07:44.452213] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:07:44.452220] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:07:44.452464] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:44.779879] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:44.794110] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:44.794156] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:07:44.837438] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:07:44.837469] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:07:44.860936] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:44.860965] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:46.311906] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:07:47.750800] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:07:47.761572] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f75a429c47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f75a456f182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f75a5069265]))) 0-: received signum (15), shutting down [2016-03-15 15:07:47.772972] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:07:47.776627] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:07:47.777197] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:07:47.777214] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:07:47.777596] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:07:47.777609] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:07:47.777615] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:07:47.777652] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:07:47.777703] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:07:47.777711] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:07:49.353595] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:07:49.357594] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:49.361992] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:49.371622] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:49.371663] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:49.371711] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:49.371718] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:07:49.372293] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:49.372337] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:49.372347] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:07:49.372603] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:49.372639] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:49.372646] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:07:49.376207] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:07:49.376994] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:49.377071] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:07:49.408989] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:49.409039] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:07:49.427226] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:07:49.427258] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:07:49.504097] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:07:49.559023] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:07:49.617725] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:49.617769] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:07:49.617776] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:07:49.617783] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:07:49.617829] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:49.617847] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:07:49.617854] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:07:49.617860] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:07:49.618252] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:49.695725] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:49.695783] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:07:49.743488] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:07:49.743516] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:07:49.764570] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:49.764616] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:50.313976] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:51.432661] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:07:52.837389] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:07:52.848024] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f4dca0fe47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f4dca3d1182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f4dcaecb265]))) 0-: received signum (15), shutting down [2016-03-15 15:07:52.859869] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:07:52.862242] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:07:52.862777] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:07:52.862791] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:07:52.863131] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:07:52.863142] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:07:52.863148] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:07:52.863183] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:07:52.863234] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:07:52.863242] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:07:54.458127] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:07:54.460782] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:54.463467] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:54.466232] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:54.466270] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:54.466319] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:54.466326] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:07:54.466864] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:54.466900] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:54.466908] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:07:54.467131] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:54.467167] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:54.467176] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:07:54.470872] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:07:54.471713] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:54.471768] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:07:54.491885] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:54.587865] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:54.587924] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:07:54.603572] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:07:54.603597] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:07:54.621500] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:07:54.636321] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:54.636354] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:07:54.636361] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:07:54.636368] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:07:54.636422] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:07:54.652987] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:54.653012] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:55.505861] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:55.641186] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:55.641245] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:07:55.656385] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:07:55.656410] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:07:55.673497] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:55.728415] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:56.528360] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:07:57.925176] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:07:57.935875] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ff9ec6f647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ff9ec9c9182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ff9ed4c3265]))) 0-: received signum (15), shutting down [2016-03-15 15:07:57.947087] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:07:57.951664] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:07:57.952338] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:07:57.952352] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:07:57.952692] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:07:57.952704] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:07:57.952710] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:07:57.952744] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:07:57.952795] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:07:57.952803] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:07:59.582244] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:07:59.585539] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:59.588980] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:59.592312] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:07:59.592352] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:59.592407] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:59.592415] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:07:59.592992] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:59.593034] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:59.593043] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:07:59.593306] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:07:59.593338] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:07:59.593344] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:07:59.597783] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:07:59.598384] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:59.598449] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:07:59.610638] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:59.681009] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:59.681079] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:07:59.701966] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:07:59.701992] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:07:59.735105] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:59.735148] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:07:59.753307] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:07:59.772205] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:59.772252] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:59.772277] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:07:59.772286] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:07:59.772293] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:07:59.782724] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:07:59.816691] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:07:59.816741] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:07:59.832634] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:07:59.832660] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:07:59.850263] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:07:59.850291] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:01.654188] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:08:03.008386] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:08:03.019727] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f13ba3c647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f13ba699182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f13bb193265]))) 0-: received signum (15), shutting down [2016-03-15 15:08:03.031493] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:08:03.035504] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:08:03.036108] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:08:03.036122] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:08:03.036469] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:08:03.036488] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:08:03.036494] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:08:03.036530] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:08:03.036580] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:08:03.036588] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:08:04.640838] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:08:04.643989] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:04.646815] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:04.649857] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:04.649895] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:04.649946] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:04.649954] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:08:04.650547] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:04.650584] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:04.650592] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:08:04.650859] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:04.650893] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:04.650905] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:08:04.654913] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:08:04.656054] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:04.656108] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:08:04.723649] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:04.723722] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:08:04.745832] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:08:04.745855] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:08:04.838420] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:04.838456] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:08:04.838464] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:08:04.838479] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:08:04.838553] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:08:04.855818] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:05.316707] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:05.783317] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:08:05.816997] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:05.817042] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:08:05.817049] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:08:05.817056] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:08:05.855682] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:05.893483] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:05.893536] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:08:05.912913] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:08:05.912947] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:08:05.930297] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:05.934211] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:06.711447] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:08:08.093455] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:08:08.103944] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fc068ba447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fc068e77182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fc069971265]))) 0-: received signum (15), shutting down [2016-03-15 15:08:08.116381] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:08:08.120431] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:08:08.121000] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:08:08.121013] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:08:08.121356] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:08:08.121367] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:08:08.121373] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:08:08.121407] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:08:08.121464] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:08:08.121472] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:08:09.701758] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:08:09.705602] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:09.710161] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:09.715555] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:09.715597] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:09.715645] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:09.715652] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:08:09.716202] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:09.716240] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:09.716248] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:08:09.716467] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:09.716532] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:09.716541] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:08:09.721891] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:08:09.723229] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:09.723285] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:08:09.747861] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:09.856608] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:09.856677] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:08:09.879115] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:08:09.879151] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:08:09.895576] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:08:09.914492] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:09.914519] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:08:09.933445] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:09.933481] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:08:09.933489] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:08:09.933496] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:08:09.942053] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:10.508829] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:10.521402] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:10.521451] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:08:10.541017] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:08:10.541052] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:08:10.635441] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:10.635479] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:11.778277] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:08:13.177544] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:08:13.191182] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fe607c5147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fe607f24182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fe608a1e265]))) 0-: received signum (15), shutting down [2016-03-15 15:08:13.203478] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:08:13.207407] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:08:13.208034] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:08:13.208048] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:08:13.208397] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:08:13.208408] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:08:13.208415] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:08:13.208450] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:08:13.208501] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:08:13.208509] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:08:14.816563] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:08:14.819878] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:14.825520] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:14.828920] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:14.828966] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:14.829019] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:14.829038] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:08:14.829694] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:14.829732] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:14.829740] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:08:14.830001] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:14.830043] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:14.830052] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:08:14.836746] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:08:14.837377] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:14.837448] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:08:14.876291] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:14.906568] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:14.941888] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:14.941957] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:08:14.960688] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:08:14.960724] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:08:15.010716] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:15.010802] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:08:15.026255] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:08:15.026274] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:08:15.070351] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:08:15.088767] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:15.088799] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:08:15.088807] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:08:15.088814] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:08:15.088888] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:08:15.108098] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:15.108133] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:15.108149] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:08:15.108155] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:08:15.108162] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:08:16.893697] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:08:18.265143] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:08:18.275804] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fb3f445147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fb3f4724182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fb3f521e265]))) 0-: received signum (15), shutting down [2016-03-15 15:08:18.287864] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:08:18.290903] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:08:18.291743] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:08:18.291761] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:08:18.292787] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:08:18.292806] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:08:18.292817] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:08:18.292872] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:08:18.293009] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:08:18.293026] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:08:19.895634] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:08:19.899161] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:19.902430] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:19.905584] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:19.905623] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:19.905670] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:19.905677] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:08:19.906260] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:19.906304] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:19.906314] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:08:19.906538] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:19.906576] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:19.906594] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:08:19.910676] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:08:19.911706] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:19.911832] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:08:20.007424] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:08:20.027706] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:20.027764] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:08:20.049414] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:08:20.049450] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:08:20.105223] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:20.105275] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:20.105296] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:08:20.105303] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:08:20.105310] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:08:20.319438] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:20.786260] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:08:20.803897] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:20.803924] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:08:20.803931] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:08:20.803938] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:08:20.810673] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:20.827816] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:20.827870] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:08:20.844257] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:08:20.844275] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:08:20.865723] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:20.865752] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:21.966866] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:08:23.353959] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:08:23.365578] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fb0c4e6047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fb0c5133182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fb0c5c2d265]))) 0-: received signum (15), shutting down [2016-03-15 15:08:23.376906] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:08:23.380840] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:08:23.381439] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:08:23.381452] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:08:23.381791] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:08:23.381802] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:08:23.381807] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:08:23.381843] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:08:23.381893] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:08:23.381901] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:08:24.981591] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:08:24.988235] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:24.991467] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:24.995429] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:24.995471] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:24.995522] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:24.995529] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:08:24.996068] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:24.996104] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:24.996111] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:08:24.996329] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:24.996373] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:24.996386] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:08:25.001730] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:08:25.003005] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:25.003061] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:08:25.020316] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:25.020371] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:08:25.047485] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:08:25.047516] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:08:25.108837] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:08:25.153230] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:25.153263] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:08:25.153271] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:08:25.153278] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:08:25.511207] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:08:25.570414] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:25.570462] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:08:25.570470] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:08:25.570477] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:08:25.588393] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:25.631122] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:25.631178] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:08:25.659049] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:08:25.659076] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:08:25.684093] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:25.684121] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:26.320184] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:27.058833] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:08:28.439570] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:08:28.450378] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fef47ce547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fef47fb8182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fef48ab2265]))) 0-: received signum (15), shutting down [2016-03-15 15:08:28.462155] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:08:28.464432] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:08:28.464942] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:08:28.464954] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:08:28.465294] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:08:28.465305] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:08:28.465310] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:08:28.465346] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:08:28.465395] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:08:28.465403] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:08:30.070888] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:08:30.074028] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:30.077036] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:30.080034] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:30.080075] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:30.080124] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:30.080132] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:08:30.080684] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:30.080721] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:30.080729] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:08:30.080946] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:30.080988] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:30.081001] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:08:30.086444] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:08:30.087401] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:30.087455] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:08:30.101367] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:30.143354] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:30.316308] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:30.316375] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:08:30.335085] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:08:30.335117] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:08:30.361251] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:30.361303] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:08:30.388189] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:08:30.388215] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:08:30.408582] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:08:30.440995] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:08:30.467739] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:30.467770] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:08:30.467778] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:08:30.467785] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:08:30.493042] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:30.493065] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:08:30.493072] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:08:30.493079] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:08:30.493119] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:30.493138] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:30.493151] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:30.493162] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:32.143313] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:08:33.526522] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:08:33.537012] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd443aab47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd443d7e182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd444878265]))) 0-: received signum (15), shutting down [2016-03-15 15:08:33.548413] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:08:33.552378] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:08:33.552985] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:08:33.552998] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:08:33.553339] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:08:33.553351] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:08:33.553356] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:08:33.553392] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:08:33.553443] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:08:33.553451] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:08:35.149011] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:08:35.152325] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:35.155342] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:35.159039] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:35.159079] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:35.159127] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:35.159134] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:08:35.159694] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:35.159730] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:35.159738] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:08:35.159966] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:35.160027] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:35.160041] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:08:35.164532] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:08:35.165640] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:35.165707] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:08:35.235301] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:35.235363] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:08:35.253343] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:08:35.253377] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:08:35.271110] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:08:35.299402] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:35.299442] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:08:35.318913] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:35.318942] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:08:35.318950] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:08:35.318957] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:08:35.319967] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:35.789982] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:35.812250] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:35.812301] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:08:35.833515] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:08:35.833548] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:08:35.863715] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:35.863746] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:36.106720] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:37.220729] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:08:38.611426] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:08:38.622032] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7feafcfef47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7feafd2c2182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7feafddbc265]))) 0-: received signum (15), shutting down [2016-03-15 15:08:38.633837] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:08:38.637089] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:08:38.637605] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:08:38.637616] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:08:38.637959] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:08:38.637971] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:08:38.637977] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:08:38.638013] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:08:38.638064] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:08:38.638071] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:08:40.271896] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:08:40.276253] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:40.279162] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:40.283294] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:40.283335] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:40.283382] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:40.283389] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:08:40.283949] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:40.283986] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:40.283994] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:08:40.284238] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:40.284274] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:40.284282] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:08:40.291286] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:08:40.292383] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:40.292436] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:08:40.362008] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:08:40.378352] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:40.378414] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:08:40.395738] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:08:40.395839] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:08:40.428759] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:40.428806] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:08:40.428814] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:08:40.428821] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:08:40.472669] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:40.514097] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:08:40.561254] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:40.561301] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:08:40.561308] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:08:40.561315] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:08:40.585235] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:40.597640] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:40.597688] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:08:40.671344] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:08:40.671387] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:08:40.687861] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:40.687885] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:41.323110] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:42.350954] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:08:43.700284] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:08:43.711194] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd51376747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd513a3a182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd514534265]))) 0-: received signum (15), shutting down [2016-03-15 15:08:43.722710] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:08:43.726647] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:08:43.727286] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:08:43.727301] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:08:43.727649] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:08:43.727669] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:08:43.727675] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:08:43.727710] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:08:43.727761] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:08:43.727769] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:08:45.269672] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:08:45.274565] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:45.281790] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:45.286056] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:45.286102] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:45.286154] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:45.286161] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:08:45.286784] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:45.286824] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:45.286832] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:08:45.287054] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:45.287108] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:45.287123] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:08:45.290648] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:08:45.291454] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:45.291513] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:08:45.320497] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:45.382177] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:08:45.400824] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:45.400877] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:08:45.417362] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:08:45.417383] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:08:45.435432] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:45.435478] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:45.435500] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:08:45.435508] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:08:45.435514] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:08:45.497808] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:08:46.516183] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:46.549035] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:46.549089] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:08:46.571646] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:08:46.571684] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:08:46.588395] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:46.653160] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:47.346931] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:08:48.785762] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:08:48.796789] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f1be89c847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f1be8c9b182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f1be9795265]))) 0-: received signum (15), shutting down [2016-03-15 15:08:48.808669] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:08:48.811017] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:08:48.813063] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:08:48.813081] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:08:48.813638] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:08:48.813656] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:08:48.813667] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:08:48.813719] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:08:48.813798] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:08:48.813813] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:08:50.413285] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:08:50.417974] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:50.421237] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:50.424883] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:50.424935] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:50.424987] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:50.424995] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:08:50.425517] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:50.425552] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:50.425560] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:08:50.425783] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:50.425821] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:50.425832] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:08:50.430258] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:08:50.431308] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:50.431381] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:08:50.446561] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:50.542318] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:50.542381] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:08:50.563019] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:08:50.563045] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:08:50.621638] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:08:50.643579] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:50.643612] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:08:50.643620] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:08:50.643627] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:08:50.643669] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:08:50.666904] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:50.792798] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:50.806102] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:50.806147] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:08:50.827148] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:08:50.827175] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:08:50.847538] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:50.847566] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:52.486969] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:08:53.872809] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:08:53.883311] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa4fa5ed47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa4fa8c0182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa4fb3ba265]))) 0-: received signum (15), shutting down [2016-03-15 15:08:53.894980] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:08:53.899039] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:08:53.899678] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:08:53.899691] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:08:53.900037] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:08:53.900048] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:08:53.900054] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:08:53.900089] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:08:53.900139] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:08:53.900147] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:08:55.520058] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:08:55.523555] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:55.526708] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:55.529683] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:08:55.529721] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:55.529767] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:55.529775] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:08:55.530313] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:55.530348] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:55.530355] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:08:55.530579] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:08:55.530622] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:08:55.530635] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:08:55.534573] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:08:55.535338] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:55.535395] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:08:55.642509] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:55.642571] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:08:55.660678] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:08:55.660712] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:08:55.702072] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:08:55.721763] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:55.721796] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:08:55.721804] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:08:55.721810] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:08:55.721852] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:08:55.741940] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:55.741962] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:56.325514] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:56.793965] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:08:56.827061] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:56.827114] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:08:56.859764] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:08:56.859797] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:08:56.908259] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:08:56.908290] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:08:57.592678] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:08:58.957815] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:08:58.968323] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f9ec705947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f9ec732c182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f9ec7e26265]))) 0-: received signum (15), shutting down [2016-03-15 15:08:58.980344] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:08:58.983249] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:08:58.983981] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:08:58.983998] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:08:58.984514] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:08:58.984531] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:08:58.984541] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:08:58.984595] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:08:58.984669] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:08:58.984683] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:09:00.597693] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:09:00.600914] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:09:00.604855] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:09:00.608107] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:09:00.608148] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:09:00.608196] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:09:00.608203] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:09:00.608804] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:09:00.608841] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:09:00.608848] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:09:00.609104] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:09:00.609150] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:09:00.609162] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:09:00.616444] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:09:00.617389] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:09:00.617445] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:09:00.631924] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:09:00.699216] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:00.699278] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:09:00.726537] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:09:00.726557] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:09:00.757640] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:09:00.775810] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:00.775850] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:00.775872] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:09:00.775879] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:09:00.775886] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:09:01.518222] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:09:01.566049] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:09:01.566095] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:09:01.566103] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:09:01.566110] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:09:01.598083] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:09:01.610547] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:09:01.610591] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:09:01.627754] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:09:01.627780] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:09:01.643753] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:01.789521] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:09:02.673572] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:09:04.044612] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:09:04.056321] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f71dcbd047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f71dcea3182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f71dd99d265]))) 0-: received signum (15), shutting down [2016-03-15 15:09:04.068144] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:09:04.070320] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:09:04.070915] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:09:04.070929] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:09:04.071277] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:09:04.071288] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:09:04.071294] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:09:04.071330] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:09:04.071381] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:09:04.071389] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:09:05.635699] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:09:05.639699] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:09:05.643147] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:09:05.646244] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:09:05.646282] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:09:05.646334] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:09:05.646342] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:09:05.646906] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:09:05.646943] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:09:05.646950] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:09:05.647169] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:09:05.647205] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:09:05.647213] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:09:05.651928] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:09:05.653062] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:09:05.653117] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:09:05.666836] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:09:05.803583] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:09:05.803642] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:09:05.818212] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:09:05.818236] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:09:05.833023] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:09:05.848598] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:05.848668] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:05.848693] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:09:05.848702] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:09:05.848709] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:09:05.849460] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:09:05.860716] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:09:05.879791] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:05.879855] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:09:05.896465] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:09:05.896499] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:09:05.911562] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:09:05.911587] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:09:05.911594] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:09:05.911601] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:09:05.911646] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:09:05.911666] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:09:05.913665] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:07.707890] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:09:09.132972] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:09:09.143758] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f90ebfcf47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f90ec2a2182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f90ecd9c265]))) 0-: received signum (15), shutting down [2016-03-15 15:09:09.155836] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:09:09.159554] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:09:09.160057] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:09:09.160069] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:09:09.160401] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:09:09.160412] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:09:09.160418] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:09:09.160453] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:09:09.160503] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:09:09.160511] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:09:10.757063] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:09:10.761937] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:09:10.765101] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:09:10.768268] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:09:10.768308] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:09:10.768358] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:09:10.768366] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:09:10.768935] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:09:10.768971] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:09:10.768978] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:09:10.769235] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:09:10.769268] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:09:10.769275] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:09:10.773216] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:09:10.774091] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:09:10.774146] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:09:10.870444] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:09:10.870508] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:09:10.890289] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:09:10.890322] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:09:10.910010] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:10.910036] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:09:10.927540] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:09:10.947015] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:09:10.947046] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:09:10.947053] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:09:10.947060] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:09:10.960472] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:09:11.328226] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:09:11.797013] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:09:11.812003] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:11.812055] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:09:11.827500] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:09:11.827527] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:09:11.844437] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:09:11.846867] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:12.830210] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:09:14.221407] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:09:14.234494] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f8aabf6147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f8aac234182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f8aacd2e265]))) 0-: received signum (15), shutting down [2016-03-15 15:09:14.246538] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:09:14.250483] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:09:14.251477] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:09:14.251496] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:09:14.251990] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:09:14.252007] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:09:14.252031] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:09:14.252084] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:09:14.252159] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:09:14.252172] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:09:16.697477] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:09:16.700719] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:09:16.704882] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:09:16.707922] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:09:16.707962] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:09:16.708013] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:09:16.708020] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:09:16.708509] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:09:16.708547] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:09:16.708555] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:09:16.708849] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:09:16.708887] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:09:16.708895] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:09:16.712488] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:09:16.713503] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:09:16.713592] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:09:16.726070] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:09:16.740168] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:09:16.768068] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:09:16.789482] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:16.789532] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:09:16.812182] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:09:16.812214] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:09:16.862545] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:09:16.862617] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:09:16.880321] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:09:16.880352] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:09:16.899380] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:16.899411] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:16.899433] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:09:16.899441] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:09:16.899448] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:09:16.899485] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:09:16.925127] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:09:16.925155] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:09:16.925162] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:09:16.925168] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:09:16.925215] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:16.925235] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:09:18.769240] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:09:19.308948] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:09:19.319442] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f54ace7247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f54ad145182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f54adc3f265]))) 0-: received signum (15), shutting down [2016-03-15 15:09:19.331888] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:09:19.334090] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:09:19.334595] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:09:19.334607] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:09:19.334983] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:09:19.334995] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:09:19.335001] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:09:19.335038] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:09:19.335096] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:09:19.335104] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:09:20.922559] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:09:20.927242] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:09:20.929994] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:09:20.936029] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:09:20.936071] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:09:20.936122] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:09:20.936130] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:09:20.936712] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:09:20.936755] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:09:20.936765] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:09:20.937021] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:09:20.937054] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:09:20.937062] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:09:20.940717] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:09:20.941552] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:09:20.941606] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:09:20.958531] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:09:20.997584] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:09:21.018709] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:09:21.064609] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:09:21.064674] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:09:21.080684] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:09:21.080708] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:09:21.099170] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:21.099202] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:21.099221] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:09:21.099236] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:09:21.099244] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:09:21.099285] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:09:21.114548] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:09:21.114567] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:09:21.114574] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:09:21.114580] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:09:21.114620] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:21.114659] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:09:21.131316] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:09:21.131339] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:09:21.161986] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:09:21.162018] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:09:21.162032] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:22.996814] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:09:24.395618] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:09:24.406960] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fde7221e47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fde724f1182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fde72feb265]))) 0-: received signum (15), shutting down [2016-03-15 15:09:24.419425] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:09:24.422210] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:09:24.423268] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:09:24.423287] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:09:24.423786] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:09:24.423803] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:09:24.423814] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:09:24.423884] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:09:24.423960] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:09:24.423974] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:09:25.993774] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:09:25.996912] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:09:25.999754] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:09:26.002601] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:09:26.002639] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:09:26.002688] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:09:26.002696] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:09:26.003288] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:09:26.003324] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:09:26.003332] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:09:26.003560] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:09:26.003604] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:09:26.003614] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:09:26.007214] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:09:26.008141] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:09:26.008192] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:09:26.126508] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:09:26.126567] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:09:26.144680] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:09:26.144700] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:09:26.179172] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:09:26.205567] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:09:26.205601] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:09:26.205610] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:09:26.205616] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 15:09:26.205658] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:09:26.226632] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:09:26.226659] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:26.330884] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:09:26.800009] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:09:26.813268] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:26.813319] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:09:26.831185] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:09:26.831210] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:09:26.846952] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:09:26.846979] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:28.064562] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:09:29.485227] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:09:29.495764] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fb30883047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fb308b03182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fb3095fd265]))) 0-: received signum (15), shutting down [2016-03-15 15:09:29.507752] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:09:29.511858] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:09:29.512467] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:09:29.512480] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:09:29.512852] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:09:29.512868] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:09:29.512878] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:09:29.512918] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:09:29.512970] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:09:29.512978] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:09:31.078332] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:09:31.081726] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:09:31.085577] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:09:31.093460] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:09:31.093504] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:09:31.093554] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:09:31.093561] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:09:31.094131] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:09:31.094179] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:09:31.094187] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:09:31.094428] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:09:31.094466] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:09:31.094474] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:09:31.103274] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:09:31.104072] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:09:31.104125] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:09:31.121593] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:31.121646] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:09:31.143354] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:09:31.143390] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:09:31.345002] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 15:09:31.364306] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:31.364339] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:09:31.364347] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:09:31.364354] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 15:09:31.364414] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 15:09:31.524714] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:09:31.537467] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:09:31.537515] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 15:09:31.556058] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:09:31.556092] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:09:31.594319] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:31.594362] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 15:09:32.331952] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:09:33.159780] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:09:34.570298] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 15:09:34.580823] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f953026c47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f953053f182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f9531039265]))) 0-: received signum (15), shutting down [2016-03-15 15:09:34.592383] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 15:09:34.596248] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 15:09:34.597624] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:09:34.597642] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:09:34.598163] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 15:09:34.598179] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 15:09:34.598189] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 15:09:34.598238] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 15:09:34.598312] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 15:09:34.598326] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 15:09:36.178567] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 15:09:36.181846] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:09:36.186531] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:09:36.191122] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 15:09:36.191163] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:09:36.191211] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:09:36.191219] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:09:36.191796] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:09:36.191832] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:09:36.191840] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 15:09:36.192097] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 15:09:36.192142] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 15:09:36.192155] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 15:09:36.195980] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 15:09:36.197294] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:09:36.197351] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 15:09:36.212439] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 15:09:36.243294] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:36.243356] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 15:09:36.264249] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 15:09:36.264283] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 15:09:36.292126] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 15:09:36.292158] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 15:09:36.292166] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 15:09:36.292173] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64