[2016-03-15 14:35:06.465165] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:35:06.484830] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:35:06.485694] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:35:06.485707] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:35:06.486095] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:35:06.486106] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:35:06.486112] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:35:06.486150] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:35:06.486200] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:35:06.486208] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:35:09.921504] E [store.c:408:gf_store_handle_retrieve] 0-: Path corresponding to /var/lib/glusterd/glusterd.info, returned error: (No such file or directory) [2016-03-15 14:35:09.921532] E [store.c:408:gf_store_handle_retrieve] 0-: Path corresponding to /var/lib/glusterd/glusterd.info, returned error: (No such file or directory) [2016-03-15 14:35:09.921539] I [glusterd-store.c:1441:glusterd_restore_op_version] 0-management: Detected new install. Setting op-version to maximum : 30504 [2016-03-15 14:35:09.921621] E [store.c:408:gf_store_handle_retrieve] 0-: Path corresponding to /var/lib/glusterd/options, returned error: (No such file or directory) Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:36:45.782158] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:36:45.783293] I [glusterd-handler.c:918:__glusterd_handle_cli_probe] 0-glusterd: Received CLI probe req 172.31.30.64 24007 [2016-03-15 14:36:45.786102] I [glusterd-handler.c:2931:glusterd_probe_begin] 0-glusterd: Unable to find peerinfo for host: 172.31.30.64 (24007) [2016-03-15 14:36:53.007545] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:36:53.007632] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:36:53.007645] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:36:53.008052] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:36:53.008082] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:36:56.158216] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:00.705728] E [store.c:408:gf_store_handle_retrieve] 0-: Path corresponding to /var/lib/glusterd/glusterd.info, returned error: (No such file or directory) [2016-03-15 14:37:00.705782] I [glusterd.c:168:glusterd_uuid_generate_save] 0-management: generated UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:37:07.138906] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:09.277697] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:11.403758] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:13.459017] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:15.515262] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:17.571812] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:19.634481] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:21.734343] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:22.147463] I [glusterd-rpc-ops.c:234:__glusterd_probe_cbk] 0-glusterd: Received probe resp from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:37:22.168148] I [glusterd-rpc-ops.c:306:__glusterd_probe_cbk] 0-glusterd: Received resp to probe req [2016-03-15 14:37:22.168215] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:37:22.367457] I [glusterd-handler.c:918:__glusterd_handle_cli_probe] 0-glusterd: Received CLI probe req 172.31.27.27 24007 [2016-03-15 14:37:22.368612] I [glusterd-handler.c:2931:glusterd_probe_begin] 0-glusterd: Unable to find peerinfo for host: 172.31.27.27 (24007) [2016-03-15 14:37:22.395611] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:22.395686] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:22.395702] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:37:22.396054] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:23.823457] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:25.879010] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:27.933933] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:30.092223] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:32.148269] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:34.206242] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:36.261697] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:38.317163] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:40.418302] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:40.571044] I [glusterd-rpc-ops.c:234:__glusterd_probe_cbk] 0-glusterd: Received probe resp from uuid: acfc1794-9080-4eb0-8f69-3abe78bbee16, host: 172.31.27.27 [2016-03-15 14:37:40.693658] I [glusterd-rpc-ops.c:306:__glusterd_probe_cbk] 0-glusterd: Received resp to probe req [2016-03-15 14:37:40.693723] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: acfc1794-9080-4eb0-8f69-3abe78bbee16, host: 172.31.27.27, port: 0 [2016-03-15 14:37:40.996043] I [glusterd-handler.c:918:__glusterd_handle_cli_probe] 0-glusterd: Received CLI probe req 172.31.26.134 24007 [2016-03-15 14:37:40.996588] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:37:41.151224] I [glusterd-handler.c:918:__glusterd_handle_cli_probe] 0-glusterd: Received CLI probe req 172.31.19.46 24007 [2016-03-15 14:37:41.153210] I [glusterd-handler.c:2931:glusterd_probe_begin] 0-glusterd: Unable to find peerinfo for host: 172.31.19.46 (24007) [2016-03-15 14:37:41.290294] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:41.290360] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:41.290368] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:37:41.290620] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:41.295737] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:37:41.380003] I [glusterd-handler.c:2346:__glusterd_handle_probe_query] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:41.380059] I [glusterd-handler.c:2398:__glusterd_handle_probe_query] 0-glusterd: Responded to 172.31.30.64, op_ret: 0, op_errno: 0, ret: 0 [2016-03-15 14:37:41.395401] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:41.395459] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:37:41.417433] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:37:41.444354] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:41.444385] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:37:41.444392] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:37:41.444442] I [glusterd-rpc-ops.c:234:__glusterd_probe_cbk] 0-glusterd: Received probe resp from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:37:41.519256] I [glusterd-rpc-ops.c:306:__glusterd_probe_cbk] 0-glusterd: Received resp to probe req [2016-03-15 14:37:41.519352] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:41.519371] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:37:41.646776] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:37:41.647238] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:37:41.657372] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fb64664247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fb646915182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fb64740f265]))) 0-: received signum (15), shutting down [2016-03-15 14:37:41.681593] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:37:41.685045] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:37:41.685543] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:37:41.685555] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:37:41.685893] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:37:41.685905] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:37:41.685910] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:37:41.685946] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:37:41.685997] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:37:41.686011] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:37:43.493948] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:37:43.497804] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:43.501458] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:43.504595] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:43.504634] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:43.504681] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:43.504688] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:37:43.505180] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:43.505218] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:43.505226] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:37:43.505440] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:43.505475] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:43.505483] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:37:43.510927] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:43.511875] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:37:43.511930] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:37:43.585772] I [glusterd-rpc-ops.c:234:__glusterd_probe_cbk] 0-glusterd: Received probe resp from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:37:43.693226] I [glusterd-rpc-ops.c:306:__glusterd_probe_cbk] 0-glusterd: Received resp to probe req [2016-03-15 14:37:43.718605] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:43.718662] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:37:43.739251] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:37:43.801542] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:37:43.826785] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:43.826814] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:37:43.826822] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:37:43.826882] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:44.426414] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:37:44.426458] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:37:44.426468] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:37:44.462144] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:44.520713] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:37:44.520770] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:37:44.555108] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:37:44.555168] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:37:44.580118] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:37:44.580141] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:37:44.619547] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:44.645063] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:37:44.684319] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:44.684355] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:37:44.684362] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:37:44.684369] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:37:45.566344] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:46.748401] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:37:46.759601] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f53b48f947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f53b4bcc182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f53b56c6265]))) 0-: received signum (15), shutting down [2016-03-15 14:37:46.770859] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:37:46.774727] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:37:46.776901] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:37:46.776914] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:37:46.777249] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:37:46.777260] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:37:46.777266] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:37:46.777302] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:37:46.777360] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:37:46.777369] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:37:48.404612] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:37:48.407576] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:48.410372] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:48.412975] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:48.413014] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:48.413063] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:48.413070] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:37:48.413605] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:48.413641] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:48.413651] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:37:48.413866] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:48.413910] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:48.413923] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:37:48.417819] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:48.418814] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:37:48.418869] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:37:48.452631] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:37:48.487374] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:37:48.487431] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:37:48.506201] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:37:48.506226] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:37:48.545514] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:37:48.545558] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:37:48.545582] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:37:48.545589] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:37:48.545604] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:37:49.152841] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:37:49.173865] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:49.173894] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:37:49.173902] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:37:49.173909] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:37:49.231739] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:37:49.247929] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:49.247980] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:37:49.280244] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:37:49.280271] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:37:49.343739] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:37:49.343776] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:50.473985] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:51.834948] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:37:51.845350] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fadbed6747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fadbf03a182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fadbfb34265]))) 0-: received signum (15), shutting down [2016-03-15 14:37:51.857941] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:37:51.861033] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:37:51.861714] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:37:51.861731] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:37:51.862244] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:37:51.862261] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:37:51.862271] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:37:51.862321] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:37:51.862398] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:37:51.862413] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:37:54.195850] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:37:54.200126] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:54.202936] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:54.207738] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:54.207776] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:54.207824] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:54.207832] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:37:54.208329] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:54.208364] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:54.208371] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:37:54.208584] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:54.208620] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:54.208628] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:37:54.212797] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:54.213779] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:37:54.213832] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:37:54.229172] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:37:54.270884] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:54.270941] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:37:54.288831] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:37:54.288854] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:37:54.351587] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:37:54.351640] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:37:54.375388] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:37:54.375412] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:37:54.397364] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:37:54.444034] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:37:54.464330] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:37:54.464369] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:37:54.464377] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:37:54.464384] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:37:54.464441] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:54.464466] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:54.464480] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:37:54.464485] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:37:54.464492] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:37:54.464527] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:37:54.464546] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:56.269085] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:56.918566] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:37:56.928716] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ffb9e03247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ffb9e305182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ffb9edff265]))) 0-: received signum (15), shutting down [2016-03-15 14:37:56.940090] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:37:56.943955] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:37:56.944544] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:37:56.944557] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:37:56.944900] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:37:56.944912] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:37:56.944918] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:37:56.944953] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:37:56.945006] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:37:56.945014] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:37:58.550342] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:37:58.553316] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:58.556329] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:58.559793] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:37:58.559832] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:58.559879] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:58.559892] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:37:58.560453] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:58.560488] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:58.560495] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:37:58.560741] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:37:58.560778] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:37:58.560785] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:37:58.565217] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:37:58.566135] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:37:58.566256] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:37:58.664947] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:58.665010] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:37:58.683845] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:37:58.683862] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:37:58.717982] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:37:58.718007] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:37:58.748188] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:37:58.781820] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:58.781851] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:37:58.781858] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:37:58.781865] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:37:58.807765] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:37:59.429950] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:37:59.447868] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:37:59.447915] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:37:59.466384] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:37:59.466402] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:37:59.482969] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:37:59.483004] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:00.621705] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:02.002265] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:38:02.012409] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f36b554647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f36b5819182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f36b6313265]))) 0-: received signum (15), shutting down [2016-03-15 14:38:02.025130] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:38:02.029369] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:38:02.029971] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:02.029984] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:02.030327] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:38:02.030338] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:38:02.030344] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:38:02.030379] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:38:02.030430] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:02.030439] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:03.606543] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:38:03.610079] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:03.613588] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:03.618206] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:03.618267] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:03.618324] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:03.618333] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:03.618900] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:03.618939] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:03.618947] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:03.619165] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:03.619203] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:03.619212] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:38:03.625440] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:03.626328] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:03.626386] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:38:03.721979] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:03.722045] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:38:03.739056] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:03.739083] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:03.762899] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:38:03.783907] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:03.783941] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:03.783949] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:03.783956] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:38:03.784010] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:38:03.823032] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:04.119151] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:04.156454] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:04.191956] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:04.192012] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:38:04.214558] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:04.214585] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:04.233037] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:04.266202] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:05.680978] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:07.085803] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:38:07.096931] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fb55961447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fb5598e7182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fb55a3e1265]))) 0-: received signum (15), shutting down [2016-03-15 14:38:07.108768] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:38:07.112689] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:38:07.113385] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:07.113402] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:07.113902] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:38:07.113919] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:38:07.113929] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:38:07.113976] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:38:07.114047] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:07.114060] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:08.662228] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:38:08.666254] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:08.670098] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:08.672997] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:08.673036] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:08.673082] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:08.673090] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:08.673610] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:08.673648] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:08.673656] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:08.673871] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:08.673911] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:08.673923] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:38:08.677763] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:08.678511] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:08.678563] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:38:08.690995] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:08.723250] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:38:08.757064] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:08.757123] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:38:08.776197] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:08.776229] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:08.812565] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:08.812631] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:38:08.830733] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:08.830768] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:08.851954] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:08.851985] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:08.852003] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:08.852010] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:08.852016] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:38:08.852055] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:38:08.872958] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:08.872985] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:08.872992] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:08.872998] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:38:08.873042] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:08.873062] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:10.733590] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:12.168270] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:38:12.178685] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f7a3ca1347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f7a3cce6182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f7a3d7e0265]))) 0-: received signum (15), shutting down [2016-03-15 14:38:12.190153] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:38:12.193237] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:38:12.193797] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:12.193809] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:12.194143] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:38:12.194154] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:38:12.194160] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:38:12.194196] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:38:12.194245] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:12.194253] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:13.850608] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:38:13.853949] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:13.858680] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:13.877202] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:13.877251] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:13.877303] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:13.877310] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:13.877891] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:13.877933] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:13.877943] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:13.878207] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:13.878240] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:13.878247] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:38:13.883748] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:13.884818] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:13.884874] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:38:13.953662] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:13.953727] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:38:13.975131] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:13.975157] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:14.039604] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:38:14.063546] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:38:14.084798] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:14.084846] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:14.084879] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:14.084892] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:14.084904] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:38:14.432905] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:14.450166] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:14.450213] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:38:14.494613] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:14.494636] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:14.515220] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:14.515250] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:15.940030] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:17.252653] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:38:17.263117] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f0bf9b2947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f0bf9dfc182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f0bfa8f6265]))) 0-: received signum (15), shutting down [2016-03-15 14:38:17.274144] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:38:17.278038] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:38:17.278627] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:17.278640] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:17.279049] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:38:17.279062] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:38:17.279068] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:38:17.279103] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:38:17.279154] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:17.279162] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:18.967144] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:38:18.970954] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:18.975137] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:18.978281] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:18.978320] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:18.978368] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:18.978375] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:18.978988] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:18.979028] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:18.979036] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:18.979267] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:18.979305] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:18.979313] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:38:18.984277] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:18.985414] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:18.985465] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:38:19.055873] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:19.055936] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:38:19.075643] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:19.075677] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:19.096435] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:19.096460] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:38:19.127817] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:38:19.161352] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:19.161386] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:19.161394] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:19.161401] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:38:19.162058] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:19.198823] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:19.199117] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:19.199165] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:38:19.221156] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:19.221190] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:19.303297] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:19.303331] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:21.041123] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:22.336655] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:38:22.347245] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f5613f6547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f5614238182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f5614d32265]))) 0-: received signum (15), shutting down [2016-03-15 14:38:22.358923] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:38:22.362123] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:38:22.363022] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:22.363041] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:22.363538] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:38:22.363556] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:38:22.363565] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:38:22.363625] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:38:22.363702] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:22.363716] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:23.976877] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:38:23.980257] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:23.983238] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:23.987282] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:23.987321] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:23.987366] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:23.987373] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:23.987941] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:23.987976] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:23.987983] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:23.988250] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:23.988281] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:23.988288] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:38:23.993906] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:23.995020] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:23.995072] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:38:24.071951] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:24.072018] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:38:24.091866] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:24.091889] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:24.173320] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:24.173374] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:24.173388] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:24.173401] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:38:24.173455] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:38:24.194978] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:24.449346] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:38:25.160351] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:25.206009] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:25.206067] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:38:25.244212] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:25.244234] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:25.274421] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:25.274503] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:26.049733] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:27.428739] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:38:27.440023] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fc9f132047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fc9f15f3182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fc9f20ed265]))) 0-: received signum (15), shutting down [2016-03-15 14:38:27.452618] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:38:27.454887] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:38:27.455395] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:27.455407] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:27.455752] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:38:27.455764] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:38:27.455770] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:38:27.455806] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:38:27.455858] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:27.455866] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:29.037004] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:38:29.040794] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:29.043763] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:29.046043] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:29.046083] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:29.046133] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:29.046141] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:29.046686] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:29.046721] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:29.046728] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:29.046983] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:29.047026] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:29.047036] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:38:29.051201] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:29.052284] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:29.052345] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:38:29.097479] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:29.097542] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:38:29.144214] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:29.144242] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:29.183894] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:38:29.205860] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:38:29.228831] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:29.228866] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:29.228887] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:29.228894] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:29.228901] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:38:29.435891] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:29.505044] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:29.505095] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:38:29.535844] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:29.535880] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:29.557864] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:29.557902] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:31.106572] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:32.516674] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:38:32.527961] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f56f9e4a47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f56fa11d182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f56fac17265]))) 0-: received signum (15), shutting down [2016-03-15 14:38:32.539509] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:38:32.543889] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:38:32.544501] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:32.544521] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:32.544861] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:38:32.544872] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:38:32.544878] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:38:32.544913] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:38:32.544964] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:32.544971] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:34.186345] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:38:34.191730] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:34.195610] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:34.198807] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:34.198848] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:34.198896] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:34.198904] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:34.199473] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:34.199508] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:34.199516] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:34.199778] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:34.199811] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:34.199818] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:38:34.203816] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:34.204667] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:34.204719] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:38:34.284395] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:34.284456] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:38:34.306738] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:34.306764] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:34.363409] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:34.363442] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:34.363456] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:34.363464] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:38:34.363507] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:38:34.381990] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:34.944484] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:38:35.437091] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:35.454898] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:35.454962] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:38:35.491412] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:35.491438] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:35.508440] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:35.508887] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:36.259786] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:37.599747] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:38:37.609998] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f539434947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f539461c182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f5395116265]))) 0-: received signum (15), shutting down [2016-03-15 14:38:37.621460] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:38:37.624802] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:38:37.625293] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:37.625305] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:37.625648] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:38:37.625662] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:38:37.625669] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:38:37.625705] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:38:37.625756] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:37.625764] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:39.252633] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:38:39.255841] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:39.258926] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:39.262058] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:39.262098] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:39.262145] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:39.262153] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:39.262712] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:39.262779] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:39.262800] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:39.263031] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:39.263066] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:39.263073] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:38:39.266631] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:39.267956] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:39.268011] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:38:39.341370] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:39.341432] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:38:39.364287] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:39.364306] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:39.381689] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:38:39.401547] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:39.401578] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:39.401586] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:39.401593] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:38:39.401639] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:38:39.420391] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:39.420412] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:40.163342] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:40.199395] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:40.199449] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:38:40.226821] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:40.226854] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:40.245658] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:40.264503] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:41.321803] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:42.686914] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:38:42.700166] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fde05c8947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fde05f5c182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fde06a56265]))) 0-: received signum (15), shutting down [2016-03-15 14:38:42.712407] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:38:42.716564] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:38:42.717246] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:42.717259] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:42.717596] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:38:42.717608] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:38:42.717614] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:38:42.717649] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:38:42.717700] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:42.717708] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:44.468688] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:38:44.471914] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:44.475103] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:44.493151] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:44.493195] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:44.493247] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:44.493254] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:44.493833] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:44.493869] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:44.493876] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:44.494155] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:44.494187] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:44.494203] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:38:44.498787] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:44.499537] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:44.499596] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:38:44.512935] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:44.529406] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:44.589665] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:44.589728] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:38:44.608864] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:44.608900] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:44.725199] I [glusterd-rpc-ops.c:234:__glusterd_probe_cbk] 0-glusterd: Received probe resp from uuid: acfc1794-9080-4eb0-8f69-3abe78bbee16, host: 172.31.27.27 [2016-03-15 14:38:44.758349] I [glusterd-rpc-ops.c:306:__glusterd_probe_cbk] 0-glusterd: Received resp to probe req [2016-03-15 14:38:44.804400] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:44.804461] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:38:44.822050] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:44.822067] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:44.843066] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:44.843097] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:44.843104] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:44.843111] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:38:44.843159] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:38:44.864180] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:38:44.888429] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:44.888465] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:44.888472] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:44.888479] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:38:44.888531] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:44.888549] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:46.555224] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:47.775481] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:38:47.786765] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f95c419847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f95c446b182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f95c4f65265]))) 0-: received signum (15), shutting down [2016-03-15 14:38:47.798473] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:38:47.801817] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:38:47.802319] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:47.802331] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:47.802668] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:38:47.802679] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:38:47.802685] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:38:47.802721] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:38:47.803770] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:47.803789] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:49.642818] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:38:49.645928] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:49.648866] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:49.651958] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:49.651997] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:49.652044] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:49.652052] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:49.652550] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:49.652585] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:49.652593] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:49.652814] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:49.652859] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:49.652872] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:38:49.658452] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:49.659714] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:49.659767] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:38:49.687854] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:49.800852] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:49.800929] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:38:49.825692] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:49.825716] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:49.848227] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:38:49.866769] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:38:49.889600] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:49.889636] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:49.889644] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:49.889651] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:38:49.889697] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:49.889719] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:50.440082] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:50.457789] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:50.457834] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:38:50.478613] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:50.478641] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:50.495427] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:50.556180] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:51.715695] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:52.862485] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:38:52.873029] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7feb224b147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7feb22784182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7feb2327e265]))) 0-: received signum (15), shutting down [2016-03-15 14:38:52.884345] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:38:52.888285] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:38:52.888885] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:52.888898] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:52.889241] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:38:52.889252] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:38:52.889258] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:38:52.889293] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:38:52.889344] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:52.889351] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:54.908905] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:38:54.916130] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:54.920630] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:54.925105] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:54.925147] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:54.925198] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:54.925205] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:54.925782] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:54.925823] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:54.925831] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:54.926045] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:54.926083] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:54.926091] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:38:54.930655] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:54.931727] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:54.931798] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:38:55.036841] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:55.036909] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:38:55.075398] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:55.075435] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:55.132623] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:38:55.605147] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:55.605186] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:55.605195] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:55.605202] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:38:55.605281] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:55.605313] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:55.605335] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:38:55.652145] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:55.669232] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:55.683801] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:55.683851] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:38:55.710631] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:55.710660] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:55.737306] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:55.787907] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:56.987820] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:57.946365] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:38:57.956840] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f04d217847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f04d244b182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f04d2f45265]))) 0-: received signum (15), shutting down [2016-03-15 14:38:57.969156] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:38:57.973179] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:38:57.973909] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:57.973926] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:57.974449] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:38:57.974466] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:38:57.974475] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:38:57.974523] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:38:57.974606] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:38:57.974621] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:38:59.693485] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:38:59.698405] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:59.703723] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:59.707023] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:38:59.707063] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:59.707114] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:59.707121] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:59.707699] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:59.707735] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:59.707742] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:38:59.707975] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:38:59.708012] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:38:59.708021] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:38:59.714494] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:38:59.715425] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:59.715482] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:38:59.735839] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:59.756775] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:38:59.890628] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:59.890698] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:38:59.915730] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:38:59.915774] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:38:59.932860] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:38:59.961468] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:38:59.997951] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:38:59.997986] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:38:59.997994] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:38:59.998000] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:38:59.998043] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:38:59.998083] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:39:00.043528] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:00.043554] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:00.114879] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:00.114915] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:00.114941] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:00.114960] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:00.114966] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:00.114973] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:39:00.115023] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:00.115045] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:01.771715] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:03.032881] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:39:03.043527] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f0e64a6747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f0e64d3a182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f0e65834265]))) 0-: received signum (15), shutting down [2016-03-15 14:39:03.056104] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:39:03.060188] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:39:03.060705] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:03.060718] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:03.061062] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:39:03.061074] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:39:03.061079] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:39:03.061115] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:39:03.061165] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:03.061173] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:04.706184] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:39:04.710711] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:04.714358] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:04.717723] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:04.717763] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:04.717812] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:04.717820] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:04.718385] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:04.718427] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:04.718436] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:04.718691] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:04.718727] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:04.718763] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:39:04.722708] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:04.723847] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:04.723930] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:39:04.759917] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:04.865852] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:04.865917] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:39:04.888075] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:04.888110] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:04.920138] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:39:04.945528] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:39:04.970421] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:04.970460] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:04.970469] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:04.970476] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:39:04.970545] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:04.970563] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:05.442991] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:05.458131] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:05.458189] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:39:05.477813] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:05.477835] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:05.517527] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:05.547653] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:06.780481] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:08.119478] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:39:08.130665] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f1f53b1347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f1f53de6182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f1f548e0265]))) 0-: received signum (15), shutting down [2016-03-15 14:39:08.142371] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:39:08.146484] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:39:08.147547] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:08.147566] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:08.148062] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:39:08.148079] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:39:08.148089] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:39:08.148141] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:39:08.148212] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:08.148223] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:09.760152] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:39:09.763003] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:09.766575] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:09.769558] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:09.769602] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:09.769653] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:09.769661] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:09.770256] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:09.770299] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:09.770308] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:09.770576] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:09.770608] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:09.770615] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:39:09.774413] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:09.775420] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:09.775475] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:39:09.865913] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:39:09.884021] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:09.884073] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:39:09.904353] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:09.904379] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:09.963676] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:09.963717] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:09.963737] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:09.963744] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:09.963751] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:39:10.168531] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:39:10.253872] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:10.253920] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:10.253928] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:10.253935] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:39:10.272673] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:10.287726] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:10.287769] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:39:10.339574] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:10.339601] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:10.360272] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:10.360300] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:11.419797] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:11.830427] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:13.203921] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:39:13.214149] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f122ce7b47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f122d14e182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f122dc48265]))) 0-: received signum (15), shutting down [2016-03-15 14:39:13.225876] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:39:13.228733] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:39:13.229235] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:13.229246] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:13.229586] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:39:13.229597] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:39:13.229603] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:39:13.229641] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:39:13.229695] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:13.229703] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:14.959654] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:39:14.964178] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:14.968290] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:14.973325] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:14.973377] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:14.973427] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:14.973435] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:14.973963] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:14.973999] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:14.974006] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:14.974230] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:14.974275] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:14.974288] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:39:14.978652] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:14.979508] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:14.979561] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:39:15.011320] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:15.086689] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:39:15.108567] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:15.108621] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:39:15.132941] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:15.132974] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:15.180161] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:15.180211] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:15.180232] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:15.180239] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:15.180245] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:39:15.827505] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:39:16.170203] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:16.186866] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:16.186920] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:39:16.237649] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:16.237677] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:16.273202] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:16.273231] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:17.035367] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:18.289858] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:39:18.300127] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7faf9368047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7faf93953182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7faf9444d265]))) 0-: received signum (15), shutting down [2016-03-15 14:39:18.311585] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:39:18.316139] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:39:18.316836] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:18.316853] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:18.317206] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:39:18.317218] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:39:18.317224] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:39:18.317257] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:39:18.317308] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:18.317316] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:20.068767] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:39:20.072796] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:20.076047] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:20.079488] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:20.079528] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:20.079579] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:20.079587] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:20.080072] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:20.080110] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:20.080118] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:20.080337] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:20.080376] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:20.080388] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:39:20.085026] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:20.086099] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:20.086152] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:39:20.118662] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:20.131399] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:20.131450] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:39:20.152628] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:20.152663] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:20.210880] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:39:20.232491] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:39:20.278875] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:20.278911] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:20.278918] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:20.278925] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:39:20.446067] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:20.461736] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:20.461792] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:39:20.499775] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:20.499819] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:20.529679] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:20.565553] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:22.143502] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:23.373672] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:39:23.384193] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fcb0147547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fcb01748182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fcb02242265]))) 0-: received signum (15), shutting down [2016-03-15 14:39:23.395794] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:39:23.397932] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:39:23.398428] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:23.398439] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:23.398864] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:39:23.398877] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:39:23.398883] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:39:23.398943] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:39:23.398998] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:23.399006] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:25.032826] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:39:25.036725] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:25.040284] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:25.043783] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:25.043823] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:25.043872] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:25.043879] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:25.044405] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:25.044444] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:25.044452] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:25.044672] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:25.044716] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:25.044729] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:39:25.048995] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:25.050093] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:25.050147] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:39:25.228259] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:39:25.254939] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:25.254981] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:25.254990] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:25.254997] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:39:25.264473] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:25.310212] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:25.310264] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:39:25.334392] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:25.334420] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:25.353538] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:25.353608] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:39:25.376685] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:25.376709] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:25.395602] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:25.395636] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:25.618187] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:25.661294] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:39:26.021497] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:26.021542] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:26.021550] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:26.021557] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:39:26.168193] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:26.168252] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:27.105017] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:28.466278] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:39:28.477659] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f5ee375f47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f5ee3a32182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f5ee452c265]))) 0-: received signum (15), shutting down [2016-03-15 14:39:28.488757] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:39:28.492681] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:39:28.493269] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:28.493282] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:28.493666] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:39:28.493678] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:39:28.493683] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:39:28.493719] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:39:28.493770] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:28.493778] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:30.118533] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:39:30.122139] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:30.125396] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:30.129936] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:30.129976] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:30.130030] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:30.130038] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:30.130616] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:30.130651] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:30.130658] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:30.130954] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:30.130987] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:30.130994] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:39:30.136306] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:30.137194] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:30.137272] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:39:30.151315] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:30.251550] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:39:30.298818] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:30.298876] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:39:30.357536] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:30.357569] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:30.484834] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:30.484871] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:39:30.532502] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:30.532540] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:30.532547] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:30.532554] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:39:30.555820] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:31.173324] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:31.202565] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:31.202621] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:39:31.218282] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:31.218299] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:31.236105] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:31.274181] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:32.192376] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:33.551535] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:39:33.562030] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f65b913c47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f65b940f182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f65b9f09265]))) 0-: received signum (15), shutting down [2016-03-15 14:39:33.573866] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:39:33.577344] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:39:33.577834] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:33.577846] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:33.578194] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:39:33.578211] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:39:33.578217] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:39:33.578253] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:39:33.578303] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:33.578311] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:35.174494] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:39:35.177757] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:35.181137] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:35.184406] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:35.184445] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:35.184492] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:35.184499] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:35.185062] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:35.185099] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:35.185107] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:35.185358] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:35.185396] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:35.185404] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:39:35.189322] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:35.189822] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:35.189930] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:39:35.204975] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:35.205040] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:39:35.224349] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:35.224403] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:35.279781] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:35.373328] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:39:35.394123] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:35.394167] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:35.394175] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:35.394182] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:39:35.394229] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:39:35.448592] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:35.462961] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:35.463011] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:39:35.481486] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:35.481523] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:35.499602] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:35.502638] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:37.244911] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:38.640711] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:39:38.651188] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f516782147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f5167af4182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f51685ee265]))) 0-: received signum (15), shutting down [2016-03-15 14:39:38.662692] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:39:38.666698] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:39:38.667590] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:38.667604] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:38.667947] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:39:38.667959] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:39:38.667965] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:39:38.668001] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:39:38.668051] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:38.668059] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:40.339524] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:39:40.342817] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:40.348630] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:40.366921] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:40.366966] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:40.367023] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:40.367031] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:40.367544] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:40.367582] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:40.367590] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:40.367813] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:40.367849] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:40.367857] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:39:40.372867] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:40.374046] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:40.374118] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:39:40.387702] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:40.387749] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:39:40.407545] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:40.407571] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:40.529542] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:39:40.568090] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:40.568133] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:40.568141] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:40.568148] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:39:40.568522] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:39:40.625353] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:41.449879] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:41.466524] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:41.466570] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:39:41.497953] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:41.497980] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:41.521531] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:41.552676] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:42.429421] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:43.725208] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:39:43.737322] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fc9b051947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fc9b07ec182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fc9b12e6265]))) 0-: received signum (15), shutting down [2016-03-15 14:39:43.748570] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:39:43.752555] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:39:43.753068] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:43.753080] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:43.753456] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:39:43.753468] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:39:43.753474] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:39:43.753508] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:39:43.753559] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:43.753567] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:45.378448] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:39:45.381286] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:45.385230] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:45.389727] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:45.389770] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:45.389856] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:45.389872] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:45.390518] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:45.390565] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:45.390579] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:45.390940] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:45.390987] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:45.391000] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:39:45.396215] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:45.396276] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:39:45.411166] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:45.513196] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:45.513278] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:39:45.536765] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:45.536803] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:45.556798] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:39:45.587554] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:45.587588] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:45.587596] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:45.587603] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:39:45.587653] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:39:45.610270] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:45.610298] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:46.176071] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:46.190768] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:46.190823] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:39:46.210574] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:46.210592] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:46.233584] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:46.239087] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:46.612736] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:47.468346] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:48.810327] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:39:48.821988] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f29763b747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f297668a182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f2977184265]))) 0-: received signum (15), shutting down [2016-03-15 14:39:48.834518] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:39:48.836848] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:39:48.837352] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:48.837363] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:48.837702] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:39:48.837713] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:39:48.837719] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:39:48.837754] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:39:48.837805] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:48.837813] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:50.437322] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:39:50.440473] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:50.444434] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:50.447248] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:50.447297] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:50.447355] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:50.447365] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:50.447915] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:50.447950] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:50.447958] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:50.448193] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:50.448230] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:50.448238] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:39:50.451843] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:50.452604] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:50.452659] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:39:50.470262] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:50.518000] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:39:50.548059] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:50.578474] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:50.578508] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:50.578515] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:50.578523] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:39:50.613080] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:50.613144] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:39:50.629949] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:50.629982] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:50.648870] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:50.648929] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:39:50.665960] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:50.665994] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:50.685905] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:39:50.706010] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:50.706043] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:50.706050] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:50.706057] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:39:50.706109] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:50.706127] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:50.706145] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:50.706157] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:52.508424] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:53.902552] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:39:53.913312] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f79dd02847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f79dd2fb182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f79dddf5265]))) 0-: received signum (15), shutting down [2016-03-15 14:39:53.925363] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:39:53.928334] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:39:53.929026] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:53.929044] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:53.929556] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:39:53.929574] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:39:53.929584] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:39:53.929636] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:39:53.929714] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:53.929729] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:55.576338] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:39:55.581034] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:55.585643] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:55.590129] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:39:55.590169] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:55.590217] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:55.590225] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:55.590767] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:55.590805] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:55.590813] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:39:55.591048] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:39:55.591092] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:39:55.591104] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:39:55.596870] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:55.597728] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:55.597786] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:39:55.713733] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:55.713798] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:39:55.760622] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:55.760659] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:55.794841] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:39:55.839246] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:55.839279] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:39:55.839286] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:39:55.839293] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:39:55.839372] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:39:55.870746] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:55.870804] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:55.890027] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:56.452828] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:39:56.469051] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:56.469102] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:39:56.867246] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:39:56.867283] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:39:56.938952] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:39:56.938987] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:39:57.653632] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:39:58.992428] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:39:59.002622] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f453968747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f453995a182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f453a454265]))) 0-: received signum (15), shutting down [2016-03-15 14:39:59.013979] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:39:59.017057] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:39:59.017563] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:59.017574] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:39:59.017919] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:39:59.017936] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:39:59.017942] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:39:59.017977] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:39:59.018027] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:39:59.018035] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:00.691920] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:40:00.696092] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:00.704454] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:00.708671] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:00.708711] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:00.708762] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:00.708769] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:00.709351] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:00.709387] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:00.709395] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:00.709656] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:00.709692] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:00.709700] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:40:00.713581] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:00.714401] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:00.714454] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:40:00.834913] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:40:00.878851] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:40:00.928067] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:00.928132] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:40:00.966433] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:00.966473] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:01.032268] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:01.032309] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:01.032336] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:01.032358] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:01.032364] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:01.032371] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:40:01.178909] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:01.240968] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:01.241034] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:40:01.258409] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:01.258429] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:01.297713] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:01.297756] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:01.615512] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:02.770987] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:04.077633] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:40:04.087933] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fdd17fc047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fdd18293182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fdd18d8d265]))) 0-: received signum (15), shutting down [2016-03-15 14:40:04.100074] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:40:04.104676] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:40:04.105201] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:04.105213] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:04.105548] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:40:04.105559] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:40:04.105565] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:40:04.105600] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:40:04.105651] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:04.105659] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:05.832975] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:40:05.836787] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:05.839790] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:05.842386] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:05.842426] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:05.842475] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:05.842482] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:05.843092] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:05.843135] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:05.843144] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:05.843363] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:05.843400] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:05.843408] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:40:05.882009] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:05.882521] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:05.882577] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:40:05.895770] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:05.909190] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:05.993123] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:05.993188] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:40:06.009622] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:06.009651] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:06.027021] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:40:06.048951] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:06.048984] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:06.048992] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:06.048999] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:40:06.066423] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:06.066496] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:40:06.083457] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:06.083483] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:06.122676] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:06.122704] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:40:06.201464] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:06.201505] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:06.201513] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:06.201520] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:40:06.201578] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:06.201599] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:07.939131] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:09.163544] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:40:09.174762] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f602cf2f47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f602d202182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f602dcfc265]))) 0-: received signum (15), shutting down [2016-03-15 14:40:09.186331] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:40:09.189315] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:40:09.190014] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:09.190033] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:09.190544] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:40:09.190561] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:40:09.190572] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:40:09.190627] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:40:09.190704] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:09.190719] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:10.806336] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:40:10.809408] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:10.813225] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:10.816345] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:10.816384] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:10.816432] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:10.816447] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:10.817119] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:10.817156] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:10.817164] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:10.817426] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:10.817476] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:10.817490] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:40:10.822267] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:10.823169] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:10.823228] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:40:10.843264] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:10.940699] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:40:10.957962] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:10.958024] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:40:10.977787] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:10.977822] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:10.997585] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:10.997617] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:10.997625] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:10.997632] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:40:10.997675] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:10.997700] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:40:11.015529] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:11.455918] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:11.487446] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:11.487510] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:40:11.520957] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:11.520994] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:11.547210] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:11.580210] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:12.878498] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:14.250098] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:40:14.260309] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd0dbfa347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd0dc276182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd0dcd70265]))) 0-: received signum (15), shutting down [2016-03-15 14:40:14.272549] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:40:14.276753] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:40:14.278220] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:14.278239] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:14.278630] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:40:14.278642] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:40:14.278648] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:40:14.278683] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:40:14.278754] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:14.278822] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:16.069528] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:40:16.072625] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:16.075489] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:16.078773] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:16.078813] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:16.078863] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:16.078871] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:16.079409] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:16.079444] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:16.079452] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:16.079683] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:16.079728] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:16.079737] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:40:16.083535] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:16.084718] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:16.084771] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:40:16.138966] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:16.139022] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:40:16.180104] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:16.180149] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:16.363679] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:40:16.391910] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:40:16.409352] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:16.409388] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:16.409397] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:16.409403] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:40:16.409447] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:16.409465] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:16.409473] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:16.409479] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:40:16.410119] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:16.429829] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:16.429879] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:40:16.447266] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:16.447292] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:16.462260] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:16.462784] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:16.902099] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:18.139661] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:19.332338] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:40:19.342816] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f13cf8e047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f13cfbb3182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f13d06ad265]))) 0-: received signum (15), shutting down [2016-03-15 14:40:19.354193] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:40:19.358153] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:40:19.358642] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:19.358654] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:19.359763] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:40:19.359820] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:40:19.359832] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:40:19.359918] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:40:19.359996] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:19.360011] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:21.011364] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:40:21.015523] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:21.019359] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:21.022851] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:21.022895] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:21.022943] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:21.022950] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:21.023475] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:21.023524] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:21.023537] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:21.023759] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:21.023802] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:21.023815] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:40:21.028377] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:21.029336] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:21.029389] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:40:21.046266] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:21.108041] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:21.108101] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:40:21.128409] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:21.128441] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:21.148133] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:40:21.173092] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:21.173124] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:21.173131] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:21.173138] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:40:21.195360] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:21.706148] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:40:22.182932] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:22.197288] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:22.197339] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:40:22.224461] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:22.224490] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:22.243635] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:22.243663] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:23.085483] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:24.415047] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:40:24.425668] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fe342aa047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fe342d73182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fe34386d265]))) 0-: received signum (15), shutting down [2016-03-15 14:40:24.436764] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:40:24.441181] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:40:24.441686] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:24.441698] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:24.442036] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:40:24.442047] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:40:24.442053] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:40:24.442088] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:40:24.442137] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:24.442145] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:26.093472] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:40:26.096539] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:26.099160] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:26.102034] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:26.102073] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:26.102123] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:26.102131] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:26.102708] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:26.102781] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:26.102792] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:26.103040] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:26.103072] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:26.103079] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:40:26.106784] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:26.107843] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:26.107913] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:40:26.125035] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:26.190822] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:26.190885] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:40:26.221537] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:26.221563] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:26.252281] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:40:26.281408] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:26.281441] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:26.281448] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:26.281455] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:40:26.298652] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:26.458208] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:40:26.489909] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:26.489940] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:26.489947] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:26.489954] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:40:26.506668] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:26.518265] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:26.518321] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:40:26.538768] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:26.538789] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:26.621363] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:26.770918] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:28.162998] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:29.500093] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:40:29.511515] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f979f3b447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f979f687182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f97a0181265]))) 0-: received signum (15), shutting down [2016-03-15 14:40:29.522825] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:40:29.526932] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:40:29.527633] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:29.527658] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:29.528168] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:40:29.528183] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:40:29.528194] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:40:29.528242] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:40:29.528318] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:29.528333] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:31.153768] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:40:31.157837] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:31.160853] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:31.164275] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:31.164314] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:31.164363] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:31.164371] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:31.164941] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:31.164983] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:31.164991] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:31.165220] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:31.165257] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:31.165266] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:40:31.168668] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:31.207435] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:40:31.280151] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:40:31.300420] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:31.300450] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:31.300457] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:31.300465] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:40:31.300509] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:40:31.324706] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:31.343558] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:31.343610] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:40:31.363565] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:31.363590] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:31.384491] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:31.384524] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:31.620704] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:32.459939] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:32.506919] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:32.506975] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:40:32.530655] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:32.530683] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:32.565033] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:32.565074] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:33.225877] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:34.582423] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:40:34.593612] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f8b419cd47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f8b41ca0182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f8b4279a265]))) 0-: received signum (15), shutting down [2016-03-15 14:40:34.605860] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:40:34.609667] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:40:34.610176] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:34.610187] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:34.610526] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:40:34.610537] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:40:34.610543] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:40:34.610579] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:40:34.610630] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:34.610638] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:36.222404] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:40:36.225709] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:36.229445] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:36.234136] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:36.234177] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:36.234227] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:36.234234] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:36.234829] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:36.234871] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:36.234880] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:36.235150] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:36.235183] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:36.235190] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:40:36.239790] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:36.241029] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:36.241083] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:40:36.256118] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:36.296653] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:40:36.311073] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:36.311124] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:40:36.331264] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:36.331300] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:36.373255] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:36.373308] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:36.373329] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:36.373336] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:36.373343] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:40:37.185260] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:40:37.225411] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:37.225446] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:37.225453] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:37.225461] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:40:37.245980] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:37.273175] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:37.273222] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:40:37.323160] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:37.323188] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:37.347629] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:37.347657] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:38.295674] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:39.670800] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:40:39.681140] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f6daca6547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f6dacd38182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f6dad832265]))) 0-: received signum (15), shutting down [2016-03-15 14:40:39.692769] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:40:39.694992] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:40:39.695706] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:39.695723] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:39.696187] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:40:39.696199] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:40:39.696205] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:40:39.696242] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:40:39.696292] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:39.696300] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:41.422706] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:40:41.425935] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:41.429134] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:41.432964] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:41.433005] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:41.433053] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:41.433060] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:41.433611] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:41.433646] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:41.433653] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:41.433887] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:41.433927] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:41.433936] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:40:41.438163] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:41.439323] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:41.439379] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:40:41.460776] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:41.460841] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:40:41.479790] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:41.479809] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:41.530183] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:41.600428] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:40:41.645631] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:41.645676] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:41.645684] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:41.645691] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:40:41.645752] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:40:41.712027] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:41.712059] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:41.712075] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:41.712083] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:40:41.712361] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:41.729711] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:41.729752] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:40:41.750424] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:41.750460] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:41.771704] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:41.771732] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:43.494905] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:44.758421] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:40:44.772002] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f78f537047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f78f5643182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f78f613d265]))) 0-: received signum (15), shutting down [2016-03-15 14:40:44.783894] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:40:44.786056] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:40:44.786560] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:44.786571] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:44.787351] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:40:44.787365] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:40:44.787372] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:40:44.787408] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:40:44.787460] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:44.787468] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:46.443954] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:40:46.448414] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:46.451740] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:46.455098] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:46.455141] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:46.455194] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:46.455201] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:46.455773] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:46.455820] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:46.455828] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:46.456090] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:46.456123] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:46.456130] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:40:46.460046] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:46.461263] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:46.461320] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:40:46.688136] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:46.688221] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:40:46.804966] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:46.805002] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:46.854011] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:40:46.884668] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:46.884705] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:46.884713] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:46.884720] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:40:46.884767] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:40:46.918386] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:46.918431] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:46.919165] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:47.462888] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:47.491658] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:47.491712] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:40:47.510598] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:47.510616] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:47.530436] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:47.552484] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:48.516715] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:49.848780] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:40:49.859964] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f44f106147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f44f1334182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f44f1e2e265]))) 0-: received signum (15), shutting down [2016-03-15 14:40:49.872156] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:40:49.875280] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:40:49.875982] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:49.875999] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:49.876511] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:40:49.876529] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:40:49.876539] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:40:49.876590] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:40:49.876672] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:49.876686] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:51.531965] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:40:51.535042] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:51.538544] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:51.541961] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:51.542007] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:51.542124] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:51.542135] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:51.542648] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:51.542685] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:51.542693] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:51.542952] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:51.542986] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:51.542994] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:40:51.547198] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:51.548210] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:51.548265] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:40:51.565034] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:51.565085] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:40:51.585006] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:51.585026] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:51.634819] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:40:51.734931] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:51.734970] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:51.734977] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:51.734984] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:40:52.188158] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:40:52.230135] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:52.230171] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:52.230178] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:52.230185] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:40:52.273055] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:52.286997] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:52.287044] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:40:52.305320] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:52.305347] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:52.323007] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:52.324164] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:52.624368] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:53.603622] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:54.938981] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:40:54.949676] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f9c9d67247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f9c9d945182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f9c9e43f265]))) 0-: received signum (15), shutting down [2016-03-15 14:40:54.961191] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:40:54.964620] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:40:54.965128] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:54.965139] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:54.965479] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:40:54.965490] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:40:54.965496] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:40:54.965531] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:40:54.965581] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:40:54.965589] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:40:56.564368] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:40:56.568400] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:56.571233] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:56.582821] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:40:56.582865] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:56.582916] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:56.582924] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:56.583567] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:56.583607] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:56.583615] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:40:56.583827] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:40:56.583864] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:40:56.583873] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:40:56.588836] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:40:56.589056] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:56.589136] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:40:56.602563] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:56.618942] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:56.618993] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:40:56.659570] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:56.659597] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:56.712113] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:40:56.747342] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:40:56.765320] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:56.765349] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:56.765357] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:56.765364] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:40:56.781795] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:56.781856] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:40:56.800887] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:40:56.800912] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:40:56.835969] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:40:56.855451] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:56.855500] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:40:56.855519] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:40:56.855526] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:40:56.855533] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:40:56.855575] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:40:58.645143] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:00.026080] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:41:00.036742] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd78efdf47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd78f2b2182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd78fdac265]))) 0-: received signum (15), shutting down [2016-03-15 14:41:00.048328] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:41:00.052552] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:41:00.053095] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:00.053107] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:00.053451] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:41:00.053463] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:41:00.053469] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:41:00.053504] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:41:00.053565] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:00.053579] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:01.719656] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:41:01.725631] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:01.729885] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:01.733418] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:01.733462] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:01.733513] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:01.733521] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:01.734102] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:01.734145] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:01.734155] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:01.734406] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:01.734439] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:01.734446] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:41:01.738992] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:01.739695] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:01.739747] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:41:01.757513] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:01.825522] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:01.825625] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:41:01.846431] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:01.846458] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:01.946794] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:01.946843] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:01.946851] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:01.946858] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:41:01.946906] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:41:01.969285] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:41:01.989341] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:02.465987] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:02.481118] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:02.481167] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:41:02.502346] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:02.502369] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:02.523093] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:02.524815] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:03.796705] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:05.110709] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:41:05.121072] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd0e65a447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd0e6877182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd0e7371265]))) 0-: received signum (15), shutting down [2016-03-15 14:41:05.133237] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:41:05.135507] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:41:05.136009] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:05.136021] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:05.136363] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:41:05.136374] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:41:05.136379] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:41:05.136421] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:41:05.136472] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:05.136480] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:06.906841] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:41:06.913137] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:06.916326] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:06.919060] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:06.919101] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:06.919150] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:06.919157] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:06.919700] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:06.919735] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:06.919742] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:06.919966] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:06.920004] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:06.920017] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:41:06.923904] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:06.925160] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:06.925213] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:41:07.013598] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:41:07.061229] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:07.061292] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:41:07.090770] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:07.090799] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:07.146890] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:07.146919] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:41:07.180761] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:07.180802] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:07.180810] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:07.180817] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:41:07.180868] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:07.199446] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:07.215565] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:07.215607] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:41:07.233398] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:07.233432] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:07.253487] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:07.270199] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:07.628297] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:08.979955] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:10.196382] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:41:10.207708] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f0b5194347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f0b51c16182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f0b52710265]))) 0-: received signum (15), shutting down [2016-03-15 14:41:10.218951] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:41:10.222658] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:41:10.223757] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:10.223777] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:10.224285] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:41:10.224302] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:41:10.224313] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:41:10.224379] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:41:10.224456] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:10.224470] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:12.069866] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:41:12.073640] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:12.083418] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:12.090292] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:12.090343] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:12.090394] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:12.090401] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:12.090992] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:12.091034] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:12.091043] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:12.091265] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:12.091310] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:12.091321] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:41:12.095319] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:12.096551] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:12.096604] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:41:12.119575] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:12.223799] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:41:12.243627] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:12.243684] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:41:12.263271] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:12.263296] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:12.286762] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:12.286806] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:12.286827] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:12.286835] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:12.286842] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:41:12.286883] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:41:12.327974] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:13.192747] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:13.210349] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:13.210400] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:41:13.267162] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:13.267191] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:13.291095] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:13.320213] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:14.153028] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:15.281332] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:41:15.291482] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f55f130247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f55f15d5182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f55f20cf265]))) 0-: received signum (15), shutting down [2016-03-15 14:41:15.303598] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:41:15.305813] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:41:15.306328] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:15.306340] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:15.306682] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:41:15.306692] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:41:15.306698] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:41:15.306755] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:41:15.306814] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:15.306822] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:16.967518] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:41:16.974338] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:16.978829] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:16.982506] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:16.982556] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:16.982606] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:16.982613] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:16.983279] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:16.983321] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:16.983331] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:16.983612] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:16.983662] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:16.983671] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:41:16.987459] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:16.988738] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:16.988792] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:41:17.013007] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:17.164834] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:17.164903] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:41:17.182585] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:17.182604] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:17.201431] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:41:17.220074] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:17.220108] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:17.220116] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:17.220124] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:41:17.220169] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:41:17.250450] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:17.250477] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:17.468728] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:17.489009] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:17.489057] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:41:17.534916] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:17.534945] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:17.553511] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:17.553545] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:19.043835] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:20.368130] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:41:20.378246] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f91b41e147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f91b44b4182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f91b4fae265]))) 0-: received signum (15), shutting down [2016-03-15 14:41:20.390784] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:41:20.393105] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:41:20.393629] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:20.393641] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:20.393983] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:41:20.393995] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:41:20.394000] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:41:20.394036] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:41:20.394087] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:20.394096] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:21.963838] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:41:21.967638] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:21.972580] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:21.976056] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:21.976097] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:21.976145] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:21.976152] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:21.976654] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:21.976700] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:21.976711] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:21.976986] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:21.977030] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:21.977043] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:41:21.984069] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:21.985170] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:21.985245] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:41:22.047522] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:41:22.074947] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:22.075011] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:41:22.095345] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:22.095370] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:22.111633] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:22.111660] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:22.111668] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:22.111675] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:41:22.134760] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:22.193920] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:41:22.216024] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:22.216052] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:22.216059] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:22.216066] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:41:22.240571] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:22.271497] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:22.271548] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:41:22.290472] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:22.290490] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:22.305920] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:22.326569] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:22.631904] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:24.040735] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:25.458670] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:41:25.469126] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f033c22747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f033c4fa182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f033cff4265]))) 0-: received signum (15), shutting down [2016-03-15 14:41:25.480254] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:41:25.484118] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:41:25.484698] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:25.484711] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:25.485049] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:41:25.485060] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:41:25.485066] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:41:25.485101] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:41:25.485151] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:25.485158] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:27.054192] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:41:27.060415] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:27.065065] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:27.069263] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:27.069309] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:27.069358] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:27.069366] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:27.069904] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:27.069941] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:27.069949] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:27.070170] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:27.070214] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:27.070227] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:41:27.074930] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:27.075822] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:27.075875] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:41:27.089608] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:27.119431] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:41:27.230532] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:27.230605] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:41:27.257894] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:27.257928] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:27.302389] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:27.302437] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:27.302457] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:27.302464] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:27.302471] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:41:27.544908] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:41:28.195630] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:28.211428] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:28.211476] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:41:28.228666] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:28.228701] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:28.251627] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:28.292970] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:29.132300] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:30.541855] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:41:30.553212] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f0e8bce047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f0e8bfb3182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f0e8caad265]))) 0-: received signum (15), shutting down [2016-03-15 14:41:30.565096] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:41:30.568621] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:41:30.569125] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:30.569137] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:30.569489] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:41:30.569500] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:41:30.569506] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:41:30.569542] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:41:30.569593] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:30.569601] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:32.267359] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:41:32.272721] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:32.277797] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:32.280915] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:32.280955] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:32.281006] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:32.281014] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:32.281591] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:32.281629] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:32.281637] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:32.281881] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:32.281918] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:32.281926] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:41:32.286103] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:32.287439] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:32.287512] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:41:32.299763] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:32.632208] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:32.632270] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:41:32.657706] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:32.657740] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:32.704116] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:41:32.751939] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:41:32.781479] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:32.781511] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:32.781519] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:32.781526] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:41:32.781606] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:32.781626] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:32.781648] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:32.781660] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:32.781666] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:32.781673] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:41:32.782372] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:32.809147] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:32.809194] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:41:32.830000] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:32.830039] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:32.875445] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:32.975575] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:34.343365] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:35.635007] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:41:35.645296] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7eff1ee3d47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7eff1f110182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7eff1fc0a265]))) 0-: received signum (15), shutting down [2016-03-15 14:41:35.656790] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:41:35.660832] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:41:35.661367] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:35.661379] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:35.661713] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:41:35.661724] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:41:35.661736] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:41:35.661771] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:41:35.661824] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:35.661832] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:37.345270] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:41:37.347796] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:37.350899] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:37.353537] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:37.353580] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:37.353632] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:37.353640] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:37.354228] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:37.354264] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:37.354271] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:37.354535] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:37.354571] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:37.354579] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:41:37.358067] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:37.359038] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:37.359092] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:41:37.414179] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:41:37.431111] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:37.431169] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:41:37.457019] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:37.457047] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:37.480614] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:37.480654] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:37.480683] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:37.480690] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:37.480696] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:41:37.632295] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:37.962418] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:41:38.472941] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:38.490318] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:38.490367] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:41:38.512860] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:38.512887] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:38.535789] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:38.548421] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:39.414379] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:40.719173] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:41:40.729479] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd5912d847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd5915ab182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd5920a5265]))) 0-: received signum (15), shutting down [2016-03-15 14:41:40.741857] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:41:40.744123] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:41:40.744624] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:40.744635] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:40.744971] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:41:40.744982] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:41:40.744988] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:41:40.745023] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:41:40.745073] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:40.745080] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:43.728836] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:41:43.738939] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:43.754641] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:43.768555] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:43.768610] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:43.768662] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:43.768670] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:43.769243] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:43.769280] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:43.769287] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:43.769552] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:43.769587] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:43.769594] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:41:43.781811] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:43.782847] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:43.782921] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:41:43.869572] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:44.381123] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:44.400176] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:44.400239] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:41:44.418486] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:44.418511] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:44.435919] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:41:44.470302] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:41:44.486472] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:44.486524] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:41:44.504316] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:44.504347] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:44.529389] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:44.529423] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:44.529438] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:44.529445] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:41:44.529494] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:44.529515] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:44.529536] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:44.529547] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:44.529552] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:44.529558] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:41:44.529591] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:44.529610] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:45.818768] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:41:45.839190] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f37ccc4447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f37ccf17182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f37cda11265]))) 0-: received signum (15), shutting down [2016-03-15 14:41:45.856321] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:41:45.859409] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:41:45.860131] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:45.860149] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:45.860655] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:41:45.860672] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:41:45.860683] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:41:45.860735] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:41:45.860818] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:45.860833] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:47.520240] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:41:47.527256] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:47.530806] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:47.553496] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:47.553544] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:47.553597] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:47.553604] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:47.554124] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:47.554166] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:47.554175] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:47.554398] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:47.554439] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:47.554452] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:41:47.562893] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:47.562950] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:41:47.576963] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:47.589896] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:47.624648] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:47.624711] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:41:47.647866] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:47.647895] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:47.720255] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:47.720321] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:41:47.754895] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:47.754925] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:47.792981] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:47.793014] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:47.793022] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:47.793029] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:41:47.793084] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:41:47.819524] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:47.819552] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:47.819566] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:47.819573] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:41:47.819616] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:41:47.840037] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:47.840066] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:47.840080] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:48.792564] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:50.848989] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:50.931628] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:41:50.942804] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f1d5523a47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f1d5550d182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f1d56007265]))) 0-: received signum (15), shutting down [2016-03-15 14:41:50.954111] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:41:50.956958] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:41:50.957668] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:50.957685] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:50.958183] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:41:50.958200] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:41:50.958209] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:41:50.958260] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:41:50.958335] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:50.958349] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:52.566292] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:41:52.569516] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:52.572610] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:52.575592] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:52.575631] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:52.575679] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:52.575687] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:52.576243] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:52.576280] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:52.576288] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:52.576527] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:52.576569] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:52.576578] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:41:52.582104] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:52.582151] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:41:52.690098] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:52.690177] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:41:52.712597] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:52.712627] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:52.731615] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:41:52.764183] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:52.764214] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:52.764222] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:52.764228] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:41:52.764276] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:41:52.816048] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:52.816081] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:52.816660] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:52.905667] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:53.476000] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:53.509123] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:53.509178] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:41:53.548125] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:53.548157] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:53.626342] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:53.626370] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:54.961653] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:56.014901] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:41:56.025252] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f69c495647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f69c4c29182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f69c5723265]))) 0-: received signum (15), shutting down [2016-03-15 14:41:56.036813] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:41:56.040897] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:41:56.041425] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:56.041437] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:56.041827] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:41:56.041841] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:41:56.041846] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:41:56.041883] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:41:56.041934] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:41:56.041942] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:41:59.252513] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:41:59.255960] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:59.258847] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:59.265142] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:41:59.265183] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:59.265234] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:59.265241] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:59.265734] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:59.265773] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:59.265781] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:41:59.265995] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:41:59.266030] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:41:59.266037] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:41:59.270324] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:41:59.271065] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:59.271117] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:41:59.296245] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:59.336045] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:59.336101] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:41:59.355621] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:59.355651] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:59.388550] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:41:59.407195] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:59.407267] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:41:59.434116] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:41:59.434151] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:41:59.485280] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:41:59.485317] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:59.485324] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:59.485331] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:41:59.485379] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:59.485403] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:41:59.485418] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:41:59.485424] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:41:59.485430] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:41:59.485464] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:41:59.555356] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:41:59.581069] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:01.098022] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:42:01.108145] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f636eb9347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f636ee66182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f636f960265]))) 0-: received signum (15), shutting down [2016-03-15 14:42:01.120287] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:42:01.122519] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:42:01.123056] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:01.123070] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:01.123444] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:42:01.123455] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:42:01.123461] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:42:01.123497] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:42:01.123549] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:01.123556] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:03.059768] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:42:03.063108] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:03.206418] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:03.213096] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:03.213146] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:03.213199] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:03.213207] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:03.213781] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:03.213818] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:03.213825] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:03.214044] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:03.214081] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:03.214092] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:42:03.225222] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:03.225893] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:03.225948] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:42:03.379176] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:03.410224] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:03.519733] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:42:03.571513] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:03.571549] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:03.571557] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:03.571564] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:42:03.571611] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:42:03.617282] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:03.617362] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:42:03.684938] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:03.684966] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:03.724382] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:03.724429] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:03.724475] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:42:03.741480] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:03.741508] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:03.763455] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:03.763502] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:03.763521] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:03.763528] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:03.763534] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:42:03.763584] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:03.763598] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:05.282557] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:06.186625] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:42:06.197428] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f36fadd847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f36fb0ab182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f36fbba5265]))) 0-: received signum (15), shutting down [2016-03-15 14:42:06.209451] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:42:06.213072] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:42:06.213613] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:06.213625] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:06.213978] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:42:06.213990] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:42:06.213996] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:42:06.214031] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:42:06.214082] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:06.214090] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:07.917917] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:42:07.921551] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:07.925207] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:07.932563] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:07.932602] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:07.932650] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:07.932658] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:07.933284] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:07.933323] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:07.933330] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:07.933558] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:07.933593] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:07.933601] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:42:07.954267] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:07.955457] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:07.955512] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:42:07.972206] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:08.064152] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:08.064250] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:42:08.122162] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:08.122196] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:08.140465] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:42:08.165199] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:08.165233] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:08.165241] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:08.165248] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:42:08.165290] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:42:08.188292] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:08.188321] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:08.478777] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:08.511754] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:08.511809] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:42:08.562647] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:08.562670] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:08.598971] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:08.599001] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:10.011316] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:11.277509] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:42:11.288905] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa7e4be747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa7e4eba182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa7e59b4265]))) 0-: received signum (15), shutting down [2016-03-15 14:42:11.300962] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:42:11.303164] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:42:11.303673] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:11.303688] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:11.304042] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:42:11.304053] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:42:11.304064] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:42:11.304101] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:42:11.304152] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:11.304160] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:12.885867] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:42:12.888854] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:12.895731] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:12.898946] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:12.898985] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:12.899036] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:12.899044] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:12.899542] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:12.899579] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:12.899587] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:12.899800] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:12.899837] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:12.899846] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:42:12.903828] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:12.905142] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:12.905195] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:42:12.985583] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:12.985650] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:42:13.010216] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:13.010251] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:13.106506] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:42:13.131300] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:13.131335] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:13.131349] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:13.131357] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:42:13.131416] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:42:13.155083] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:13.204608] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:13.237614] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:13.237663] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:42:13.266611] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:13.266632] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:13.284897] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:13.305598] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:13.638867] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:14.960059] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:16.367013] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:42:16.377276] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7eff68f3b47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7eff6920e182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7eff69d08265]))) 0-: received signum (15), shutting down [2016-03-15 14:42:16.388741] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:42:16.392825] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:42:16.393455] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:16.393469] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:16.393815] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:42:16.393826] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:42:16.393832] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:42:16.393867] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:42:16.393918] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:16.393926] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:17.997417] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:42:18.000375] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:18.003235] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:18.006047] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:18.006097] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:18.006148] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:18.006156] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:18.006754] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:18.006797] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:18.006805] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:18.007059] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:18.007097] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:18.007105] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:42:18.010728] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:18.011352] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:18.011437] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:42:18.027241] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:18.086225] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:18.086291] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:42:18.113823] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:18.113857] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:18.176571] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:42:18.233357] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:18.233399] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:18.233423] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:18.233429] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:18.233436] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:42:18.233486] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:42:19.205734] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:19.219628] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:19.219686] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:42:19.240659] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:19.240688] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:19.260521] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:19.354019] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:20.068170] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:21.451252] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:42:21.461683] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa4eabd647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa4eaea9182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa4eb9a3265]))) 0-: received signum (15), shutting down [2016-03-15 14:42:21.473543] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:42:21.477667] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:42:21.478387] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:21.478406] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:21.478972] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:42:21.478991] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:42:21.479001] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:42:21.479054] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:42:21.479132] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:21.479147] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:23.146822] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:42:23.149872] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:23.152504] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:23.155422] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:23.155464] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:23.155512] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:23.155520] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:23.156094] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:23.156131] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:23.156139] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:23.156363] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:23.156401] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:23.156412] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:42:23.162596] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:23.163273] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:23.163340] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:42:23.210716] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:23.311193] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:42:23.329453] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:23.329511] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:42:23.346502] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:23.346524] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:23.377628] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:42:23.396747] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:23.396790] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:23.396832] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:23.396843] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:23.396850] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:42:23.396893] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:23.481614] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:23.531794] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:23.531849] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:42:23.550638] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:23.550657] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:23.574455] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:23.574654] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:25.218620] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:26.537754] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:42:26.548097] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f6fa5b1047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f6fa5de3182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f6fa68dd265]))) 0-: received signum (15), shutting down [2016-03-15 14:42:26.560039] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:42:26.562236] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:42:26.562761] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:26.562777] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:26.563142] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:42:26.563154] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:42:26.563160] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:42:26.563195] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:42:26.563246] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:26.563254] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:28.120652] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:42:28.124103] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:28.126923] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:28.129743] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:28.129781] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:28.129832] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:28.129840] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:28.130338] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:28.130376] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:28.130383] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:28.130605] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:28.130641] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:28.130649] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:42:28.136095] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:28.211565] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:42:28.298331] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:42:28.349651] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:28.349685] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:28.349693] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:28.349700] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:42:28.349745] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:42:28.366533] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:28.379207] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:28.379251] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:42:28.398756] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:28.398775] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:28.415829] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:28.426086] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:28.641474] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:29.482963] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:29.497009] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:29.497057] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:42:29.531828] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:29.531856] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:29.548852] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:29.548879] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:30.192859] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:31.625358] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:42:31.636515] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f07e24c147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f07e2794182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f07e328e265]))) 0-: received signum (15), shutting down [2016-03-15 14:42:31.648025] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:42:31.651312] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:42:31.652360] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:31.652377] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:31.652773] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:42:31.652785] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:42:31.652791] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:42:31.652827] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:42:31.652878] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:31.652886] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:33.241906] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:42:33.245298] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:33.249659] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:33.253135] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:33.253177] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:33.253226] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:33.253234] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:33.253814] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:33.253857] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:33.253866] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:33.254084] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:33.254116] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:33.254123] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:42:33.259793] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:33.261569] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:33.261623] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:42:33.277240] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:33.312575] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:33.312638] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:42:33.345235] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:33.345271] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:33.404283] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:42:33.432127] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:42:33.498957] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:33.499002] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:33.499010] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:33.499017] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:42:33.499074] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:34.208507] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:34.225885] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:34.225942] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:42:34.253643] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:34.253667] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:34.281648] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:34.349987] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:35.317401] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:36.713354] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:42:36.723856] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f621f26347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f621f536182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f6220030265]))) 0-: received signum (15), shutting down [2016-03-15 14:42:36.737040] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:42:36.741321] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:42:36.741850] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:36.741862] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:36.742194] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:42:36.742205] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:42:36.742210] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:42:36.742245] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:42:36.742298] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:36.742306] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:38.302962] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:42:38.306908] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:38.310811] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:38.313524] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:38.313564] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:38.313615] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:38.313623] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:38.314111] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:38.314148] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:38.314156] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:38.314387] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:38.314424] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:38.314432] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:42:38.318963] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:38.319639] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:38.319692] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:42:38.332387] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:38.402698] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:38.402834] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:42:38.445743] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:38.445777] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:38.490008] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:42:38.509285] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:38.509316] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:38.509324] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:38.509331] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:42:38.509381] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:42:38.528416] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:38.904321] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:38.905096] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:38.920916] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:38.920974] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:42:38.944713] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:38.944748] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:38.969109] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:39.244920] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:40.375262] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:41.798898] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:42:41.809330] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f13c280a47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f13c2add182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f13c35d7265]))) 0-: received signum (15), shutting down [2016-03-15 14:42:41.821479] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:42:41.824568] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:42:41.825261] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:41.825279] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:41.825799] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:42:41.825817] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:42:41.825827] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:42:41.825879] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:42:41.825957] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:41.825971] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:43.426764] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:42:43.430046] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:43.449593] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:43.452813] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:43.452854] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:43.452916] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:43.452927] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:43.453475] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:43.453510] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:43.453517] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:43.453748] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:43.453791] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:43.453801] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:42:43.459526] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:43.460454] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:43.460511] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:42:43.527440] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:43.527514] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:42:43.545980] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:43.546013] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:43.583603] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:43.583636] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:42:43.604117] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:42:43.622808] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:43.622838] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:43.622845] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:43.622852] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:42:43.633371] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:43.643918] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:44.485663] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:44.526458] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:44.526517] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:42:44.551642] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:44.551668] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:44.573544] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:44.573570] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:45.515579] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:46.889505] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:42:46.902948] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f0ca2fbc47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f0ca328f182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f0ca3d89265]))) 0-: received signum (15), shutting down [2016-03-15 14:42:46.914459] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:42:46.916946] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:42:46.917456] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:46.917468] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:46.917809] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:42:46.917820] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:42:46.917826] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:42:46.917862] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:42:46.917913] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:46.917921] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:48.607575] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:42:48.612497] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:48.618716] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:48.622964] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:48.623008] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:48.623060] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:48.623068] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:48.623687] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:48.623723] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:48.623731] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:48.623988] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:48.624025] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:48.624032] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:42:48.628421] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:48.629119] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:48.629174] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:42:48.677172] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:48.677260] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:42:48.700735] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:48.700768] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:48.739481] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:48.739515] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:48.739522] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:48.739529] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:42:48.754436] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:42:48.781217] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:49.210970] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:42:49.276563] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:49.276608] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:49.276616] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:49.276623] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:42:49.313495] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:49.326368] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:49.326412] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:42:49.346287] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:49.346312] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:49.365143] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:49.371907] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:49.644950] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:50.685292] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:51.977350] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:42:51.988852] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa92a9c447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa92ac97182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa92b791265]))) 0-: received signum (15), shutting down [2016-03-15 14:42:52.000172] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:42:52.004172] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:42:52.004837] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:52.004850] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:52.005191] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:42:52.005202] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:42:52.005208] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:42:52.005242] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:42:52.005292] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:52.005300] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:53.622136] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:42:53.625150] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:53.629397] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:53.632240] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:53.632279] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:53.632324] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:53.632331] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:53.632928] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:53.632964] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:53.632972] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:53.633243] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:53.633275] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:53.633283] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:42:53.637199] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:53.637814] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:53.637865] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:42:53.650571] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:53.690707] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:53.690774] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:42:53.734939] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:53.734970] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:53.767310] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:53.796448] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:53.796480] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:53.796488] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:53.796495] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:42:53.796535] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:42:53.854008] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:53.854068] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:42:53.878324] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:53.878351] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:53.895756] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:53.895784] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:42:53.914629] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:53.914659] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:53.914666] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:53.914673] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:42:53.914720] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:53.914771] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:55.700201] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:57.059945] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:42:57.070460] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f219990847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f2199bdb182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f219a6d5265]))) 0-: received signum (15), shutting down [2016-03-15 14:42:57.082196] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:42:57.085515] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:42:57.086568] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:57.086587] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:57.087126] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:42:57.087144] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:42:57.087154] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:42:57.087208] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:42:57.087291] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:42:57.087306] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:42:59.089632] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:42:59.093932] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:59.097724] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:59.108265] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:42:59.108309] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:59.108360] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:59.108368] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:59.108899] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:59.108934] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:59.108942] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:42:59.109167] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:42:59.109205] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:42:59.109216] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:42:59.113771] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:42:59.114796] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:59.114865] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:42:59.132416] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:59.221232] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:59.221298] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:42:59.242038] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:59.242072] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:59.280775] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:59.280805] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:42:59.320635] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:42:59.344363] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:59.344403] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:59.344423] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:42:59.344430] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:42:59.344437] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:42:59.488865] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:42:59.506134] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:42:59.506184] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:42:59.553303] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:42:59.553326] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:42:59.569285] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:42:59.570189] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:01.170560] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:02.149091] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:43:02.159679] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa71533147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa715604182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa7160fe265]))) 0-: received signum (15), shutting down [2016-03-15 14:43:02.171606] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:43:02.173848] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:43:02.174366] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:02.174377] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:02.174718] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:43:02.174729] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:43:02.174758] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:43:02.174800] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:43:02.174850] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:02.174858] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:03.985413] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:43:03.989081] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:03.993939] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:03.997592] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:03.997630] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:03.997680] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:03.997687] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:03.998255] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:03.998292] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:03.998300] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:03.998521] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:03.998597] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:03.998607] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:43:04.005773] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:04.006772] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:04.006832] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:43:04.065744] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:43:04.183353] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:04.183427] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:43:05.216400] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:05.216445] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:05.239882] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:05.239931] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:05.239956] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:05.239963] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:05.239970] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:43:05.301892] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:43:05.328930] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:05.328966] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:05.328974] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:05.328981] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:43:05.329626] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:05.390184] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:05.406512] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:05.406566] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:43:05.552435] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:05.552470] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:05.570361] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:05.570385] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:06.062574] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:07.236873] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:43:07.247602] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f78db0b547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f78db388182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f78dbe82265]))) 0-: received signum (15), shutting down [2016-03-15 14:43:07.259164] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:43:07.263043] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:43:07.263673] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:07.263686] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:07.264040] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:43:07.264058] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:43:07.264064] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:43:07.264099] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:43:07.264150] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:07.264158] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:08.907432] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:43:08.910889] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:08.914149] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:08.917392] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:08.917433] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:08.917483] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:08.917490] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:08.918023] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:08.918058] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:08.918066] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:08.918293] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:08.918330] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:08.918339] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:43:08.922754] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:08.923487] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:08.923541] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:43:08.936866] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:08.985204] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:08.985283] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:43:09.004550] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:09.004568] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:09.140872] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:09.140909] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:09.140923] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:09.140931] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:43:09.140976] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:43:09.160861] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:43:09.190053] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:10.215641] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:11.484812] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:11.484932] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:11.484988] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:43:11.511258] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:11.511280] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:11.534127] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:11.542701] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:12.448625] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:43:12.459928] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd192d9847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd19306b182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd193b65265]))) 0-: received signum (15), shutting down [2016-03-15 14:43:12.471217] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:43:12.474131] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:43:12.475723] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:12.475742] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:12.476130] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:43:12.476143] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:43:12.476149] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:43:12.476185] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:43:12.476237] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:12.476245] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:14.072167] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:43:14.077536] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:14.083900] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:14.087146] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:14.087187] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:14.087239] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:14.087246] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:14.087813] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:14.087848] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:14.087856] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:14.088088] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:14.088134] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:14.088144] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:43:14.092004] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:14.092819] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:14.092872] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:43:14.104854] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:14.141354] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:14.141413] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:43:14.159702] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:14.159735] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:14.240756] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:43:14.264526] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:43:14.280859] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:14.280896] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:14.280903] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:14.280910] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:43:14.280963] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:14.491673] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:14.505571] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:14.505614] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:43:14.522676] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:14.522700] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:14.591297] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:14.591330] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:16.149048] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:17.536432] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:43:17.547343] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f4507dd347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f45080a6182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f4508ba0265]))) 0-: received signum (15), shutting down [2016-03-15 14:43:17.559337] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:43:17.563299] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:43:17.563986] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:17.564000] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:17.564378] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:43:17.564389] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:43:17.564395] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:43:17.564432] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:43:17.564484] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:17.564492] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:19.221740] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:43:19.224992] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:19.229393] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:19.233345] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:19.233387] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:19.233436] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:19.233444] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:19.234021] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:19.234058] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:19.234066] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:19.234320] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:19.234352] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:19.234359] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:43:19.238722] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:19.239609] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:19.239663] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:43:19.303979] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:19.304041] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:43:19.325652] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:19.325686] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:19.342779] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:19.342805] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:43:19.379714] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:43:19.417669] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:19.417703] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:19.417711] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:19.417718] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:43:19.417763] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:19.650272] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:20.492746] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:20.547899] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:20.547956] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:43:20.618254] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:20.618291] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:20.639943] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:20.639987] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:21.296568] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:22.622285] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:43:22.632545] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fb17cfad47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fb17d280182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fb17dd7a265]))) 0-: received signum (15), shutting down [2016-03-15 14:43:22.643789] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:43:22.647504] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:43:22.648008] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:22.648020] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:22.648353] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:43:22.648364] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:43:22.648370] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:43:22.648405] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:43:22.648457] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:22.648465] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:24.322822] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:43:24.326212] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:24.330194] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:24.335273] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:24.335313] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:24.335363] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:24.335370] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:24.335907] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:24.335942] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:24.335950] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:24.336170] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:24.336218] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:24.336232] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:43:24.343567] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:24.344238] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:24.344293] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:43:24.361602] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:24.407102] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:24.407168] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:43:24.433766] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:24.433801] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:24.487417] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:24.487453] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:24.487461] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:24.487468] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:43:24.487514] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:43:24.528070] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:43:24.552017] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:25.218958] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:25.261152] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:25.261213] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:43:25.279823] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:25.279848] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:25.303469] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:25.334022] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:26.400714] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:27.707642] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:43:27.718100] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fafa152547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fafa17f8182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fafa22f2265]))) 0-: received signum (15), shutting down [2016-03-15 14:43:27.730290] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:43:27.734186] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:43:27.734704] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:27.734716] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:27.735649] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:43:27.735667] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:43:27.735678] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:43:27.735736] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:43:27.735812] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:27.735826] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:29.358894] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:43:29.362487] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:29.365878] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:29.369514] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:29.369555] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:29.369606] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:29.369614] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:29.370189] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:29.370225] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:29.370232] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:29.370460] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:29.370504] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:29.370517] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:43:29.376558] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:29.377294] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:29.377347] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:43:29.393123] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:29.508123] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:29.508188] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:43:29.562555] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:29.562602] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:29.628934] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:43:29.655493] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:43:29.673807] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:29.673838] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:29.673846] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:29.673853] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:43:29.703020] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:29.703046] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:29.703103] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:29.703127] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:29.703135] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:29.703142] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:43:29.703892] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:29.722252] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:29.722298] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:43:29.749503] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:29.749536] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:29.772622] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:29.808421] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:31.434544] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:32.792800] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:43:32.804274] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f579ea2f47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f579ed02182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f579f7fc265]))) 0-: received signum (15), shutting down [2016-03-15 14:43:32.816263] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:43:32.820259] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:43:32.821140] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:32.821297] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:32.821854] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:43:32.821871] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:43:32.821882] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:43:32.821933] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:43:32.822058] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:32.822073] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:34.518958] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:43:34.522197] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:34.525825] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:34.532058] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:34.532103] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:34.532155] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:34.532163] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:34.532684] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:34.532742] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:34.532755] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:34.533000] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:34.533044] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:34.533057] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:43:34.536858] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:34.537896] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:34.537950] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:43:34.600174] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:34.600237] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:43:34.616804] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:34.616823] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:34.651995] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:34.652022] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:43:34.667684] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:43:34.684201] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:34.684230] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:34.684238] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:34.684245] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:43:34.684287] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:35.125689] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:35.495603] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:35.508401] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:35.508452] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:43:35.526942] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:35.526969] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:35.556044] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:35.594308] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:36.593053] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:37.876873] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:43:37.887492] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f53cadcc47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f53cb09f182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f53cbb99265]))) 0-: received signum (15), shutting down [2016-03-15 14:43:37.898865] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:43:37.902945] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:43:37.903559] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:37.903573] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:37.903916] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:43:37.903928] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:43:37.903934] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:43:37.903969] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:43:37.904020] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:37.904028] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:39.606709] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:43:39.610473] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:39.613437] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:39.616287] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:39.616328] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:39.616379] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:39.616386] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:39.616975] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:39.617017] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:39.617026] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:39.617294] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:39.617326] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:39.617333] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:43:39.621098] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:39.622107] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:39.622159] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:43:39.658374] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:43:39.677971] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:39.678040] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:43:39.752162] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:39.752191] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:40.049155] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:40.049201] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:40.049227] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:40.049235] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:40.049242] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:43:40.221067] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:43:40.262578] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:40.262614] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:40.262622] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:40.262629] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:43:40.281247] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:40.297916] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:40.297961] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:43:40.319597] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:40.319619] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:40.336860] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:40.338636] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:40.653703] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:41.677689] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:42.962724] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:43:42.973389] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f92255ad47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f9225880182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f922637a265]))) 0-: received signum (15), shutting down [2016-03-15 14:43:42.984882] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:43:42.988163] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:43:42.988675] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:42.988687] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:42.989025] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:43:42.989036] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:43:42.989042] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:43:42.989077] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:43:42.989127] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:42.989135] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:44.810360] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:43:44.814192] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:44.817152] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:44.820649] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:44.820698] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:44.820748] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:44.820755] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:44.821348] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:44.821386] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:44.821393] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:44.821616] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:44.821653] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:44.821662] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:43:44.825472] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:44.826717] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:44.826828] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:43:44.841312] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:44.858336] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:44.889119] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:44.889188] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:43:44.924850] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:44.924884] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:44.942959] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:43:44.961077] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:44.961112] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:44.961123] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:44.961130] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:43:44.981234] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:44.981346] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:43:44.998074] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:44.998101] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:45.112693] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:45.112734] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:43:45.128344] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:45.128370] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:45.128377] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:45.128384] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:43:45.128443] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:45.128465] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:46.882112] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:48.047832] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:43:48.059862] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd253d6b47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd25403e182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd254b38265]))) 0-: received signum (15), shutting down [2016-03-15 14:43:48.072054] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:43:48.074333] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:43:48.074923] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:48.074937] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:48.075278] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:43:48.075289] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:43:48.075294] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:43:48.075330] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:43:48.075380] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:48.075388] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:49.670561] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:43:49.673557] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:49.677484] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:49.681055] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:49.681096] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:49.681146] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:49.681154] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:49.681709] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:49.681743] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:49.681751] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:49.681978] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:49.682020] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:49.682033] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:43:49.685492] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:49.686258] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:49.686311] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:43:49.701765] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:49.738155] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:49.738216] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:43:49.757605] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:49.757631] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:49.879179] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:49.879214] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:49.879222] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:49.879229] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:43:49.901360] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:43:49.919101] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:50.498063] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:43:50.534212] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:50.534258] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:50.534265] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:50.534284] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:43:50.543280] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:50.559004] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:50.559059] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:43:50.588998] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:50.589033] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:50.607703] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:50.614596] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:51.742959] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:53.139167] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:43:53.150508] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fc96b78047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fc96ba53182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fc96c54d265]))) 0-: received signum (15), shutting down [2016-03-15 14:43:53.162665] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:43:53.166621] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:43:53.167371] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:53.167385] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:53.167731] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:43:53.167742] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:43:53.167748] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:43:53.167785] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:43:53.167835] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:53.167843] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:54.773524] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:43:54.776944] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:54.780309] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:54.784329] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:54.784369] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:54.784418] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:54.784425] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:54.785010] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:54.785051] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:54.785060] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:54.785290] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:54.785327] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:54.785336] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:43:54.791564] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:54.791836] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:54.791887] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:43:54.859992] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:43:54.892405] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:54.892462] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:43:54.914883] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:54.914909] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:55.000544] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:55.000588] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:55.000611] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:55.000617] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:55.000624] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:43:55.224130] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:43:55.256636] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:55.256668] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:43:55.256675] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:43:55.256682] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:43:55.267342] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:55.278931] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:55.278976] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:43:55.296154] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:43:55.296188] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:43:55.318780] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:55.323146] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:43:55.656553] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:56.849222] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:58.226948] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:43:58.238217] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fefce80047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fefcead3182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fefcf5cd265]))) 0-: received signum (15), shutting down [2016-03-15 14:43:58.249839] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:43:58.253882] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:43:58.254593] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:58.254610] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:58.255554] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:43:58.255568] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:43:58.255574] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:43:58.255609] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:43:58.255660] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:43:58.255668] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:43:59.925556] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:43:59.929567] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:59.934489] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:59.937810] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:43:59.937853] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:59.937902] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:59.937910] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:59.938452] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:59.938489] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:59.938497] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:43:59.938722] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:43:59.938790] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:43:59.938804] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:43:59.943609] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:43:59.944233] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:59.944285] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:43:59.958980] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:43:59.994953] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:43:59.995019] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:44:00.014264] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:00.014291] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:00.041870] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:44:00.079591] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:00.079637] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:00.079661] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:00.079668] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:00.079675] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:44:01.225314] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:44:01.262455] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:01.262490] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:01.262497] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:01.262504] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:44:01.294497] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:01.308905] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:01.308952] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:44:01.332593] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:01.332627] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:01.351735] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:01.355351] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:02.001604] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:03.312310] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:44:03.322964] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fba5f8d747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fba5fbaa182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fba606a4265]))) 0-: received signum (15), shutting down [2016-03-15 14:44:03.334718] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:44:03.339148] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:44:03.340164] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:03.340180] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:03.340520] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:44:03.340532] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:44:03.340538] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:44:03.340572] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:44:03.340623] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:03.340631] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:05.001131] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:44:05.004230] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:05.007360] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:05.012772] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:05.012813] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:05.012863] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:05.012870] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:05.013419] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:05.013475] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:05.013484] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:05.013715] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:05.013750] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:05.013757] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:44:05.019028] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:05.019847] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:05.019899] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:44:05.172929] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:05.216969] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:05.217038] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:44:05.247422] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:05.247460] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:05.305622] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:44:05.329682] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:05.329719] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:05.329727] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:05.329734] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:44:05.329779] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:44:05.363435] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:05.363464] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:05.501749] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:05.629273] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:05.629329] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:44:05.677137] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:05.677187] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:05.752908] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:05.752947] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:07.076364] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:08.396724] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:44:08.407302] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f69bb2d847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f69bb5ab182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f69bc0a5265]))) 0-: received signum (15), shutting down [2016-03-15 14:44:08.420323] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:44:08.424488] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:44:08.424999] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:08.425011] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:08.425357] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:44:08.425368] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:44:08.425374] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:44:08.425409] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:44:08.425460] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:08.425468] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:10.069903] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:44:10.072625] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:10.075061] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:10.077351] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:10.077390] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:10.077439] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:10.077447] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:10.077943] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:10.077982] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:10.077990] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:10.078207] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:10.078245] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:10.078254] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:44:10.099958] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:10.100279] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:10.100338] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:44:10.112282] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:10.112342] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:44:10.131362] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:10.131395] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:10.203022] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:44:10.251932] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:10.251968] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:10.251976] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:10.251983] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:44:10.252036] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:44:10.276123] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:10.288254] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:10.288304] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:44:10.304251] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:10.304276] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:10.322938] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:10.338308] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:10.659267] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:12.157823] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:13.483836] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:44:13.495456] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f708f02b47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f708f2fe182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f708fdf8265]))) 0-: received signum (15), shutting down [2016-03-15 14:44:13.507323] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:44:13.511925] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:44:13.512631] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:13.512648] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:13.513119] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:44:13.513132] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:44:13.513138] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:44:13.513179] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:44:13.513231] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:13.513239] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:15.093869] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:44:15.096667] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:15.099639] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:15.102356] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:15.102397] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:15.102448] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:15.102456] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:15.103010] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:15.103046] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:15.103053] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:15.103286] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:15.103331] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:15.103343] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:44:15.107728] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:15.108580] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:15.108637] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:44:15.133406] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:15.172221] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:44:15.191482] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:15.191538] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:44:15.210582] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:15.210605] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:15.264742] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:15.264787] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:15.264810] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:15.264824] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:15.264831] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:44:16.145948] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:44:16.241936] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:16.283346] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:16.283404] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:44:16.299543] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:16.299568] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:16.316539] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:16.334015] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:17.164843] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:18.570205] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:44:18.580863] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7eff1393247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7eff13c05182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7eff146ff265]))) 0-: received signum (15), shutting down [2016-03-15 14:44:18.592827] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:44:18.595147] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:44:18.595686] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:18.595699] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:18.596036] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:44:18.596047] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:44:18.596053] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:44:18.596092] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:44:18.596143] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:18.596151] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:20.296541] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:44:20.299695] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:20.302931] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:20.305825] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:20.305866] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:20.305916] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:20.305932] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:20.306504] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:20.306540] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:20.306547] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:20.306818] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:20.306852] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:20.306859] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:44:20.311378] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:20.312339] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:20.312393] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:44:20.334617] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:20.352265] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:20.352318] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:44:20.374167] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:20.374189] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:20.453272] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:20.453310] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:20.453323] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:20.453334] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:44:20.453387] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:44:20.490754] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:44:20.567615] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:20.624663] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:20.624721] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:44:20.686579] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:20.686620] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:20.707302] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:20.707332] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:22.368248] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:23.657680] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:44:23.668317] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ff55f50c47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ff55f7df182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ff5602d9265]))) 0-: received signum (15), shutting down [2016-03-15 14:44:23.680264] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:44:23.683419] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:44:23.684128] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:23.684146] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:23.684658] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:44:23.684676] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:44:23.684685] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:44:23.684737] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:44:23.684812] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:23.684827] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:25.387122] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:44:25.390818] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:25.393734] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:25.396700] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:25.396740] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:25.396789] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:25.396796] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:25.397352] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:25.397387] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:25.397394] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:25.397611] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:25.397648] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:25.397658] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:44:25.405574] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:25.406643] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:25.406695] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:44:25.503628] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:25.503701] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:44:25.530068] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:25.530098] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:25.550868] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:44:25.570838] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:25.570869] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:25.570877] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:25.570884] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:44:25.570930] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:44:25.609891] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:25.609919] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:25.661922] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:26.505481] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:26.525599] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:26.525653] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:44:26.544842] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:26.544864] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:26.579116] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:26.610317] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:27.463526] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:28.744194] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:44:28.754659] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fb316a8e47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fb316d61182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fb31785b265]))) 0-: received signum (15), shutting down [2016-03-15 14:44:28.766876] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:44:28.770874] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:44:28.771464] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:28.771477] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:28.771823] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:44:28.771835] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:44:28.771841] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:44:28.771877] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:44:28.771928] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:28.771936] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:30.416959] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:44:30.420680] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:30.432950] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:30.440951] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:30.440993] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:30.441041] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:30.441049] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:30.441572] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:30.441609] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:30.441617] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:30.441846] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:30.441890] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:30.441907] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:44:30.446333] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:30.447594] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:30.447651] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:44:30.523915] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:30.523996] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:44:30.561565] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:30.561601] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:30.637443] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:30.637481] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:30.637489] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:30.637496] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:44:30.711873] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:44:30.769423] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:31.231085] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:44:31.275806] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:31.275838] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:31.275846] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:31.275853] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:44:31.284833] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:31.303006] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:31.303053] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:44:31.343895] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:31.343924] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:31.408852] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:31.408885] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:31.662677] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:32.502864] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:33.830178] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:44:33.841551] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f60c7e1147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f60c80e4182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f60c8bde265]))) 0-: received signum (15), shutting down [2016-03-15 14:44:33.852851] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:44:33.856854] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:44:33.857366] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:33.857377] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:33.857715] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:44:33.857727] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:44:33.857732] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:44:33.857766] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:44:33.857820] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:33.857828] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:35.472932] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:44:35.475968] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:35.478728] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:35.482606] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:35.482646] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:35.482692] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:35.482700] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:35.483313] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:35.483356] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:35.483364] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:35.483621] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:35.483654] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:35.483660] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:44:35.488056] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:35.488716] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:35.488768] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:44:35.500456] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:35.547010] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:35.547076] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:44:35.623614] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:35.623654] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:35.736357] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:35.736390] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:35.736398] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:35.736405] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:44:35.736451] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:44:35.766869] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:44:35.815548] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:35.815582] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:35.815590] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:35.815597] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:44:35.816037] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:35.830186] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:35.830230] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:44:35.849586] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:35.849612] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:35.872154] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:35.872182] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:37.543888] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:38.915603] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:44:38.926016] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fe7f14e647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fe7f17b9182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fe7f22b3265]))) 0-: received signum (15), shutting down [2016-03-15 14:44:38.937325] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:44:38.941419] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:44:38.941966] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:38.941978] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:38.942320] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:44:38.942337] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:44:38.942343] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:44:38.942378] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:44:38.942427] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:38.942435] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:40.565922] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:44:40.568709] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:40.571788] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:40.576697] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:40.576737] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:40.576783] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:40.576790] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:40.577277] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:40.577314] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:40.577322] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:40.577541] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:40.577575] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:40.577583] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:44:40.581620] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:40.582667] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:40.582719] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:44:40.660549] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:40.660611] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:44:40.678155] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:40.678173] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:40.731251] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:44:40.748247] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:40.748275] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:40.748288] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:40.748296] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:44:40.748364] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:40.749075] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:40.878927] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:44:41.508315] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:41.522904] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:41.522952] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:44:41.558971] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:41.559005] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:41.575940] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:41.575968] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:42.638790] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:44.001864] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:44:44.012147] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f74943ac47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f749467f182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f7495179265]))) 0-: received signum (15), shutting down [2016-03-15 14:44:44.023889] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:44:44.028432] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:44:44.029006] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:44.029019] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:44.029362] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:44:44.029373] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:44:44.029379] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:44:44.029415] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:44:44.029465] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:44.029473] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:45.680809] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:44:45.685246] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:45.688118] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:45.691347] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:45.691388] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:45.691436] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:45.691443] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:45.692045] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:45.692090] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:45.692100] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:45.692366] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:45.692398] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:45.692406] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:44:45.696034] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:45.697169] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:45.697222] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:44:45.849608] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:44:45.868221] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:45.868278] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:44:45.892470] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:45.892497] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:45.910686] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:45.910716] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:45.910724] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:45.910730] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:44:45.910785] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:44:45.931353] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:46.234498] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:46.253228] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:46.253281] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:44:46.273814] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:46.273840] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:46.308215] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:46.311988] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:46.665472] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:46.743476] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:47.753065] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:49.085691] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:44:49.098969] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fafc393547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fafc3c08182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fafc4702265]))) 0-: received signum (15), shutting down [2016-03-15 14:44:49.110168] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:44:49.114123] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:44:49.114626] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:49.114638] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:49.115505] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:44:49.115524] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:44:49.115534] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:44:49.115585] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:44:49.115662] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:49.115675] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:50.805772] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:44:50.809076] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:50.813678] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:50.819390] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:50.819431] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:50.819482] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:50.819490] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:50.820046] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:50.820089] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:50.820099] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:50.820342] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:50.820388] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:50.820396] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:44:50.830168] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:50.830837] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:50.830891] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:44:50.852713] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:50.871961] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:50.917257] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:50.917322] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:44:50.934353] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:50.934370] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:50.971193] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:50.971260] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:44:51.005618] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:51.005655] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:51.025509] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:51.025536] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:51.025543] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:51.025550] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:44:51.025590] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:44:51.043300] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:51.043329] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:51.043336] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:51.043343] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:44:51.043393] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:44:51.098871] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:51.098905] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:51.098920] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:52.886587] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:54.171979] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:44:54.183409] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f5239b0547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f5239dd8182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f523a8d2265]))) 0-: received signum (15), shutting down [2016-03-15 14:44:54.195136] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:44:54.199519] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:44:54.200221] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:54.200242] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:54.200759] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:44:54.200776] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:44:54.200786] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:44:54.200837] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:44:54.200913] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:54.200927] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:55.781858] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:44:55.785147] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:55.788823] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:55.792680] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:44:55.792719] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:55.792769] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:55.792777] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:55.793382] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:55.793425] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:55.793434] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:44:55.793696] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:44:55.793729] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:44:55.793736] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:44:55.797792] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:55.798624] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:55.798676] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:44:55.814786] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:55.976996] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:55.977062] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:44:55.999615] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:55.999650] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:56.036750] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:44:56.057094] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:56.057129] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:44:56.057137] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:44:56.057143] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:44:56.057194] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:44:56.087264] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:56.087288] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:56.511234] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:44:56.530073] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:56.530124] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:44:56.548500] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:44:56.548520] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:44:56.571838] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:44:56.624740] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:44:57.854425] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:44:59.256375] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:44:59.266811] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fabf09f347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fabf0cc6182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fabf17c0265]))) 0-: received signum (15), shutting down [2016-03-15 14:44:59.279506] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:44:59.284775] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:44:59.285480] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:59.285498] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:44:59.285992] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:44:59.286005] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:44:59.286011] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:44:59.286047] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:44:59.286099] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:44:59.286107] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:00.934817] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:45:00.938193] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:00.943996] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:00.947079] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:00.947120] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:00.947168] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:00.947176] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:00.947759] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:00.947795] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:00.947804] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:00.948032] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:00.948106] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:00.948116] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:45:00.952369] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:00.953124] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:00.953188] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:45:01.027809] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:01.027869] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:45:01.042735] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:01.042760] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:01.076520] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:45:01.096041] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:01.096074] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:01.096081] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:01.096088] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:45:01.096134] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:01.236810] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:45:01.285674] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:01.285715] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:01.285723] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:01.285730] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:45:01.333105] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:01.391524] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:01.391587] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:45:01.413722] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:01.413763] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:01.434849] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:01.434890] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:01.668247] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:03.009224] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:04.342226] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:45:04.352890] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f149c64547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f149c918182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f149d412265]))) 0-: received signum (15), shutting down [2016-03-15 14:45:04.365238] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:45:04.368361] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:45:04.369063] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:04.369081] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:04.369591] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:45:04.369612] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:45:04.369624] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:45:04.369677] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:45:04.369754] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:04.369768] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:06.195257] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:45:06.237375] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:06.282966] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:06.294900] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:06.294952] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:06.295004] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:06.295011] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:06.295527] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:06.295563] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:06.295571] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:06.295789] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:06.295834] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:06.295847] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:45:06.303149] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:06.303785] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:06.303870] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:45:06.348669] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:06.403136] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:45:06.469081] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:06.469143] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:45:06.535251] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:06.535289] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:06.584677] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:06.584725] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:06.584749] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:06.584756] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:06.584763] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:45:07.238066] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:45:07.275771] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:07.275804] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:07.275811] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:07.275818] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:45:07.276218] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:07.297973] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:07.298021] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:45:07.327155] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:07.327182] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:07.376465] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:07.376504] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:08.360380] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:09.432464] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:45:09.443334] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ff57799e47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ff577c71182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ff57876b265]))) 0-: received signum (15), shutting down [2016-03-15 14:45:09.454894] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:45:09.459153] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:45:09.459862] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:09.459881] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:09.460410] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:45:09.460427] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:45:09.460437] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:45:09.460491] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:45:09.460569] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:09.460583] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:11.361761] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:45:11.369827] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:11.373513] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:11.378189] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:11.378234] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:11.378284] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:11.378292] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:11.378920] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:11.378958] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:11.378966] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:11.379227] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:11.379259] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:11.379266] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:45:11.384676] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:11.385053] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:11.385156] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:45:11.404253] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:11.429792] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:11.429861] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:45:11.457108] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:11.457148] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:11.586715] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:45:11.707977] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:45:11.790500] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:11.790537] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:11.790545] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:11.790552] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:45:11.790619] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:11.790632] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:11.790638] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:11.790644] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:45:11.791508] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:11.827419] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:11.827478] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:45:11.853519] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:11.853547] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:11.887724] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:11.887756] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:13.442483] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:14.517531] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:45:14.529167] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa9e690647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa9e6bd9182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa9e76d3265]))) 0-: received signum (15), shutting down [2016-03-15 14:45:14.540725] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:45:14.544293] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:45:14.544809] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:14.544821] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:14.545164] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:45:14.545175] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:45:14.545188] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:45:14.545224] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:45:14.545273] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:14.545281] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:16.373027] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:45:16.377637] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:16.380657] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:16.384608] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:16.384649] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:16.384698] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:16.384706] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:16.385274] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:16.385312] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:16.385320] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:16.385553] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:16.385592] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:16.385601] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:45:16.394153] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:16.395449] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:16.395506] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:45:16.496340] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:16.496412] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:45:16.528520] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:16.528550] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:16.667712] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:45:16.717338] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:16.717376] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:16.717393] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:16.717400] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:45:16.764712] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:45:16.795802] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:16.795833] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:16.796535] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:17.515643] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:17.557553] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:17.557617] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:45:17.671475] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:17.671518] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:17.738653] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:17.738688] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:18.451018] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:19.608786] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:45:19.619349] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f78a677b47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f78a6a4e182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f78a7548265]))) 0-: received signum (15), shutting down [2016-03-15 14:45:19.631257] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:45:19.635732] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:45:19.636688] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:19.636710] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:19.637303] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:45:19.637321] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:45:19.637332] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:45:19.637426] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:45:19.637497] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:19.637509] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:21.558557] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:45:21.572223] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:21.583855] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:21.609134] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:21.609186] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:21.609238] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:21.609251] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:21.609796] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:21.609831] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:21.609839] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:21.610061] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:21.610104] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:21.610113] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:45:21.625532] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:21.626612] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:21.626666] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:45:21.691780] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:21.988356] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:21.988424] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:45:22.031254] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:22.031294] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:22.075470] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:45:22.102768] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:45:22.137142] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:22.137176] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:22.137184] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:22.137191] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:45:22.137247] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:22.137272] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:22.241498] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:22.272278] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:22.272335] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:45:22.304349] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:22.304381] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:22.347741] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:22.347783] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:23.681829] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:24.692226] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:45:24.702465] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ffc9d8a547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ffc9db78182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ffc9e672265]))) 0-: received signum (15), shutting down [2016-03-15 14:45:24.714216] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:45:24.716362] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:45:24.716863] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:24.716875] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:24.717212] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:45:24.717223] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:45:24.717229] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:45:24.717264] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:45:24.717314] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:24.717322] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:26.602857] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:45:26.607760] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:26.622438] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:26.644226] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:26.644272] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:26.644323] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:26.644331] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:26.644880] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:26.644915] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:26.644923] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:26.645156] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:26.645193] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:26.645202] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:45:26.665094] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:26.665554] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:26.665615] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:45:26.768059] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:26.768139] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:45:26.980480] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:26.980520] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:27.012625] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:27.065248] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:27.192041] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:45:27.222639] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:27.222672] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:27.222680] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:27.222687] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:45:27.222767] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:45:27.259592] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:27.259646] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:45:27.290421] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:27.290448] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:27.326505] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:27.326532] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:27.326546] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:27.326553] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:45:27.326596] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:27.326617] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:28.722462] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:29.778716] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:45:29.789105] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ff31afa247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ff31b275182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ff31bd6f265]))) 0-: received signum (15), shutting down [2016-03-15 14:45:29.801048] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:45:29.804106] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:45:29.804665] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:29.804682] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:29.805097] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:45:29.805109] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:45:29.805115] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:45:29.805151] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:45:29.805203] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:29.805210] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:31.429183] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:45:31.433575] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:31.436764] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:31.439784] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:31.439823] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:31.439872] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:31.439879] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:31.440477] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:31.440512] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:31.440520] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:31.440780] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:31.440812] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:31.440818] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:45:31.445132] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:31.446230] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:31.446283] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:45:31.510992] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:31.511053] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:45:31.558627] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:31.558663] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:31.657585] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:31.657619] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:31.657627] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:31.657635] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:45:31.678407] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:45:31.697371] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:31.698047] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:32.254356] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:45:32.518592] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:32.558120] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:32.558182] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:45:32.598638] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:32.598668] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:32.616564] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:32.616592] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:33.502115] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:34.865662] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:45:34.877582] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fede846147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fede8734182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fede922e265]))) 0-: received signum (15), shutting down [2016-03-15 14:45:34.888891] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:45:34.892684] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:45:34.893296] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:34.893312] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:34.894713] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:45:34.894745] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:45:34.894759] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:45:34.894810] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:45:34.894887] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:34.894901] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:36.580569] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:45:36.583909] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:36.587947] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:36.591662] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:36.591704] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:36.591754] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:36.591761] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:36.592292] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:36.592327] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:36.592334] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:36.592554] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:36.592592] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:36.592604] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:45:36.603333] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:36.604699] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:36.604752] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:45:36.661752] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:36.661824] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:45:36.688096] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:36.688129] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:36.750192] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:45:36.770371] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:45:36.819890] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:36.819936] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:36.819943] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:36.819950] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:45:36.820001] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:37.244606] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:37.257739] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:37.257784] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:45:37.278447] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:37.278472] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:37.303206] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:37.316870] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:37.674506] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:38.660518] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:39.949545] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:45:39.960043] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f61c03a447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f61c0677182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f61c1171265]))) 0-: received signum (15), shutting down [2016-03-15 14:45:39.972050] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:45:39.976219] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:45:39.976741] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:39.976753] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:39.977094] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:45:39.977106] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:45:39.977118] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:45:39.977153] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:45:39.977204] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:39.977212] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:41.693333] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:45:41.696696] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:41.699730] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:41.703489] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:41.703530] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:41.703578] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:41.703586] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:41.704172] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:41.704215] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:41.704225] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:41.704485] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:41.704521] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:41.704528] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:45:41.708757] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:41.709585] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:41.709636] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:45:41.727965] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:41.742908] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:41.801183] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:45:41.825882] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:41.825937] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:45:41.844040] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:41.844075] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:41.887019] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:41.887088] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:45:41.906508] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:41.906541] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:41.928114] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:41.928152] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:41.928173] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:41.928180] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:41.928187] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:45:41.928239] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:45:41.951749] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:41.951776] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:41.951782] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:41.951789] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:45:41.973599] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:41.973630] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:43.766354] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:45.036810] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:45:45.047587] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fcaf7e4847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fcaf811b182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fcaf8c15265]))) 0-: received signum (15), shutting down [2016-03-15 14:45:45.060122] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:45:45.064065] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:45:45.064664] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:45.064678] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:45.065100] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:45:45.065113] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:45:45.065120] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:45:45.065156] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:45:45.065206] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:45.065220] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:46.796382] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:45:46.806216] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:46.811729] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:46.816618] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:46.816661] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:46.816711] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:46.816719] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:46.817279] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:46.817314] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:46.817322] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:46.817553] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:46.817592] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:46.817605] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:45:46.839912] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:46.840739] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:46.840798] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:45:46.870845] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:46.974506] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:46.974572] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:45:47.008883] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:47.008916] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:47.050174] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:45:47.073562] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:47.073592] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:47.073600] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:47.073607] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:45:47.130429] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:47.520925] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:45:47.620584] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:47.620631] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:47.620639] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:47.620646] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:45:47.693951] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:47.724079] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:47.724136] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:45:47.762881] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:47.762926] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:47.799897] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:47.876127] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:48.897727] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:50.123730] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:45:50.137196] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fcdcc22a47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fcdcc4fd182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fcdccff7265]))) 0-: received signum (15), shutting down [2016-03-15 14:45:50.149568] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:45:50.153099] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:45:50.153615] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:50.153626] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:50.153970] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:45:50.153981] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:45:50.153987] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:45:50.154022] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:45:50.154071] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:50.154079] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:51.763601] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:45:51.792618] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:51.799237] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:51.808294] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:51.808345] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:51.808396] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:51.808403] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:51.809003] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:51.809059] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:51.809094] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:51.809412] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:51.809446] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:51.809453] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:45:51.819415] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:51.820397] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:51.820450] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:45:51.947420] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:51.947486] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:45:51.965748] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:51.965774] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:52.020116] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:45:52.072259] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:52.072300] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:52.072307] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:52.072315] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:45:52.072362] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:45:52.155798] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:52.155829] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:52.247369] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:52.262659] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:52.262712] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:45:52.305988] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:52.306023] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:52.328303] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:52.352981] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:52.677095] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:53.876239] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:55.217051] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:45:55.229316] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fcdf67ef47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fcdf6ac2182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fcdf75bc265]))) 0-: received signum (15), shutting down [2016-03-15 14:45:55.241130] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:45:55.244739] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:45:55.245247] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:55.245259] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:55.245602] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:45:55.245614] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:45:55.245620] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:45:55.245658] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:45:55.245709] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:45:55.245717] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:45:56.856252] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:45:56.860098] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:56.863135] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:56.866577] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:45:56.866616] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:56.866665] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:56.866673] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:56.867202] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:56.867241] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:56.867258] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:45:56.867480] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:45:56.867515] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:45:56.867523] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:45:56.871877] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:45:56.872660] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:56.872713] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:45:56.901305] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:56.916520] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:45:56.974895] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:56.974956] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:45:56.993785] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:56.993813] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:57.033075] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:45:57.069175] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:57.069247] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:45:57.089130] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:45:57.089164] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:45:57.109982] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:45:57.178440] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:57.178476] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:57.178484] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:57.178491] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:45:57.178547] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:45:57.178580] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:57.178592] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:45:57.178598] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:45:57.178604] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:45:57.178635] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:57.182979] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:45:58.928482] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:00.312153] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:46:00.322775] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa6468cc47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa646b9f182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa647699265]))) 0-: received signum (15), shutting down [2016-03-15 14:46:00.334397] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:46:00.338516] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:46:00.339826] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:00.339847] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:00.340325] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:46:00.340337] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:46:00.340344] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:46:00.340379] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:46:00.340431] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:00.340439] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:02.102995] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:46:02.106457] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:02.109743] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:02.112554] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:02.112595] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:02.112646] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:02.112654] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:02.113244] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:02.113287] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:02.113297] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:02.113535] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:02.113601] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:02.113610] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:46:02.116846] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:02.117739] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:02.117817] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:46:02.132515] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:02.483705] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:02.483775] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:46:02.542608] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:02.542643] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:02.613261] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:46:02.652212] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:02.652284] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:46:02.676727] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:02.676759] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:02.676767] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:02.676774] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:46:02.677227] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:02.755773] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:02.755835] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:46:02.775087] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:02.775113] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:02.802234] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:02.802261] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:02.802275] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:02.802282] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:46:02.802325] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:02.802344] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:02.802363] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:04.174245] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:05.396999] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:46:05.407342] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f1047aaa47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f1047d7d182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f1048877265]))) 0-: received signum (15), shutting down [2016-03-15 14:46:05.419624] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:46:05.423701] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:46:05.424319] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:05.424333] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:05.424680] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:46:05.424692] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:46:05.424697] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:46:05.424733] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:46:05.424785] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:05.424793] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:07.194040] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:46:07.200212] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:07.214178] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:07.217681] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:07.217721] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:07.217771] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:07.217778] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:07.218391] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:07.218439] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:07.218450] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:07.218666] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:07.218711] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:07.218722] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:46:07.223810] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:07.224189] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:07.224240] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:46:07.244095] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:07.244141] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:46:07.294351] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:07.294386] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:07.359815] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:46:07.411323] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:07.411360] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:07.411368] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:07.411375] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:46:07.411424] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:46:07.429626] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:07.429654] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:07.429662] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:07.429668] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:46:07.430055] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:07.446930] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:07.446976] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:46:07.474560] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:07.474595] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:07.499843] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:07.502087] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:07.679915] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:09.280483] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:10.482141] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:46:10.492559] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f401f04047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f401f313182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f401fe0d265]))) 0-: received signum (15), shutting down [2016-03-15 14:46:10.504518] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:46:10.507456] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:46:10.508155] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:10.508173] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:10.508676] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:46:10.508693] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:46:10.508704] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:46:10.508772] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:46:10.508849] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:10.508864] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:12.312739] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:46:12.319920] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:12.326328] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:12.333557] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:12.333600] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:12.333649] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:12.333656] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:12.334190] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:12.334226] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:12.334233] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:12.334451] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:12.334496] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:12.334507] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:46:12.343154] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:12.344233] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:12.344288] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:46:12.382094] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:12.496128] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:46:12.529860] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:12.529927] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:46:12.561208] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:12.561246] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:12.625732] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:12.625777] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:12.625800] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:12.625807] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:12.625814] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:46:12.625858] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:46:12.662663] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:13.251478] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:13.273458] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:13.273512] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:46:13.324346] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:13.324378] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:13.356004] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:13.366095] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:14.399704] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:15.572500] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:46:15.583811] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f84c570947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f84c59dc182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f84c64d6265]))) 0-: received signum (15), shutting down [2016-03-15 14:46:15.595594] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:46:15.599675] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:46:15.600251] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:15.600264] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:15.600612] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:46:15.600624] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:46:15.600630] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:46:15.600665] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:46:15.600717] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:15.600725] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:17.375254] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:46:17.378866] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:17.382450] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:17.385225] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:17.385264] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:17.385311] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:17.385319] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:17.385892] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:17.385930] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:17.385938] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:17.386200] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:17.386233] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:17.386241] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:46:17.390285] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:17.391238] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:17.391352] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:46:17.406662] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:17.406740] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:46:17.425905] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:17.425948] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:17.463157] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:17.546157] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:46:17.609212] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:17.609251] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:17.609259] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:17.609266] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:46:17.609958] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:17.624820] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:17.624862] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:46:17.643688] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:17.643715] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:17.758686] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:17.758728] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:46:17.777193] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:17.777222] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:17.777229] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:17.777236] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:46:19.446615] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:20.656933] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:46:20.667154] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f2040a1547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f2040ce8182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f20417e2265]))) 0-: received signum (15), shutting down [2016-03-15 14:46:20.678504] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:46:20.682489] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:46:20.683753] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:20.683773] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:20.684316] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:46:20.684334] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:46:20.684341] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:46:20.684377] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:46:20.684430] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:20.684439] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:22.308915] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:46:22.331946] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:22.335552] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:22.341106] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:22.341151] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:22.341203] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:22.341211] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:22.341794] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:22.341832] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:22.341839] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:22.342076] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:22.342112] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:22.342120] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:46:22.351532] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:22.352633] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:22.352688] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:46:22.506757] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:22.506826] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:46:22.578319] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:22.578354] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:22.600145] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:46:22.633893] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:22.633929] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:22.633944] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:22.633952] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:46:22.633998] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:46:22.658257] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:22.658281] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:22.682625] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:23.528365] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:23.611913] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:23.611977] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:46:23.669912] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:23.669949] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:23.688700] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:23.688744] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:24.407557] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:25.739736] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:46:25.749905] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f395651547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f39567e8182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f39572e2265]))) 0-: received signum (15), shutting down [2016-03-15 14:46:25.761729] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:46:25.763918] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:46:25.764423] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:25.764434] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:25.764768] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:46:25.764779] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:46:25.764785] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:46:25.764821] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:46:25.764872] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:25.764881] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:27.416825] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:46:27.422261] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:27.426551] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:27.430052] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:27.430108] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:27.430181] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:27.430193] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:27.430857] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:27.430921] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:27.430932] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:27.431180] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:27.431217] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:27.431225] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:46:27.435768] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:27.436844] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:27.436899] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:46:27.515057] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:27.515123] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:46:27.535270] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:27.535300] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:27.607618] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:46:27.627383] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:27.627418] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:27.627425] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:27.627433] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:46:27.627492] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:28.000022] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:46:28.254372] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:28.271536] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:28.271587] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:46:28.293924] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:28.293952] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:28.317563] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:28.336651] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:28.683760] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:29.492595] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:30.826840] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:46:30.837503] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f7e6892d47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f7e68c00182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f7e696fa265]))) 0-: received signum (15), shutting down [2016-03-15 14:46:30.848746] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:46:30.852687] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:46:30.853232] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:30.853245] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:30.854090] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:46:30.854108] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:46:30.854118] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:46:30.854171] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:46:30.854247] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:30.854261] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:32.532487] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:46:32.536746] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:32.542081] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:32.550155] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:32.550196] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:32.550245] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:32.550252] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:32.550773] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:32.550811] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:32.550819] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:32.551057] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:32.551102] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:32.551125] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:46:32.557701] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:32.558899] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:32.558956] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:46:32.594393] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:32.608104] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:32.650884] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:32.650954] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:46:32.717298] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:32.717325] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:32.783447] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:32.783504] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:46:32.847576] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:32.847611] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:32.869065] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:32.869097] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:32.869105] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:32.869111] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:46:32.869155] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:46:32.897488] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:46:32.916999] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:32.917026] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:32.917033] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:32.917046] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:46:32.917107] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:32.917121] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:34.614662] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:35.911479] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:46:35.922727] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fbcdb13047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fbcdb403182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fbcdbefd265]))) 0-: received signum (15), shutting down [2016-03-15 14:46:35.933989] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:46:35.937020] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:46:35.937524] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:35.937536] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:35.937876] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:46:35.937887] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:46:35.937893] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:46:35.937929] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:46:35.937979] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:35.937987] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:37.703381] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:46:37.708028] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:37.711105] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:37.713936] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:37.713977] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:37.714026] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:37.714033] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:37.714611] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:37.714645] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:37.714653] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:37.714934] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:37.714968] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:37.714975] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:46:37.718885] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:37.719634] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:37.719700] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:46:37.747138] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:37.984407] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:37.984495] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:46:38.044532] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:38.044570] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:38.090102] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:46:38.125967] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:46:38.187304] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:38.187345] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:38.187353] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:38.187359] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:46:38.187422] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:38.187444] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:38.531538] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:38.549693] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:38.549754] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:46:38.580628] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:38.580661] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:38.608646] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:38.610882] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:39.775511] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:40.999510] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:46:41.009694] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fabd8ac447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fabd8d97182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fabd9891265]))) 0-: received signum (15), shutting down [2016-03-15 14:46:41.020876] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:46:41.024873] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:46:41.025379] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:41.025391] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:41.025726] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:46:41.025737] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:46:41.025743] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:46:41.025779] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:46:41.025844] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:41.025854] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:42.755768] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:46:42.777329] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:42.785849] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:42.799696] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:42.799741] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:42.799793] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:42.799800] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:42.800383] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:42.800425] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:42.800434] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:42.800688] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:42.800720] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:42.800727] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:46:42.834863] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:42.835548] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:42.835604] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:46:42.937472] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:46:42.988813] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:42.988875] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:46:43.049491] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:43.049517] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:43.127554] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:43.127602] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:43.127626] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:43.127634] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:43.127640] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:46:43.256748] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:46:43.291110] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:43.291142] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:43.291149] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:43.291156] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:46:43.320363] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:43.342751] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:43.342818] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:46:43.376449] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:43.376469] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:43.416716] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:43.416749] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:44.157312] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:44.890446] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:46.083961] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:46:46.094394] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f2e128b647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f2e12b89182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f2e13683265]))) 0-: received signum (15), shutting down [2016-03-15 14:46:46.106431] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:46:46.110315] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:46:46.111123] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:46.111138] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:46.111589] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:46:46.111601] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:46:46.111607] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:46:46.111646] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:46:46.111697] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:46.111705] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:47.883220] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:46:47.889499] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:47.893232] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:47.897786] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:47.897830] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:47.897884] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:47.897893] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:47.898499] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:47.898543] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:47.898553] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:47.898796] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:47.898835] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:47.898844] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:46:47.904478] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:47.905141] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:47.905192] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:46:47.925288] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:47.925348] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:46:48.020152] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:48.020208] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:48.346199] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:48.638711] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:48.713316] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:46:48.793234] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:48.793272] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:48.793285] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:48.793297] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:46:48.793362] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:46:48.817189] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:48.817216] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:48.817223] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:48.817230] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:46:48.817281] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:48.817321] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:46:48.838210] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:48.838235] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:48.866751] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:48.866787] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:49.961090] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:51.168638] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:46:51.181205] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd18ae0247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd18b0d5182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd18bbcf265]))) 0-: received signum (15), shutting down [2016-03-15 14:46:51.192558] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:46:51.196608] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:46:51.197126] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:51.197138] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:51.197466] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:46:51.197483] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:46:51.197489] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:46:51.197523] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:46:51.197572] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:51.197580] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:52.907024] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:46:52.910255] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:52.914827] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:52.918687] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:52.918728] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:52.918812] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:52.918821] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:52.919383] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:52.919421] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:52.919429] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:52.919654] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:52.919692] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:52.919701] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:46:52.926075] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:52.927272] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:52.927327] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:46:52.956303] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:53.123557] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:46:53.216689] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:53.216749] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:46:53.293165] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:53.293201] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:53.347379] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:46:53.433329] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:53.433377] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:53.433401] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:53.433408] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:53.433415] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:46:53.433458] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:53.534371] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:53.558713] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:53.558797] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:46:53.593574] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:53.593603] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:53.642417] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:53.867278] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:54.982602] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:56.253340] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:46:56.264724] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fca270c047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fca27393182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fca27e8d265]))) 0-: received signum (15), shutting down [2016-03-15 14:46:56.276296] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:46:56.280256] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:46:56.280862] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:56.280875] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:56.281224] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:46:56.281235] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:46:56.281241] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:46:56.281276] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:46:56.281326] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:46:56.281334] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:46:57.975664] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:46:58.018808] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:58.075916] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:58.123316] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:46:58.123373] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:58.123425] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:58.123433] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:58.124022] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:58.124066] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:58.124075] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:46:58.124342] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:46:58.124376] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:46:58.124383] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:46:58.177283] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:46:58.177532] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:58.177590] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:46:58.221572] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:58.221630] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:46:58.289901] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:58.289930] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:58.438455] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:46:58.464951] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:46:58.485364] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:58.485399] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:58.485407] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:58.485414] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:46:58.485458] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:58.485484] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:46:58.485491] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:46:58.485497] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:46:58.485958] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:46:58.516117] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:58.516164] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:46:58.566859] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:46:58.566888] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:46:58.648854] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:46:58.648896] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:46:58.688952] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:00.235275] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:01.337931] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:47:01.348518] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f90cef9c47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f90cf26f182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f90cfd69265]))) 0-: received signum (15), shutting down [2016-03-15 14:47:01.360438] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:47:01.364543] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:47:01.365055] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:01.365067] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:01.365403] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:47:01.365414] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:47:01.365421] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:47:01.365455] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:47:01.365505] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:01.365513] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:03.218404] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:47:03.222132] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:03.225340] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:03.230936] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:03.230982] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:03.231034] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:03.231051] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:03.231645] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:03.231683] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:03.231691] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:03.231915] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:03.231959] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:03.231972] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:47:03.235718] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:03.236637] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:03.236691] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:47:03.263369] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:03.375036] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:47:03.431594] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:03.431675] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:47:03.496735] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:03.496771] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:03.731729] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:03.731786] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:03.731811] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:03.731818] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:03.731825] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:47:04.260895] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:47:04.311549] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:04.311595] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:04.311602] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:04.311621] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:47:04.348576] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:04.364913] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:04.364966] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:47:04.390008] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:04.390038] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:04.426833] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:04.426876] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:05.292436] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:06.422058] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:47:06.432733] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f7403c8c47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f7403f5f182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f7404a59265]))) 0-: received signum (15), shutting down [2016-03-15 14:47:06.444295] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:47:06.448388] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:47:06.448941] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:06.448953] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:06.449289] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:47:06.449301] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:47:06.449307] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:47:06.449341] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:47:06.449391] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:06.449399] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:08.319911] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:47:08.344961] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:08.368217] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:08.376821] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:08.376867] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:08.376917] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:08.376924] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:08.377508] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:08.377546] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:08.377554] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:08.377822] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:08.377855] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:08.377862] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:47:08.388380] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:08.389295] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:08.389352] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:47:08.461338] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:08.644420] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:47:08.676582] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:08.676641] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:47:08.707679] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:08.707713] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:08.795643] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:08.795685] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:08.795709] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:08.795717] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:08.795724] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:47:08.795780] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:47:08.825047] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:08.825080] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:08.825088] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:08.825096] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:47:08.826011] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:08.841344] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:08.841394] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:47:08.860534] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:08.860569] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:08.894266] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:08.894297] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:10.446353] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:11.506060] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:47:11.516564] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa2a6ba547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa2a6e78182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa2a7972265]))) 0-: received signum (15), shutting down [2016-03-15 14:47:11.528403] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:47:11.532200] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:47:11.532711] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:11.532724] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:11.533636] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:47:11.533653] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:47:11.533664] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:47:11.533719] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:47:11.533856] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:11.533872] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:13.563294] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:47:13.584346] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:13.597106] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:13.633006] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:13.633059] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:13.633110] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:13.633117] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:13.633692] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:13.633728] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:13.633736] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:13.633953] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:13.633998] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:13.634011] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:47:13.647312] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:13.647923] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:13.647977] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:47:13.704005] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:13.943861] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:13.943926] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:47:13.988223] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:13.988260] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:14.044288] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:47:14.080343] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:14.080410] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:47:14.109200] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:14.109236] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:14.132572] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:47:14.156872] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:14.156903] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:14.156911] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:14.156917] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:47:14.156964] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:14.156990] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:14.157003] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:14.157009] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:14.157015] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:47:14.157058] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:14.157078] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:14.157124] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:14.157390] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:15.703693] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:16.593929] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:47:16.605352] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f635855647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f6358829182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f6359323265]))) 0-: received signum (15), shutting down [2016-03-15 14:47:16.616536] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:47:16.620536] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:47:16.621077] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:16.621090] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:16.621434] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:47:16.621445] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:47:16.621451] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:47:16.621485] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:47:16.621536] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:16.621544] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:18.453546] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:47:18.480646] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:18.495895] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:18.512845] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:18.512896] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:18.512946] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:18.512954] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:18.513537] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:18.513581] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:18.513591] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:18.513849] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:18.513882] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:18.513889] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:47:18.526563] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:18.527309] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:18.527366] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:47:18.699334] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:18.776084] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:47:18.806038] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:47:18.831804] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:18.831864] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:47:18.862044] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:18.862073] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:18.889094] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:18.889135] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:18.889157] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:18.889164] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:18.889170] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:47:18.925523] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:19.264382] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:19.300342] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:19.300396] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:47:19.339561] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:19.339589] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:19.369652] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:19.370893] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:20.582900] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:21.676986] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:47:21.687163] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fedba5c347d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fedba896182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fedbb390265]))) 0-: received signum (15), shutting down [2016-03-15 14:47:21.699598] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:47:21.704310] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:47:21.704828] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:21.704840] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:21.705181] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:47:21.705192] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:47:21.705198] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:47:21.705233] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:47:21.705285] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:21.705294] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:23.586171] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:47:23.611886] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:23.635739] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:23.648006] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:23.648052] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:23.648104] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:23.648112] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:23.648670] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:23.648706] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:23.648714] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:23.648953] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:23.648987] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:23.648994] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:47:23.663680] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:23.664445] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:23.664499] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:47:23.734721] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:23.851166] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:23.851237] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:47:23.888615] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:23.888643] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:24.032127] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:24.122719] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:24.122755] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:24.122762] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:24.122770] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:47:24.122841] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:24.122883] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:47:24.175919] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:24.175954] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:24.203639] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:47:24.224788] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:47:24.257704] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:24.257735] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:24.257742] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:24.257748] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:47:24.257795] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:24.257816] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:25.720819] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:26.762157] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:47:26.772335] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f75c3c5047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f75c3f23182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f75c4a1d265]))) 0-: received signum (15), shutting down [2016-03-15 14:47:26.784759] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:47:26.788607] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:47:26.789114] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:26.789125] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:26.789474] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:47:26.789485] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:47:26.789490] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:47:26.789526] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:47:26.789577] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:26.789585] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:28.760209] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:47:28.765325] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:28.770073] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:28.776229] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:28.776269] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:28.776320] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:28.776327] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:28.776896] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:28.776939] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:28.776949] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:28.777208] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:28.777240] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:28.777246] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:47:28.783026] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:28.784075] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:28.784131] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:47:28.800689] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:28.866075] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:28.866157] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:47:28.884991] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:28.885025] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:28.905316] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:28.905345] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:47:28.925170] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:47:28.942782] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:28.942812] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:28.942819] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:28.942826] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:47:28.942869] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:29.541198] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:29.556131] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:29.556179] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:47:29.573741] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:29.573766] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:29.591576] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:29.604914] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:30.839438] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:31.848279] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:47:31.858889] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7faae307047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7faae3343182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7faae3e3d265]))) 0-: received signum (15), shutting down [2016-03-15 14:47:31.870905] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:47:31.875209] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:47:31.875937] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:31.875954] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:31.876487] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:47:31.876504] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:47:31.876514] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:47:31.876597] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:47:31.876675] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:31.876690] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:33.635827] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:47:33.640967] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:33.645360] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:33.650452] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:33.650504] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:33.650555] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:33.650563] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:33.651229] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:33.651268] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:33.651276] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:33.651503] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:33.651545] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:33.651556] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:47:33.657186] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:33.657712] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:33.657766] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:47:33.680200] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:33.680256] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:47:33.778660] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:33.778693] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:34.130588] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:47:34.272497] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:47:34.298778] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:34.298823] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:34.298839] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:34.298846] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:47:34.299577] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:34.356405] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:34.356470] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:47:34.413592] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:34.413639] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:34.436520] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:34.436552] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:34.695452] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:35.714116] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:36.933675] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:47:36.945374] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fbecf04a47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fbecf31d182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fbecfe17265]))) 0-: received signum (15), shutting down [2016-03-15 14:47:36.956727] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:47:36.960681] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:47:36.962152] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:36.962170] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:36.962688] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:47:36.962705] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:47:36.962716] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:47:36.962785] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:47:36.962860] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:36.962874] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:38.668402] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:47:38.673807] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:38.678529] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:38.685610] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:38.685651] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:38.685699] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:38.685706] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:38.686260] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:38.686295] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:38.686302] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:38.686527] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:38.686563] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:38.686571] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:47:38.706847] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:38.707844] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:38.707898] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:47:38.738651] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:38.738718] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:47:38.789813] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:38.789842] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:38.850052] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:38.917373] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:38.964920] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:47:38.985918] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:38.985952] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:38.985959] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:38.985966] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:47:39.013708] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:39.013777] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:47:39.054855] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:39.054892] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:39.090079] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:47:39.149753] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:39.149786] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:39.149793] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:39.149800] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:47:39.149856] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:39.149878] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:40.763772] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:42.017359] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:47:42.027653] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fc74bac647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fc74bd99182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fc74c893265]))) 0-: received signum (15), shutting down [2016-03-15 14:47:42.040249] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:47:42.042444] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:47:42.043267] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:42.043285] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:42.043915] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:47:42.043931] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:47:42.043941] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:47:42.043994] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:47:42.044073] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:42.044087] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:43.861005] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:47:43.868250] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:43.877698] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:43.882279] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:43.882320] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:43.882370] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:43.882378] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:43.882993] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:43.883037] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:43.883046] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:43.883304] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:43.883338] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:43.883354] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:47:43.891768] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:43.892647] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:43.892701] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:47:43.930887] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:43.982370] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:43.982433] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:47:44.100499] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:44.100533] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:44.279517] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:44.279558] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:44.279578] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:44.279585] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:44.279592] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:47:44.279631] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:47:44.301201] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:47:44.544147] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:44.568498] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:44.568546] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:47:44.592729] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:44.592757] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:44.650336] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:44.650372] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:45.948205] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:47.108246] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:47:47.118590] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ff6258f947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ff625bcc182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ff6266c6265]))) 0-: received signum (15), shutting down [2016-03-15 14:47:47.130315] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:47:47.134274] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:47:47.135207] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:47.135227] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:47.135725] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:47:47.135742] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:47:47.135752] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:47:47.135803] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:47:47.135879] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:47.135892] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:49.162851] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:47:49.176551] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:49.187425] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:49.192305] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:49.192347] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:49.192398] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:49.192405] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:49.192951] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:49.192987] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:49.192995] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:49.193224] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:49.193265] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:49.193277] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:47:49.200496] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:49.201178] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:49.201248] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:47:49.283319] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:49.283388] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:47:49.355016] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:49.355064] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:49.434001] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:49.434036] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:49.434044] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:49.434051] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:47:49.494739] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:47:49.570546] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:47:49.672749] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:49.672789] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:49.672807] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:49.672813] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:49.672820] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:47:49.673302] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:49.716298] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:49.716360] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:47:49.789159] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:49.789203] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:49.845440] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:49.845475] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:49.845738] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:51.257959] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:52.191215] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:47:52.203203] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f828251047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f82827e3182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f82832dd265]))) 0-: received signum (15), shutting down [2016-03-15 14:47:52.214646] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:47:52.218429] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:47:52.219488] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:52.219507] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:52.220014] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:47:52.220031] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:47:52.220042] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:47:52.220096] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:47:52.220172] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:52.220186] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:54.252747] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:47:54.258222] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:54.269913] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:54.275558] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:54.275601] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:54.275652] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:54.275660] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:54.276209] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:54.276243] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:54.276251] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:54.276477] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:54.276513] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:54.276521] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:47:54.283802] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:54.284732] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:54.284785] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:47:54.307664] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:54.327053] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:54.415775] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:54.415843] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:47:54.475767] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:54.475805] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:54.564524] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:47:54.660449] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:47:54.772789] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:54.772830] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:54.772838] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:54.772846] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:47:54.772896] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:54.772936] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:47:54.845203] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:54.845232] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:54.902434] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:54.902469] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:54.902505] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:54.902531] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:54.902542] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:54.902553] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:47:54.902612] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:54.902635] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:56.341461] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:57.278054] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:47:57.289371] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f3cf22c247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f3cf2595182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f3cf308f265]))) 0-: received signum (15), shutting down [2016-03-15 14:47:57.300973] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:47:57.304919] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:47:57.305542] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:57.305555] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:57.305906] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:47:57.305919] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:47:57.305925] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:47:57.305960] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:47:57.306010] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:47:57.306017] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:47:59.007704] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:47:59.011466] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:59.023353] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:59.037439] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:47:59.037486] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:59.037538] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:59.037545] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:59.038116] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:59.038161] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:59.038170] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:47:59.038394] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:47:59.038432] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:47:59.038441] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:47:59.042088] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:47:59.042900] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:59.042954] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:47:59.087426] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:59.418508] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:59.418573] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:47:59.465263] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:59.465297] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:59.489557] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:47:59.513739] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:47:59.560193] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:59.560233] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:47:59.560241] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:47:59.560248] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:47:59.560300] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:59.582420] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:59.583161] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:47:59.602479] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:47:59.602525] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:47:59.630028] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:47:59.630062] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:47:59.656248] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:47:59.656278] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:01.100000] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:02.363827] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:48:02.374175] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f3900be147d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f3900eb4182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f39019ae265]))) 0-: received signum (15), shutting down [2016-03-15 14:48:02.385838] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:48:02.389918] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:48:02.390517] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:02.390530] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:02.390967] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:48:02.390979] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:48:02.390992] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:48:02.391031] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:48:02.391086] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:02.391094] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:04.267129] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:48:04.272523] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:04.275650] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:04.278850] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:04.278892] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:04.278941] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:04.278948] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:04.279926] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:04.279969] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:04.279978] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:04.280202] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:04.280240] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:04.280248] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:48:04.289666] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:04.290401] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:04.290455] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:48:04.322620] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:04.361391] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:04.361453] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:48:04.392525] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:04.392560] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:04.504086] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:48:04.572018] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:48:04.606054] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:04.606128] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:48:04.633881] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:04.633909] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:04.715277] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:04.715311] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:04.715339] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:04.715364] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:04.715371] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:04.715378] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:48:04.715430] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:04.715443] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:04.715449] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:04.715456] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:48:04.716166] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:06.346918] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:07.449515] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:48:07.460032] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ffbd726a47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ffbd753d182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ffbd8037265]))) 0-: received signum (15), shutting down [2016-03-15 14:48:07.471926] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:48:07.474031] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:48:07.474528] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:07.474540] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:07.474907] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:48:07.474919] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:48:07.474925] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:48:07.474960] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:48:07.475011] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:07.475019] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:09.417117] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:48:09.447460] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:09.482546] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:09.492705] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:09.492755] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:09.492806] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:09.492814] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:09.493346] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:09.493400] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:09.493410] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:09.493693] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:09.493740] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:09.493753] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:48:09.507291] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:09.508253] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:09.508309] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:48:09.557088] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:09.622527] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:09.622587] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:48:09.654722] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:09.654758] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:09.771041] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:09.771076] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:09.771084] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:09.771092] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:48:09.771136] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:48:09.852558] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:10.273994] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:48:10.338479] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:10.338514] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:10.338521] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:10.338528] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:48:10.370665] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:10.458717] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:10.458803] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:48:10.564027] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:10.564059] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:10.649795] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:10.649827] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:11.564442] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:12.542401] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:48:12.553295] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f803b97b47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f803bc4e182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f803c748265]))) 0-: received signum (15), shutting down [2016-03-15 14:48:12.564813] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:48:12.568664] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:48:12.569275] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:12.569289] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:12.569630] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:48:12.569642] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:48:12.569647] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:48:12.569682] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:48:12.569733] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:12.569740] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:14.700606] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:48:14.708723] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:14.720151] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:14.728189] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:14.728241] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:14.728291] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:14.728298] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:14.728859] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:14.728902] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:14.728912] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:14.729137] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:14.729175] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:14.729183] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:48:14.736474] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:14.737234] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:14.737287] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:48:14.765183] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:14.765241] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:48:14.797885] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:14.797920] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:14.830795] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:14.926581] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:48:14.996831] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:15.443723] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:15.443761] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:15.443769] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:15.443776] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:48:15.443822] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:48:15.476601] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:15.476636] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:15.476643] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:15.476650] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:48:15.476695] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:15.476735] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:48:15.513801] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:15.513828] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:15.563152] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:15.563185] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:16.794234] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:17.626528] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:48:17.637636] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fe92249847d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fe92276b182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fe923265265]))) 0-: received signum (15), shutting down [2016-03-15 14:48:17.648750] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:48:17.652795] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:48:17.653289] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:17.653301] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:17.653644] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:48:17.653656] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:48:17.653662] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:48:17.653696] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:48:17.653779] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:17.653788] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:19.700571] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:48:19.705434] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:19.709122] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:19.712352] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:19.712392] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:19.712441] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:19.712448] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:19.713001] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:19.713046] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:19.713055] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:19.713291] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:19.713330] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:19.713339] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:48:19.717688] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:19.718315] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:19.718386] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:48:19.751380] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:19.929919] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:19.929985] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:48:19.962304] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:19.962331] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:19.986541] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:48:20.013589] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:48:20.060610] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:20.060656] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:20.060664] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:20.060671] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:48:20.060719] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:20.060735] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:20.551143] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:20.624690] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:20.624749] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:48:20.722909] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:20.722951] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:20.810807] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:20.810840] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:21.773870] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:22.713119] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:48:22.723460] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f096b1b647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f096b489182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f096bf83265]))) 0-: received signum (15), shutting down [2016-03-15 14:48:22.738221] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:48:22.741274] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:48:22.741996] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:22.742046] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:22.742568] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:48:22.742584] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:48:22.742594] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:48:22.742650] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:48:22.742726] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:22.743225] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:24.686850] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:48:24.695533] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:24.702289] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:24.714946] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:24.714995] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:24.715047] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:24.715054] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:24.715635] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:24.715670] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:24.715678] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:24.715919] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:24.715956] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:24.715964] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:48:24.736372] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:24.737436] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:24.737495] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:48:24.774016] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:24.774074] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:48:24.819010] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:24.819040] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:24.996741] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:48:25.109825] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:48:25.161620] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:25.161660] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:25.161668] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:25.161675] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:48:25.277431] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:25.371643] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:25.371703] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:48:25.468241] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:25.468274] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:25.529228] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:25.529266] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:25.704400] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:26.792729] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:27.805339] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:48:27.815832] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f01d2afa47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f01d2dcd182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f01d38c7265]))) 0-: received signum (15), shutting down [2016-03-15 14:48:27.827951] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:48:27.830935] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:48:27.831636] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:27.831653] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:27.832158] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:48:27.832175] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:48:27.832185] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:48:27.832238] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:48:27.832314] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:27.832328] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:30.267568] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:48:30.275870] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:30.281230] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:30.289545] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:30.289588] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:30.289638] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:30.289646] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:30.290245] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:30.290283] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:30.290291] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:30.290522] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:30.290567] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:30.290581] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:48:30.294662] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:30.295338] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:30.295393] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:48:30.314846] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:30.314903] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:48:30.351109] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:30.351169] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:30.373434] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:30.406737] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:30.429404] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:48:30.451008] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:30.451038] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:30.451045] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:30.451052] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:48:30.480450] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:48:30.569988] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:30.570061] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:48:30.590758] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:30.590777] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:30.618703] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:30.618743] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:30.618760] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:30.618766] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:30.618773] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:48:30.618832] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:32.352723] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:32.896375] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:48:32.907098] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f19afba647d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f19afe79182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f19b0973265]))) 0-: received signum (15), shutting down [2016-03-15 14:48:32.918776] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:48:32.922469] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:48:32.924261] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:32.924279] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:32.924801] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:48:32.924818] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:48:32.924828] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:48:32.924881] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:48:32.924959] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:32.924974] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:34.840209] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:48:34.851232] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:34.866935] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:34.877338] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:34.877380] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:34.877433] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:34.877440] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:34.878029] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:34.878065] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:34.878073] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:34.878326] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:34.878363] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:34.878370] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:48:34.914107] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:34.914836] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:34.914894] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:48:35.019590] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:35.216349] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:48:35.340684] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:48:35.428276] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:35.428343] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:48:35.527835] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:35.527893] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:35.706948] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:35.706986] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:35.707014] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:35.707039] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:35.707047] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:35.707054] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:48:35.707876] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:35.752834] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:35.752894] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:48:35.827639] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:35.827674] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:35.897514] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:35.897552] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:36.970339] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:37.980943] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:48:37.992243] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f96be6ed47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f96be9c0182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f96bf4ba265]))) 0-: received signum (15), shutting down [2016-03-15 14:48:38.004697] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:48:38.008764] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:48:38.009396] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:38.009410] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:38.009739] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:48:38.009750] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:48:38.009756] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:48:38.009790] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:48:38.009841] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:38.009849] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:39.788621] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:48:39.792338] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:39.795738] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:39.798905] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:39.798945] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:39.798992] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:39.798999] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:39.799576] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:39.799617] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:39.799627] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:39.799888] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:39.799920] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:39.799927] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:48:39.804880] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:39.805566] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:39.805620] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:48:39.872429] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:48:39.893638] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:39.893698] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:48:39.939592] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:39.939621] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:40.097122] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:40.097171] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:40.097196] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:40.097203] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:40.097210] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:48:40.279834] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:48:40.372593] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:40.372625] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:40.372632] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:40.372639] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:48:40.397881] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:40.413869] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:40.413915] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:48:40.433445] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:40.433477] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:40.453098] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:40.501981] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:40.707427] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:41.861476] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:43.066192] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:48:43.076418] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f214ab0047d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f214add3182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f214b8cd265]))) 0-: received signum (15), shutting down [2016-03-15 14:48:43.087850] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:48:43.090892] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:48:43.092027] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:43.092045] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:43.092547] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:48:43.092564] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:48:43.092574] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:48:43.092626] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:48:43.092703] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:43.092713] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:44.988340] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:48:45.029912] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:45.034522] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:45.039314] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:45.039374] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:45.039429] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:45.039436] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:45.040052] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:45.040113] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:45.040127] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:45.040384] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:45.040416] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:45.040423] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:48:45.049945] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:45.050367] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:45.050421] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:48:45.089503] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:45.373042] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:45.373123] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:48:45.434671] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:45.434700] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:45.486374] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:45.559075] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:45.559112] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:45.559120] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:45.559127] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:48:45.559173] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:48:45.601839] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:45.601876] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:45.601883] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:45.601900] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:48:45.601963] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:48:45.626759] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:45.626828] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:48:45.692049] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:45.692077] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:45.841388] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:45.841433] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:47.107180] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:48.149982] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:48:48.160666] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa2092a447d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fa209577182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fa20a071265]))) 0-: received signum (15), shutting down [2016-03-15 14:48:48.172470] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:48:48.174676] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:48:48.175205] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:48.175217] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:48.175597] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:48:48.175608] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:48:48.175614] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:48:48.175650] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:48:48.175700] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:48.175708] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:50.054887] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:48:50.059143] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:50.062851] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:50.067125] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:50.067167] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:50.067217] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:50.067225] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:50.067796] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:50.067832] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:50.067849] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:50.068070] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:50.068137] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:50.068146] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:48:50.073703] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:50.074962] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:50.075037] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:48:50.093235] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:50.135854] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:50.135911] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:48:50.159525] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:50.159557] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:50.272976] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:48:50.300311] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:50.300345] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:50.300352] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:50.300359] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:48:50.300415] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:50.556634] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:48:50.660226] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:50.660269] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:50.660276] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:50.660283] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:48:50.660776] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:50.716197] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:50.716255] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:48:50.765241] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:50.765285] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:50.854398] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:50.854429] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:52.129963] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:53.238988] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:48:53.254610] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f2fe36aa47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f2fe397d182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f2fe4477265]))) 0-: received signum (15), shutting down [2016-03-15 14:48:53.266654] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:48:53.268843] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:48:53.269358] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:53.269370] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:53.269716] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:48:53.269727] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:48:53.269733] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:48:53.269770] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:48:53.269821] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:53.269830] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:55.139437] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:48:55.147145] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:55.151059] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:55.157055] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:48:55.157115] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:55.157188] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:55.157203] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:55.157881] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:55.157929] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:55.157942] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:48:55.158278] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:48:55.158343] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:48:55.158364] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:48:55.166237] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:55.167026] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:55.167110] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:48:55.197526] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:55.197580] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:48:55.231410] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:55.231447] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:55.324252] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:48:55.442863] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:48:55.497527] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:55.497581] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:55.497595] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:55.497603] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:48:55.497649] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:55.497666] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:48:55.497673] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:48:55.497679] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:48:55.498144] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:55.518258] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:55.518302] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:48:55.540266] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:48:55.540292] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:48:55.559925] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:48:55.570516] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:48:55.709753] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:48:57.222932] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:48:58.335270] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:48:58.346573] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ff2dc1c547d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7ff2dc498182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7ff2dcf92265]))) 0-: received signum (15), shutting down [2016-03-15 14:48:58.358139] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:48:58.360708] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:48:58.361212] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:58.361224] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:48:58.361573] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:48:58.361584] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:48:58.361589] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:48:58.361625] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:48:58.361675] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:48:58.361682] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:00.286783] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:49:00.297753] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:00.301345] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:00.306895] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:00.306939] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:00.306988] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:00.306995] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:00.307521] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:00.307560] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:00.307568] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:00.307793] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:00.307824] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:00.307831] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:49:00.321165] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:00.322797] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:00.322873] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:49:00.379086] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:00.507018] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:00.507101] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:49:00.533304] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:00.533327] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:00.666936] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:00.666980] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:49:00.666991] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:49:00.667006] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:49:00.667071] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:49:00.711509] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:00.711546] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:49:01.284391] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:01.307677] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:01.307728] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:49:01.342075] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:01.342107] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:01.363704] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:01.409856] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:02.378995] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:03.423286] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:49:03.434057] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f073e50747d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f073e7da182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f073f2d4265]))) 0-: received signum (15), shutting down [2016-03-15 14:49:03.446404] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:49:03.449686] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:49:03.450393] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:03.450410] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:03.451087] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:49:03.451111] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:49:03.451121] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:49:03.451176] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:49:03.451254] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:03.451269] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:05.524439] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:49:05.533996] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:05.544707] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:05.556124] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:05.556171] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:05.556222] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:05.556229] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:05.556792] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:05.556828] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:05.556835] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:05.557058] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:05.557102] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:05.557115] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:49:05.574569] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:05.575552] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:05.575635] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:49:05.609856] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:05.710967] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:05.790873] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:05.790953] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:49:05.813396] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:05.813426] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:05.837154] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:49:05.857235] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:05.857266] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:49:05.857273] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:49:05.857280] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:49:05.857326] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:49:05.877694] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:05.877721] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:49:05.877727] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:49:05.877733] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:49:05.877782] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:05.877805] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:05.877843] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:49:05.896300] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:05.896326] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:05.917489] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:05.917520] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:05.917535] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:07.632043] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:08.512103] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:49:08.522421] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fe65df0c47d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fe65e1df182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fe65ecd9265]))) 0-: received signum (15), shutting down [2016-03-15 14:49:08.534876] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:49:08.537964] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:49:08.538673] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:08.538691] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:08.539408] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:49:08.539463] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:49:08.539473] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:49:08.539528] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:49:08.539608] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:08.539623] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:10.270190] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:49:10.273267] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:10.280497] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:10.285338] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:10.285380] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:10.285431] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:10.285439] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:10.286086] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:10.286129] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:10.286138] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:10.286392] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:10.286425] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:10.286431] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:49:10.290290] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:10.291490] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:10.291546] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:49:10.311944] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:10.497747] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:10.497829] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:49:10.590768] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:10.590814] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:10.666347] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:49:10.717282] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:49:10.786114] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:10.786151] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:49:10.786159] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:49:10.786166] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:49:10.786208] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:10.786246] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:49:10.814810] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:10.814843] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:10.870467] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:10.870506] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:10.870534] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:10.870553] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:49:10.870560] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:49:10.870567] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:49:10.870644] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:10.870659] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:10.871359] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:12.346690] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:13.599625] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:49:13.610014] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f417d55947d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f417d82c182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7f417e326265]))) 0-: received signum (15), shutting down [2016-03-15 14:49:13.622291] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:49:13.624647] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:49:13.625159] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:13.625170] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:13.625508] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:49:13.625519] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:49:13.625525] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:49:13.625561] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:49:13.625611] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:13.625619] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:15.419570] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:49:15.427842] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:15.431341] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:15.439760] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:15.439804] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:15.439852] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:15.439860] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:15.440440] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:15.440475] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:15.440483] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:15.440733] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:15.440771] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:15.440778] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:49:15.446929] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:15.447810] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:15.447883] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:49:15.469515] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:15.562433] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:49:15.867549] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:15.867630] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.19.46 (0), ret: 0 [2016-03-15 14:49:15.939421] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:15.939458] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:16.022262] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:16.022309] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:16.022333] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:49:16.022340] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:49:16.022347] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, hostname:172.31.30.64 [2016-03-15 14:49:16.286941] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64, port: 0 [2016-03-15 14:49:16.348430] I [glusterd-handler.c:2212:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:16.348461] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: c4f17e9a-893b-48f0-a014-1a05cca09d01, hostname:172.31.26.134 [2016-03-15 14:49:16.348469] I [glusterd-handler.c:2266:__glusterd_handle_friend_update] 0-: Received my uuid as Friend [2016-03-15 14:49:16.348476] I [glusterd-handler.c:2257:__glusterd_handle_friend_update] 0-: Received uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, hostname:172.31.19.46 [2016-03-15 14:49:16.360717] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:16.374892] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:16.374936] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:49:16.398636] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:16.398654] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:16.438736] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f [2016-03-15 14:49:16.460078] I [glusterd-rpc-ops.c:553:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:17.502600] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:18.688701] I [glusterd-handler.c:1114:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2016-03-15 14:49:18.699960] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd951d9247d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fd952065182] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x7fd952b5f265]))) 0-: received signum (15), shutting down [2016-03-15 14:49:18.712413] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.8 (/usr/sbin/glusterd -p /var/run/glusterd.pid) [2016-03-15 14:49:18.716503] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory [2016-03-15 14:49:18.717137] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:18.717151] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:18.717496] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) [2016-03-15 14:49:18.717513] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device [2016-03-15 14:49:18.717519] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2016-03-15 14:49:18.717554] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-03-15 14:49:18.717604] I [socket.c:3645:socket_init] 0-socket.management: SSL support is NOT enabled [2016-03-15 14:49:18.717611] I [socket.c:3660:socket_init] 0-socket.management: using system polling thread [2016-03-15 14:49:20.404246] I [glusterd-store.c:1421:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30504 [2016-03-15 14:49:20.472281] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:20.486317] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:20.492502] I [glusterd-handler.c:2912:glusterd_friend_add] 0-management: connect returned 0 [2016-03-15 14:49:20.492548] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:20.492601] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:20.492609] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:20.493215] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:20.493250] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:20.493258] I [socket.c:3660:socket_init] 0-management: using system polling thread [2016-03-15 14:49:20.493508] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2016-03-15 14:49:20.493540] I [socket.c:3645:socket_init] 0-management: SSL support is NOT enabled [2016-03-15 14:49:20.493547] I [socket.c:3660:socket_init] 0-management: using system polling thread Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option transport.socket.listen-backlog 128 7: option transport.socket.read-fail-log off 8: option transport.socket.keepalive-interval 2 9: option transport.socket.keepalive-time 10 10: option transport-type rdma 11: option working-directory /var/lib/glusterd 12: end-volume 13: +------------------------------------------------------------------------------+ [2016-03-15 14:49:20.500725] I [glusterd-handler.c:1169:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-03-15 14:49:20.501420] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:20.501474] I [glusterd.c:138:glusterd_uuid_init] 0-management: retrieved UUID: c4f17e9a-893b-48f0-a014-1a05cca09d01 [2016-03-15 14:49:20.520007] I [glusterd-handler.c:2050:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f [2016-03-15 14:49:20.520065] I [glusterd-handler.c:3085:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 172.31.30.64 (0), ret: 0 [2016-03-15 14:49:20.541582] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46 [2016-03-15 14:49:20.541606] I [glusterd-sm.c:496:glusterd_ac_send_friend_update] 0-: Added uuid: af33463d-1b32-4ffb-a4f0-46ce16151e2f, host: 172.31.30.64 [2016-03-15 14:49:20.566260] I [glusterd-handshake.c:563:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30504 [2016-03-15 14:49:20.644826] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 3b4c1fb9-b325-4204-98fd-2eb739fa867f, host: 172.31.19.46, port: 0 [2016-03-15 14:49:20.770686] I [glusterd-rpc-ops.c:356:__glusterd_friend_add_cbk] 0-glusterd: Received AC