Log start: 160322-132843 - 10.67.29.90 - moshell 16.0c - /home/uabbiha/SMX/Integration-Step3/HU57862/RGUST16/glusterfs-22Mar.txt

EVOA_8300-10_6B> lhsh 000300 ls /var/log/glusterfs/

160322-13:28:51 10.67.29.90 16.0c CPP_MOM-CPP-LSV184-gen1_gen1_COMPLETE stopfile=/tmp/13910
$ lhsh 000300 ls /var/log/glusterfs/
bricks
cli.log
cmd_history.log
etc-glusterfs-glusterd.vol.log
glfsheal-c_glusterfs.log
glustershd.log
mnt-c.log
$ 

EVOA_8300-10_6B> lhsh 000300 ls /var/log/glusterfs/bricks/

160322-13:29:00 10.67.29.90 16.0c CPP_MOM-CPP-LSV184-gen1_gen1_COMPLETE stopfile=/tmp/13910
$ lhsh 000300 ls /var/log/glusterfs/bricks/
opt-lvmdir-c2-brick.log
$ 

EVOA_8300-10_6B> lhsh 000300 cat /var/log/glusterfs/bricks/opt-lvmdir-c2-brick.log

160322-13:29:10 10.67.29.90 16.0c CPP_MOM-CPP-LSV184-gen1_gen1_COMPLETE stopfile=/tmp/13910
$ lhsh 000300 cat /var/log/glusterfs/bricks/opt-lvmdir-c2-brick.log
[2016-03-22 12:25:16.511732] I [MSGID: 100030] [glusterfsd.c:2318:main] 0-/usr/sbin/glusterfsd: Started running /usr/sbin/glusterfsd version 3.7.6 (args: /usr/sbin/glusterfsd -s 10.32.0.48 --volfile-id c_glusterfs.10.32.0.48.opt-lvmdir-c2-brick -p /system/glusterd/vols/c_glusterfs/run/10.32.0.48-opt-lvmdir-c2-brick.pid -S /var/run/gluster/676dea2d0a76d84a15022e8ceef3a57c.socket --brick-name /opt/lvmdir/c2/brick -l /var/log/glusterfs/bricks/opt-lvmdir-c2-brick.log --xlator-option *-posix.glusterd-uuid=356dabed-4314-48be-9eef-d2711d01270d --brick-port 49729 --xlator-option c_glusterfs-server.listen-port=49729)
[2016-03-22 12:25:16.531187] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:16.544709] I [graph.c:269:gf_add_cmdline_options] 0-c_glusterfs-server: adding option 'listen-port' for volume 'c_glusterfs-server' with value '49729'
[2016-03-22 12:25:16.544763] I [graph.c:269:gf_add_cmdline_options] 0-c_glusterfs-posix: adding option 'glusterd-uuid' for volume 'c_glusterfs-posix' with value '356dabed-4314-48be-9eef-d2711d01270d'
[2016-03-22 12:25:16.545226] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2016-03-22 12:25:16.545261] I [MSGID: 115034] [server.c:403:_check_for_auth_option] 0-/opt/lvmdir/c2/brick: skip format check for non-addr auth option auth.login./opt/lvmdir/c2/brick.allow
[2016-03-22 12:25:16.545298] I [MSGID: 115034] [server.c:403:_check_for_auth_option] 0-/opt/lvmdir/c2/brick: skip format check for non-addr auth option auth.login.06afe471-c710-4493-9ab0-b74b9e66f01f.password
[2016-03-22 12:25:16.546871] I [rpcsvc.c:2215:rpcsvc_set_outstanding_rpc_limit] 0-rpc-service: Configured rpc.outstanding-rpc-limit with value 64
[2016-03-22 12:25:16.547052] W [MSGID: 101002] [options.c:957:xl_opt_validate] 0-c_glusterfs-server: option 'listen-port' is deprecated, preferred is 'transport.socket.listen-port', continuing with correction
[2016-03-22 12:25:16.550618] I [trash.c:2366:init] 0-c_glusterfs-trash: no option specified for 'eliminate', using NULL
[2016-03-22 12:25:16.552338] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-server: option 'rpc-auth.auth-glusterfs' is not recognized
[2016-03-22 12:25:16.552389] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-server: option 'rpc-auth.auth-unix' is not recognized
[2016-03-22 12:25:16.552423] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-server: option 'rpc-auth.auth-null' is not recognized
[2016-03-22 12:25:16.552478] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-quota: option 'timeout' is not recognized
[2016-03-22 12:25:16.552511] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-marker: option 'quota-version' is not recognized
[2016-03-22 12:25:16.552551] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-trash: option 'brick-path' is not recognized
[2016-03-22 12:25:16.563944] W [MSGID: 113036] [posix.c:2018:posix_rename] 0-c_glusterfs-posix: found directory at /opt/lvmdir/c2/brick/.trashcan/ while expecting ENOENT [File exists]
Final graph:
+------------------------------------------------------------------------------+
  1: volume c_glusterfs-posix
  2:     type storage/posix
  3:     option glusterd-uuid 356dabed-4314-48be-9eef-d2711d01270d
  4:     option directory /opt/lvmdir/c2/brick
  5:     option volume-id 0992c376-e66e-4bc4-a0c7-2ca28f894aa2
  6: end-volume
  7:  
  8: volume c_glusterfs-trash
  9:     type features/trash
 10:     option trash-dir .trashcan
 11:     option brick-path /opt/lvmdir/c2/brick
 12:     option trash-internal-op off
 13:     subvolumes c_glusterfs-posix
 14: end-volume
 15:  
 16: volume c_glusterfs-changelog
 17:     type features/changelog
 18:     option changelog-brick /opt/lvmdir/c2/brick
 19:     option changelog-dir /opt/lvmdir/c2/brick/.glusterfs/changelogs
 20:     option changelog-barrier-timeout 120
 21:     subvolumes c_glusterfs-trash
 22: end-volume
 23:  
 24: volume c_glusterfs-bitrot-stub
 25:     type features/bitrot-stub
 26:     option export /opt/lvmdir/c2/brick
 27:     subvolumes c_glusterfs-changelog
 28: end-volume
 29:  
 30: volume c_glusterfs-access-control
 31:     type features/access-control
 32:     subvolumes c_glusterfs-bitrot-stub
 33: end-volume
 34:  
 35: volume c_glusterfs-locks
 36:     type features/locks
 37:     subvolumes c_glusterfs-access-control
 38: end-volume
 39:  
 40: volume c_glusterfs-upcall
 41:     type features/upcall
 42:     option cache-invalidation off
 43:     subvolumes c_glusterfs-locks
 44: end-volume
 45:  
 46: volume c_glusterfs-io-threads
 47:     type performance/io-threads
 48:     subvolumes c_glusterfs-upcall
 49: end-volume
 50:  
 51: volume c_glusterfs-marker
 52:     type features/marker
 53:     option volume-uuid 0992c376-e66e-4bc4-a0c7-2ca28f894aa2
 54:     option timestamp-file /system/glusterd/vols/c_glusterfs/marker.tstamp
 55:     option quota-version 0
 56:     option xtime off
 57:     option gsync-force-xtime off
 58:     option quota off
 59:     option inode-quota off
 60:     subvolumes c_glusterfs-io-threads
 61: end-volume
 62:  
 63: volume c_glusterfs-barrier
 64:     type features/barrier
 65:     option barrier disable
 66:     option barrier-timeout 120
 67:     subvolumes c_glusterfs-marker
 68: end-volume
 69:  
 70: volume c_glusterfs-index
 71:     type features/index
 72:     option index-base /opt/lvmdir/c2/brick/.glusterfs/indices
 73:     subvolumes c_glusterfs-barrier
 74: end-volume
 75:  
 76: volume c_glusterfs-quota
 77:     type features/quota
 78:     option volume-uuid c_glusterfs
 79:     option server-quota off
 80:     option timeout 0
 81:     option deem-statfs off
 82:     subvolumes c_glusterfs-index
 83: end-volume
 84:  
 85: volume c_glusterfs-worm
 86:     type features/worm
 87:     option worm off
 88:     subvolumes c_glusterfs-quota
 89: end-volume
 90:  
 91: volume c_glusterfs-read-only
 92:     type features/read-only
 93:     option read-only off
 94:     subvolumes c_glusterfs-worm
 95: end-volume
 96:  
 97: volume /opt/lvmdir/c2/brick
 98:     type debug/io-stats
 99:     option latency-measurement off
100:     option count-fop-hits off
101:     subvolumes c_glusterfs-read-only
102: end-volume
103:  
104: volume c_glusterfs-server
105:     type protocol/server
106:     option transport.socket.listen-port 49729
107:     option rpc-auth.auth-glusterfs on
108:     option rpc-auth.auth-unix on
109:     option rpc-auth.auth-null on
110:     option rpc-auth-allow-insecure on
111:     option transport-type tcp
112:     option auth.login./opt/lvmdir/c2/brick.allow 06afe471-c710-4493-9ab0-b74b9e66f01f
113:     option auth.login.06afe471-c710-4493-9ab0-b74b9e66f01f.password 6440d9dd-6ccb-44a1-a0eb-c6898ca84178
114:     option auth.addr./opt/lvmdir/c2/brick.allow *
115:     option transport.tcp-user-timeout 4
116:     subvolumes /opt/lvmdir/c2/brick
117: end-volume
118:  
+------------------------------------------------------------------------------+
[2016-03-22 12:25:20.541714] W [glusterfsd.c:1236:cleanup_and_exit] (-->/lib64/libpthread.so.0() [0x80f02fb730] -->/usr/sbin/glusterfsd(glusterfs_sigwaiter-0x1bdfc) [0x10009b04] -->/usr/sbin/glusterfsd(cleanup_and_exit-0x1c02c) [0x100098bc] ) 0-: received signum (15), shutting down
[2016-03-22 12:25:24.298403] I [MSGID: 100030] [glusterfsd.c:2318:main] 0-/usr/sbin/glusterfsd: Started running /usr/sbin/glusterfsd version 3.7.6 (args: /usr/sbin/glusterfsd -s 10.32.0.48 --volfile-id c_glusterfs.10.32.0.48.opt-lvmdir-c2-brick -p /system/glusterd/vols/c_glusterfs/run/10.32.0.48-opt-lvmdir-c2-brick.pid -S /var/run/gluster/676dea2d0a76d84a15022e8ceef3a57c.socket --brick-name /opt/lvmdir/c2/brick -l /var/log/glusterfs/bricks/opt-lvmdir-c2-brick.log --xlator-option *-posix.glusterd-uuid=356dabed-4314-48be-9eef-d2711d01270d --brick-port 49730 --xlator-option c_glusterfs-server.listen-port=49730)
[2016-03-22 12:25:24.320613] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:24.335118] I [graph.c:269:gf_add_cmdline_options] 0-c_glusterfs-server: adding option 'listen-port' for volume 'c_glusterfs-server' with value '49730'
[2016-03-22 12:25:24.335175] I [graph.c:269:gf_add_cmdline_options] 0-c_glusterfs-posix: adding option 'glusterd-uuid' for volume 'c_glusterfs-posix' with value '356dabed-4314-48be-9eef-d2711d01270d'
[2016-03-22 12:25:24.335555] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2016-03-22 12:25:24.335614] I [MSGID: 115034] [server.c:403:_check_for_auth_option] 0-/opt/lvmdir/c2/brick: skip format check for non-addr auth option auth.login./opt/lvmdir/c2/brick.allow
[2016-03-22 12:25:24.335648] I [MSGID: 115034] [server.c:403:_check_for_auth_option] 0-/opt/lvmdir/c2/brick: skip format check for non-addr auth option auth.login.06afe471-c710-4493-9ab0-b74b9e66f01f.password
[2016-03-22 12:25:24.337406] I [rpcsvc.c:2215:rpcsvc_set_outstanding_rpc_limit] 0-rpc-service: Configured rpc.outstanding-rpc-limit with value 64
[2016-03-22 12:25:24.337578] W [MSGID: 101002] [options.c:957:xl_opt_validate] 0-c_glusterfs-server: option 'listen-port' is deprecated, preferred is 'transport.socket.listen-port', continuing with correction
[2016-03-22 12:25:24.340042] I [trash.c:2366:init] 0-c_glusterfs-trash: no option specified for 'eliminate', using NULL
[2016-03-22 12:25:24.340946] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-server: option 'rpc-auth.auth-glusterfs' is not recognized
[2016-03-22 12:25:24.340983] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-server: option 'rpc-auth.auth-unix' is not recognized
[2016-03-22 12:25:24.341015] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-server: option 'rpc-auth.auth-null' is not recognized
[2016-03-22 12:25:24.341065] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-quota: option 'timeout' is not recognized
[2016-03-22 12:25:24.341096] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-marker: option 'quota-version' is not recognized
[2016-03-22 12:25:24.341130] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-trash: option 'brick-path' is not recognized
[2016-03-22 12:25:24.352454] W [MSGID: 113036] [posix.c:2018:posix_rename] 0-c_glusterfs-posix: found directory at /opt/lvmdir/c2/brick/.trashcan/ while expecting ENOENT [File exists]
Final graph:
+------------------------------------------------------------------------------+
  1: volume c_glusterfs-posix
  2:     type storage/posix
  3:     option glusterd-uuid 356dabed-4314-48be-9eef-d2711d01270d
  4:     option directory /opt/lvmdir/c2/brick
  5:     option volume-id 0992c376-e66e-4bc4-a0c7-2ca28f894aa2
  6: end-volume
  7:  
  8: volume c_glusterfs-trash
  9:     type features/trash
 10:     option trash-dir .trashcan
 11:     option brick-path /opt/lvmdir/c2/brick
 12:     option trash-internal-op off
 13:     subvolumes c_glusterfs-posix
 14: end-volume
 15:  
 16: volume c_glusterfs-changelog
 17:     type features/changelog
 18:     option changelog-brick /opt/lvmdir/c2/brick
 19:     option changelog-dir /opt/lvmdir/c2/brick/.glusterfs/changelogs
 20:     option changelog-barrier-timeout 120
 21:     subvolumes c_glusterfs-trash
 22: end-volume
 23:  
 24: volume c_glusterfs-bitrot-stub
 25:     type features/bitrot-stub
 26:     option export /opt/lvmdir/c2/brick
 27:     subvolumes c_glusterfs-changelog
 28: end-volume
 29:  
 30: volume c_glusterfs-access-control
 31:     type features/access-control
 32:     subvolumes c_glusterfs-bitrot-stub
 33: end-volume
 34:  
 35: volume c_glusterfs-locks
 36:     type features/locks
 37:     subvolumes c_glusterfs-access-control
 38: end-volume
 39:  
 40: volume c_glusterfs-upcall
 41:     type features/upcall
 42:     option cache-invalidation off
 43:     subvolumes c_glusterfs-locks
 44: end-volume
 45:  
 46: volume c_glusterfs-io-threads
 47:     type performance/io-threads
 48:     subvolumes c_glusterfs-upcall
 49: end-volume
 50:  
 51: volume c_glusterfs-marker
 52:     type features/marker
 53:     option volume-uuid 0992c376-e66e-4bc4-a0c7-2ca28f894aa2
 54:     option timestamp-file /system/glusterd/vols/c_glusterfs/marker.tstamp
 55:     option quota-version 0
 56:     option xtime off
 57:     option gsync-force-xtime off
 58:     option quota off
 59:     option inode-quota off
 60:     subvolumes c_glusterfs-io-threads
 61: end-volume
 62:  
 63: volume c_glusterfs-barrier
 64:     type features/barrier
 65:     option barrier disable
 66:     option barrier-timeout 120
 67:     subvolumes c_glusterfs-marker
 68: end-volume
 69:  
 70: volume c_glusterfs-index
 71:     type features/index
 72:     option index-base /opt/lvmdir/c2/brick/.glusterfs/indices
 73:     subvolumes c_glusterfs-barrier
 74: end-volume
 75:  
 76: volume c_glusterfs-quota
 77:     type features/quota
 78:     option volume-uuid c_glusterfs
 79:     option server-quota off
 80:     option timeout 0
 81:     option deem-statfs off
 82:     subvolumes c_glusterfs-index
 83: end-volume
 84:  
 85: volume c_glusterfs-worm
 86:     type features/worm
 87:     option worm off
 88:     subvolumes c_glusterfs-quota
 89: end-volume
 90:  
 91: volume c_glusterfs-read-only
 92:     type features/read-only
 93:     option read-only off
 94:     subvolumes c_glusterfs-worm
 95: end-volume
 96:  
 97: volume /opt/lvmdir/c2/brick
 98:     type debug/io-stats
 99:     option latency-measurement off
100:     option count-fop-hits off
101:     subvolumes c_glusterfs-read-only
102: end-volume
103:  
104: volume c_glusterfs-server
105:     type protocol/server
106:     option transport.socket.listen-port 49730
107:     option rpc-auth.auth-glusterfs on
108:     option rpc-auth.auth-unix on
109:     option rpc-auth.auth-null on
110:     option rpc-auth-allow-insecure on
111:     option transport-type tcp
112:     option auth.login./opt/lvmdir/c2/brick.allow 06afe471-c710-4493-9ab0-b74b9e66f01f
113:     option auth.login.06afe471-c710-4493-9ab0-b74b9e66f01f.password 6440d9dd-6ccb-44a1-a0eb-c6898ca84178
114:     option auth.addr./opt/lvmdir/c2/brick.allow *
115:     option transport.tcp-user-timeout 4
116:     subvolumes /opt/lvmdir/c2/brick
117: end-volume
118:  
+------------------------------------------------------------------------------+
[2016-03-22 12:25:24.438428] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:25:24.438573] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-3016-2016/03/22-12:25:24:376927-c_glusterfs-client-418-0-0 (version: 3.7.6)
[2016-03-22 12:25:24.494248] I [glusterfsd-mgmt.c:57:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2016-03-22 12:25:24.599471] I [glusterfsd-mgmt.c:1513:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2016-03-22 12:25:24.603037] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:25:24.603139] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-3863-2016/03/22-12:20:24:176727-c_glusterfs-client-418-4-0 (version: 3.7.6)
[2016-03-22 12:25:24.772451] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:25:24.772508] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-9876-2016/03/22-12:25:24:721512-c_glusterfs-client-418-0-0 (version: 3.7.6)
[2016-03-22 12:25:24.849859] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:25:24.849909] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-3084-2016/03/22-12:25:24:794681-c_glusterfs-client-418-0-0 (version: 3.7.6)
[2016-03-22 12:25:28.249987] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-9876-2016/03/22-12:25:24:721512-c_glusterfs-client-418-0-0
[2016-03-22 12:25:28.250076] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-9876-2016/03/22-12:25:24:721512-c_glusterfs-client-418-0-0
[2016-03-22 12:25:28.284263] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:25:28.284352] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-9822-2016/03/22-12:25:24:267102-c_glusterfs-client-418-0-0 (version: 3.7.6)
[2016-03-22 12:25:28.332752] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:25:28.332804] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-10534-2016/03/22-12:25:28:282378-c_glusterfs-client-418-0-0 (version: 3.7.6)
[2016-03-22 12:25:32.241368] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-10534-2016/03/22-12:25:28:282378-c_glusterfs-client-418-0-0
[2016-03-22 12:25:32.241465] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-10534-2016/03/22-12:25:28:282378-c_glusterfs-client-418-0-0
[2016-03-22 12:25:32.456135] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:25:32.456177] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-3445-2016/03/22-12:25:32:413154-c_glusterfs-client-418-0-0 (version: 3.7.6)
[2016-03-22 12:25:34.517017] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-3445-2016/03/22-12:25:32:413154-c_glusterfs-client-418-0-0
[2016-03-22 12:25:34.517103] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-3445-2016/03/22-12:25:32:413154-c_glusterfs-client-418-0-0
[2016-03-22 12:25:38.538986] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:25:38.539060] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-5234-2016/03/22-12:25:38:493848-c_glusterfs-client-418-0-0 (version: 3.7.6)
[2016-03-22 12:25:42.239936] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-5234-2016/03/22-12:25:38:493848-c_glusterfs-client-418-0-0
[2016-03-22 12:25:42.240015] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-5234-2016/03/22-12:25:38:493848-c_glusterfs-client-418-0-0
[2016-03-22 12:25:42.518261] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:25:42.518303] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-12305-2016/03/22-12:25:42:465431-c_glusterfs-client-418-0-0 (version: 3.7.6)
[2016-03-22 12:25:44.576230] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-12305-2016/03/22-12:25:42:465431-c_glusterfs-client-418-0-0
[2016-03-22 12:25:44.576308] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-12305-2016/03/22-12:25:42:465431-c_glusterfs-client-418-0-0
[2016-03-22 12:25:58.114947] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:25:58.114984] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-5536-2016/03/22-12:25:58:72993-c_glusterfs-client-418-0-0 (version: 3.7.6)
[2016-03-22 12:26:01.241093] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-5536-2016/03/22-12:25:58:72993-c_glusterfs-client-418-0-0
[2016-03-22 12:26:01.241178] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-5536-2016/03/22-12:25:58:72993-c_glusterfs-client-418-0-0
[2016-03-22 12:26:01.508497] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:26:01.508534] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-17300-2016/03/22-12:26:01:464261-c_glusterfs-client-418-0-0 (version: 3.7.6)
[2016-03-22 12:26:03.566280] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-17300-2016/03/22-12:26:01:464261-c_glusterfs-client-418-0-0
[2016-03-22 12:26:03.566356] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-17300-2016/03/22-12:26:01:464261-c_glusterfs-client-418-0-0
[2016-03-22 12:26:04.621006] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:26:04.621044] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-5810-2016/03/22-12:26:04:577106-c_glusterfs-client-418-0-0 (version: 3.7.6)
[2016-03-22 12:26:06.676780] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-5810-2016/03/22-12:26:04:577106-c_glusterfs-client-418-0-0
[2016-03-22 12:26:06.676853] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-5810-2016/03/22-12:26:04:577106-c_glusterfs-client-418-0-0
[2016-03-22 12:26:06.942648] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:26:06.942686] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-17561-2016/03/22-12:26:06:899162-c_glusterfs-client-418-0-0 (version: 3.7.6)
[2016-03-22 12:26:09.242835] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-17561-2016/03/22-12:26:06:899162-c_glusterfs-client-418-0-0
[2016-03-22 12:26:09.242912] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-17561-2016/03/22-12:26:06:899162-c_glusterfs-client-418-0-0
$ 

EVOA_8300-10_6B> lhsh 000300 cat /var/log/glusterfs/cli.log

160322-13:29:22 10.67.29.90 16.0c CPP_MOM-CPP-LSV184-gen1_gen1_COMPLETE stopfile=/tmp/13910
$ lhsh 000300 cat /var/log/glusterfs/cli.log
[2016-03-22 12:25:16.405242] I [cli.c:721:main] 0-cli: Started running gluster with version 3.7.6
[2016-03-22 12:25:16.419695] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:16.420029] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:16.507835] I [cli-rpc-ops.c:769:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2016-03-22 12:25:16.508312] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:25:16.551072] I [cli.c:721:main] 0-cli: Started running gluster with version 3.7.6
[2016-03-22 12:25:16.569941] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:16.570118] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:16.572097] I [cli-rpc-ops.c:769:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2016-03-22 12:25:16.572868] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:25:32.393898] I [cli.c:721:main] 0-cli: Started running /usr/sbin/gluster with version 3.7.6
[2016-03-22 12:25:32.406341] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:32.407198] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:34.517273] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:25:38.475140] I [cli.c:721:main] 0-cli: Started running /usr/sbin/gluster with version 3.7.6
[2016-03-22 12:25:38.487426] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:38.487581] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:41.486996] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:42.240159] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:25:58.054902] I [cli.c:721:main] 0-cli: Started running /usr/sbin/gluster with version 3.7.6
[2016-03-22 12:25:58.067288] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:58.068002] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:26:01.066734] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:26:01.241307] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:26:04.557521] I [cli.c:721:main] 0-cli: Started running /usr/sbin/gluster with version 3.7.6
[2016-03-22 12:26:04.571285] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:26:04.571393] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:26:06.676976] I [input.c:36:cli_batch] 0-: Exiting with: 0
$ 

EVOA_8300-10_6B> lhsh 000300 cat /var/log/glusterfs/cmd_history.log

160322-13:29:23 10.67.29.90 16.0c CPP_MOM-CPP-LSV184-gen1_gen1_COMPLETE stopfile=/tmp/13910
$ lhsh 000300 cat /var/log/glusterfs/cmd_history.log
$ 

EVOA_8300-10_6B> lhsh 000300 cat /var/log/glusterfs/etc-glusterfs-glusterd.vol.log

160322-13:29:24 10.67.29.90 16.0c CPP_MOM-CPP-LSV184-gen1_gen1_COMPLETE stopfile=/tmp/13910
$ lhsh 000300 cat /var/log/glusterfs/etc-glusterfs-glusterd.vol.log
[2016-03-22 12:25:14.907917] I [MSGID: 100030] [glusterfsd.c:2318:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.7.6 (args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level DEBUG)
[2016-03-22 12:25:14.910207] D [MSGID: 0] [glusterfsd.c:1984:glusterfs_pidfile_update] 0-glusterfsd: pidfile /var/run/glusterd.pid updated with pid 1769
[2016-03-22 12:25:14.910406] D [logging.c:1790:__gf_log_inject_timer_event] 0-logging-infra: Starting timer now. Timeout = 120, current buf size = 5
[2016-03-22 12:25:14.911108] D [MSGID: 0] [glusterfsd.c:638:get_volfp] 0-glusterfsd: loading volume file /etc/glusterfs/glusterd.vol
[2016-03-22 12:25:14.916702] I [MSGID: 106478] [glusterd.c:1350:init] 0-management: Maximum allowed open file descriptors set to 65536
[2016-03-22 12:25:14.919697] I [MSGID: 106479] [glusterd.c:1399:init] 0-management: Using /system/glusterd as working directory
[2016-03-22 12:25:14.920418] D [MSGID: 0] [glusterd.c:410:glusterd_rpcsvc_options_build] 0-glusterd: listen-backlog value: 128
[2016-03-22 12:25:14.920772] D [rpcsvc.c:2317:rpcsvc_init] 0-rpc-service: RPC service inited.
[2016-03-22 12:25:14.920819] D [rpcsvc.c:1874:rpcsvc_program_register] 0-rpc-service: New program registered: GF-DUMP, Num: 123451501, Ver: 1, Port: 0
[2016-03-22 12:25:14.920918] D [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3.7.6/rpc-transport/socket.so
[2016-03-22 12:25:14.932696] D [socket.c:3845:socket_init] 0-socket.management: Configued transport.tcp-user-timeout=0
[2016-03-22 12:25:14.932758] D [socket.c:3928:socket_init] 0-socket.management: SSL support on the I/O path is NOT enabled
[2016-03-22 12:25:14.932777] D [socket.c:3931:socket_init] 0-socket.management: SSL support for glusterd is NOT enabled
[2016-03-22 12:25:14.932796] D [socket.c:3948:socket_init] 0-socket.management: using system polling thread
[2016-03-22 12:25:14.932837] D [name.c:564:server_fill_address_family] 0-socket.management: option address-family not specified, defaulting to inet
[2016-03-22 12:25:14.933124] D [rpcsvc.c:1874:rpcsvc_program_register] 0-rpc-service: New program registered: GlusterD svc peer, Num: 1238437, Ver: 2, Port: 0
[2016-03-22 12:25:14.933149] D [rpcsvc.c:1874:rpcsvc_program_register] 0-rpc-service: New program registered: GlusterD svc cli read-only, Num: 1238463, Ver: 2, Port: 0
[2016-03-22 12:25:14.933171] D [rpcsvc.c:1874:rpcsvc_program_register] 0-rpc-service: New program registered: GlusterD svc mgmt, Num: 1238433, Ver: 2, Port: 0
[2016-03-22 12:25:14.933192] D [rpcsvc.c:1874:rpcsvc_program_register] 0-rpc-service: New program registered: GlusterD svc mgmt v3, Num: 1238433, Ver: 3, Port: 0
[2016-03-22 12:25:14.933213] D [rpcsvc.c:1874:rpcsvc_program_register] 0-rpc-service: New program registered: Gluster Portmap, Num: 34123456, Ver: 1, Port: 0
[2016-03-22 12:25:14.933234] D [rpcsvc.c:1874:rpcsvc_program_register] 0-rpc-service: New program registered: Gluster Handshake, Num: 14398633, Ver: 2, Port: 0
[2016-03-22 12:25:14.933261] D [rpcsvc.c:1874:rpcsvc_program_register] 0-rpc-service: New program registered: Gluster MGMT Handshake, Num: 1239873, Ver: 1, Port: 0
[2016-03-22 12:25:14.933351] D [rpcsvc.c:2317:rpcsvc_init] 0-rpc-service: RPC service inited.
[2016-03-22 12:25:14.933373] D [rpcsvc.c:1874:rpcsvc_program_register] 0-rpc-service: New program registered: GF-DUMP, Num: 123451501, Ver: 1, Port: 0
[2016-03-22 12:25:14.933413] D [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3.7.6/rpc-transport/socket.so
[2016-03-22 12:25:14.933517] D [socket.c:3794:socket_init] 0-socket.management: disabling nodelay
[2016-03-22 12:25:14.933540] D [socket.c:3845:socket_init] 0-socket.management: Configued transport.tcp-user-timeout=0
[2016-03-22 12:25:14.933565] D [socket.c:3928:socket_init] 0-socket.management: SSL support on the I/O path is NOT enabled
[2016-03-22 12:25:14.933584] D [socket.c:3931:socket_init] 0-socket.management: SSL support for glusterd is NOT enabled
[2016-03-22 12:25:14.933603] D [socket.c:3948:socket_init] 0-socket.management: using system polling thread
[2016-03-22 12:25:14.933726] D [rpcsvc.c:1874:rpcsvc_program_register] 0-rpc-service: New program registered: GlusterD svc cli, Num: 1238463, Ver: 2, Port: 0
[2016-03-22 12:25:14.933750] D [rpcsvc.c:1874:rpcsvc_program_register] 0-rpc-service: New program registered: Gluster Handshake (CLI Getspec), Num: 14398633, Ver: 2, Port: 0
[2016-03-22 12:25:14.933848] D [MSGID: 0] [glusterd-utils.c:5974:glusterd_sm_tr_log_init] 0-glusterd: returning 0
[2016-03-22 12:25:14.933884] D [MSGID: 0] [glusterd.c:1691:init] 0-management: cannot get run-with-valgrind value
[2016-03-22 12:25:14.937896] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:25:14.937964] D [MSGID: 0] [store.c:439:gf_store_handle_retrieve] 0-: Returning 0
[2016-03-22 12:25:14.938326] D [MSGID: 0] [store.c:307:gf_store_retrieve_value] 0-: key operating-version found
[2016-03-22 12:25:14.938397] I [MSGID: 106513] [glusterd-store.c:2047:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30706
[2016-03-22 12:25:14.938760] D [MSGID: 0] [glusterd-utils.c:512:glusterd_volinfo_new] 0-management: Returning 0
[2016-03-22 12:25:14.945387] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:25:14.945437] D [MSGID: 0] [store.c:439:gf_store_handle_retrieve] 0-: Returning 0
[2016-03-22 12:25:14.945495] D [MSGID: 0] [store.c:501:gf_store_iter_new] 0-: Returning with 0
[2016-03-22 12:25:14.945834] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:14.945862] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = type value = 2
[2016-03-22 12:25:14.945902] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:14.945928] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = count value = 2
[2016-03-22 12:25:14.945994] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:14.946020] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = status value = 1
[2016-03-22 12:25:14.946061] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:14.946088] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = sub_count value = 2
[2016-03-22 12:25:14.946125] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:14.946151] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = stripe_count value = 1
[2016-03-22 12:25:14.946189] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:14.946215] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = replica_count value = 2
[2016-03-22 12:25:14.946253] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:14.946279] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = disperse_count value = 0
[2016-03-22 12:25:14.946317] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:14.946343] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = redundancy_count value = 0
[2016-03-22 12:25:14.946381] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:14.946407] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = version value = 2273
[2016-03-22 12:25:14.946444] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:14.946470] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = transport-type value = 0
[2016-03-22 12:25:14.946507] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:14.946534] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = volume-id value = 0992c376-e66e-4bc4-a0c7-2ca28f894aa2
[2016-03-22 12:25:14.946632] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:14.946661] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = username value = 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:25:14.946709] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:14.946736] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = password value = 6440d9dd-6ccb-44a1-a0eb-c6898ca84178
[2016-03-22 12:25:14.946780] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:14.946809] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = op-version value = 3
[2016-03-22 12:25:14.946858] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:14.946885] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = client-op-version value = 3
[2016-03-22 12:25:14.946923] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:14.946959] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = quota-version value = 0
[2016-03-22 12:25:14.947000] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:14.947026] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = parent_volname value = N/A
[2016-03-22 12:25:14.947070] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:14.947102] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = restored_from_snap value = 00000000-0000-0000-0000-000000000000
[2016-03-22 12:25:14.947143] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:14.947169] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = snap-max-hard-limit value = 256
[2016-03-22 12:25:14.947207] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:14.947234] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = nfs.disable value = on
[2016-03-22 12:25:14.947333] D [MSGID: 0] [glusterd-store.c:2725:glusterd_store_update_volinfo] 0-management: Parsed as Volume-set:key=nfs.disable,value:on
[2016-03-22 12:25:14.947371] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:14.947397] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = network.ping-timeout value = 4
[2016-03-22 12:25:14.947434] D [MSGID: 0] [glusterd-store.c:2725:glusterd_store_update_volinfo] 0-management: Parsed as Volume-set:key=network.ping-timeout,value:4
[2016-03-22 12:25:14.947472] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:14.947499] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = performance.readdir-ahead value = on
[2016-03-22 12:25:14.947536] D [MSGID: 0] [glusterd-store.c:2725:glusterd_store_update_volinfo] 0-management: Parsed as Volume-set:key=performance.readdir-ahead,value:on
[2016-03-22 12:25:14.947575] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:14.947601] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = brick-0 value = 10.32.0.48:-opt-lvmdir-c2-brick
[2016-03-22 12:25:14.947641] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:14.947667] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = brick-1 value = 10.32.1.144:-opt-lvmdir-c2-brick
[2016-03-22 12:25:14.947707] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with -1
[2016-03-22 12:25:14.947809] D [MSGID: 0] [store.c:501:gf_store_iter_new] 0-: Returning with 0
[2016-03-22 12:25:14.947867] D [MSGID: 0] [glusterd-utils.c:884:glusterd_brickinfo_new] 0-management: Returning 0
[2016-03-22 12:25:14.947943] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:14.948642] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0" repeated 22 times between [2016-03-22 12:25:14.947943] and [2016-03-22 12:25:14.948235]
[2016-03-22 12:25:14.948641] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:25:14.948725] D [MSGID: 0] [store.c:439:gf_store_handle_retrieve] 0-: Returning 0
[2016-03-22 12:25:14.948776] D [MSGID: 0] [store.c:501:gf_store_iter_new] 0-: Returning with 0
[2016-03-22 12:25:14.949053] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:16.269598] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0" repeated 7 times between [2016-03-22 12:25:14.949053] and [2016-03-22 12:25:16.269579]
[2016-03-22 12:25:16.269596] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with -1
[2016-03-22 12:25:16.269758] D [MSGID: 0] [glusterd-utils.c:884:glusterd_brickinfo_new] 0-management: Returning 0
[2016-03-22 12:25:16.269803] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:16.269905] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:25:16.269943] D [MSGID: 0] [store.c:439:gf_store_handle_retrieve] 0-: Returning 0
[2016-03-22 12:25:16.270001] D [MSGID: 0] [store.c:501:gf_store_iter_new] 0-: Returning with 0
[2016-03-22 12:25:16.270459] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:16.270612] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0" repeated 7 times between [2016-03-22 12:25:16.270459] and [2016-03-22 12:25:16.270596]
[2016-03-22 12:25:16.270611] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with -1
[2016-03-22 12:25:16.270732] D [MSGID: 0] [glusterd-store.c:2402:glusterd_store_retrieve_bricks] 0-management: Returning with 0
[2016-03-22 12:25:16.271728] D [MSGID: 0] [glusterd-utils.c:2052:glusterd_volume_compute_cksum] 0-management: Returning with 0
[2016-03-22 12:25:16.279423] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:25:16.279846] D [MSGID: 0] [store.c:307:gf_store_retrieve_value] 0-: key version found
[2016-03-22 12:25:16.279946] D [MSGID: 0] [store.c:460:gf_store_handle_destroy] 0-: Returning 0
[2016-03-22 12:25:16.285622] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:25:16.285859] D [MSGID: 0] [glusterd-utils.c:2052:glusterd_volume_compute_cksum] 0-management: Returning with 0
[2016-03-22 12:25:16.292741] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:25:16.293271] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:16.307829] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:25:16.293358] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:16.307826] D [MSGID: 0] [store.c:460:gf_store_handle_destroy] 0-: Returning 0
[2016-03-22 12:25:16.308012] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:25:16.308041] D [MSGID: 0] [store.c:439:gf_store_handle_retrieve] 0-: Returning 0
[2016-03-22 12:25:16.308092] D [MSGID: 0] [store.c:501:gf_store_iter_new] 0-: Returning with 0
[2016-03-22 12:25:16.308634] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:16.308726] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0" repeated 3 times between [2016-03-22 12:25:16.308634] and [2016-03-22 12:25:16.308708]
[2016-03-22 12:25:16.308725] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with -1
[2016-03-22 12:25:16.308846] D [MSGID: 0] [glusterd-store.c:3067:glusterd_store_retrieve_volumes] 0-management: Returning with 0
[2016-03-22 12:25:16.315429] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:25:16.315458] D [MSGID: 0] [store.c:439:gf_store_handle_retrieve] 0-: Returning 0
[2016-03-22 12:25:16.315502] D [MSGID: 0] [store.c:501:gf_store_iter_new] 0-: Returning with 0
[2016-03-22 12:25:16.315874] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:16.315913] D [MSGID: 0] [glusterd-utils.c:5974:glusterd_sm_tr_log_init] 0-glusterd: returning 0
[2016-03-22 12:25:16.315973] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:16.316023] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:25:16.316012] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:16.316022] D [MSGID: 0] [glusterd-peer-utils.c:458:glusterd_peer_hostname_new] 0-glusterd: Returning 0
[2016-03-22 12:25:16.316104] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with -1
[2016-03-22 12:25:16.316169] I [MSGID: 106498] [glusterd-handler.c:3579:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0
[2016-03-22 12:25:16.316301] D [MSGID: 0] [glusterd-handler.c:3391:glusterd_transport_inet_options_build] 0-glusterd: Returning 0
[2016-03-22 12:25:16.316371] I [rpc-clnt.c:984:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2016-03-22 12:25:16.316394] D [rpc-clnt.c:996:rpc_clnt_connection_init] 0-management: setting ping-timeout to 30
[2016-03-22 12:25:16.316425] D [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3.7.6/rpc-transport/socket.so
[2016-03-22 12:25:16.316551] D [socket.c:3845:socket_init] 0-management: Configued transport.tcp-user-timeout=-1
[2016-03-22 12:25:16.316579] D [socket.c:3928:socket_init] 0-management: SSL support on the I/O path is NOT enabled
[2016-03-22 12:25:16.316598] D [socket.c:3931:socket_init] 0-management: SSL support for glusterd is NOT enabled
[2016-03-22 12:25:16.316617] D [socket.c:3948:socket_init] 0-management: using system polling thread
[2016-03-22 12:25:16.318081] D [MSGID: 0] [common-utils.c:333:gf_resolve_ip6] 0-resolver: returning ip-10.32.1.144 (port-24007) for hostname: 10.32.1.144 and port: 24007
[2016-03-22 12:25:16.318131] D [socket.c:2833:socket_fix_ssl_opts] 0-management: disabling SSL for portmapper connection
[2016-03-22 12:25:16.318195] W [socket.c:869:__socket_keepalive] 0-socket: failed to set TCP_USER_TIMEOUT -1000 on socket 13, Invalid argument
[2016-03-22 12:25:16.318221] E [socket.c:2965:socket_connect] 0-management: Failed to set keep-alive: Invalid argument
[2016-03-22 12:25:16.318670] D [MSGID: 0] [glusterd-handler.c:3323:glusterd_rpc_create] 0-management: returning 0
[2016-03-22 12:25:16.318721] D [MSGID: 0] [glusterd-store.c:4118:glusterd_store_retrieve_peers] 0-management: Returning with 0
[2016-03-22 12:25:16.319289] D [MSGID: 0] [glusterd-store.c:3501:glusterd_store_retrieve_missed_snaps_list] 0-management: EOF for missed_snap_list
[2016-03-22 12:25:16.319343] D [MSGID: 0] [glusterd-store.c:3607:glusterd_store_retrieve_snaps] 0-management: Returning with 0
[2016-03-22 12:25:16.319786] D [MSGID: 0] [glusterd-peer-utils.c:133:glusterd_peerinfo_find_by_hostname] 0-management: Unable to find friend: 10.32.0.48
[2016-03-22 12:25:16.319954] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.0.48 
[2016-03-22 12:25:16.320765] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.0.48 is local address at interface efd
[2016-03-22 12:25:16.320858] D [MSGID: 0] [store.c:307:gf_store_retrieve_value] 0-: key UUID found
[2016-03-22 12:25:16.320950] D [MSGID: 0] [glusterd-store.c:2122:glusterd_retrieve_uuid] 0-management: Returning 0
[2016-03-22 12:25:16.321025] I [MSGID: 106544] [glusterd.c:159:glusterd_uuid_init] 0-management: retrieved UUID: 356dabed-4314-48be-9eef-d2711d01270d
[2016-03-22 12:25:16.321059] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:16.321092] D [MSGID: 0] [glusterd-utils.c:925:glusterd_resolve_brick] 0-management: Returning 0
[2016-03-22 12:25:16.321129] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.1.144 found.. state: 3
[2016-03-22 12:25:16.321164] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:16.321195] D [MSGID: 0] [glusterd-utils.c:925:glusterd_resolve_brick] 0-management: Returning 0
[2016-03-22 12:25:16.321240] D [MSGID: 0] [glusterd-store.c:4339:glusterd_restore] 0-management: Returning 0
[2016-03-22 12:25:16.328451] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:25:16.328505] D [MSGID: 0] [store.c:439:gf_store_handle_retrieve] 0-: Returning 0
[2016-03-22 12:25:16.328589] D [MSGID: 0] [store.c:501:gf_store_iter_new] 0-: Returning with 0
[2016-03-22 12:25:16.329134] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:25:16.329181] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with -1
[2016-03-22 12:25:16.329238] D [MSGID: 0] [store.c:460:gf_store_handle_destroy] 0-: Returning 0
Final graph:
[2016-03-22 12:25:16.329481] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
+------------------------------------------------------------------------------+
  1: volume management
  2:     type mgmt/glusterd
  3:     option rpc-auth.auth-glusterfs on
  4:     option rpc-auth.auth-unix on
  5:     option rpc-auth.auth-null on
  6:     option rpc-auth-allow-insecure on
  7:     option transport.socket.listen-backlog 128
  8:     option ping-timeout 30
  9:     option transport.socket.read-fail-log off
 10:     option transport.socket.keepalive-interval 2
 11:     option transport.socket.keepalive-time 10
 12:     option transport-type socket
 13:     option working-directory /system/glusterd
 14: end-volume
 15:  
+------------------------------------------------------------------------------+
[2016-03-22 12:25:16.329700] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.329774] D [MSGID: 0] [glusterd-handler.c:5042:__glusterd_peer_rpc_notify] 0-management: got RPC_CLNT_CONNECT
[2016-03-22 12:25:16.329992] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:16.330004] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.330046] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2016-03-22 12:25:16.330902] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(+0x1a188)[0x3fff79d23188] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_submit-0x1a4d8)[0x3fff79d1e0a8] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_submit_request-0x129078)[0x3fff766c1de8] ))))) 0-: 10.32.1.144:24007: ping timer event already removed
[2016-03-22 12:25:16.331125] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.331172] D [MSGID: 0] [glusterd-handshake.c:2098:__glusterd_peer_dump_version_cbk] 0-management: Proceeding to op-version handshake with peer 10.32.1.144
[2016-03-22 12:25:16.331240] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.331384] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.331620] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.331668] D [MSGID: 0] [glusterd-handshake.c:1660:gd_validate_peer_op_version] 0-management: Peer 10.32.1.144 accepted
[2016-03-22 12:25:16.331903] I [MSGID: 106163] [glusterd-handshake.c:1193:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30706
[2016-03-22 12:25:16.332183] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:16.354067] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:25:16.332271] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:16.354064] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.354188] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:25:16.354180] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.354187] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_CONNECTED'
[2016-03-22 12:25:16.354253] D [MSGID: 0] [glusterd-handshake.c:1613:glusterd_event_connected_inject] 0-glusterd: returning 0
[2016-03-22 12:25:16.354289] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.354339] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_CONNECTED'
[2016-03-22 12:25:16.354368] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.354808] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:25:16.354406] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.354807] D [MSGID: 0] [glusterd-utils.c:2504:glusterd_add_volume_to_dict] 0-management: Returning with 0
[2016-03-22 12:25:16.356200] D [MSGID: 0] [glusterd-rpc-ops.c:1579:glusterd_rpc_friend_add] 0-glusterd: Returning 0
[2016-03-22 12:25:16.356331] D [MSGID: 0] [glusterd-sm.c:334:glusterd_ac_friend_add] 0-glusterd: Returning with 0
[2016-03-22 12:25:16.356364] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.356403] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Peer in Cluster' to 'Peer in Cluster' due to event 'GD_FRIEND_EVENT_CONNECTED'
[2016-03-22 12:25:16.357218] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:16.357268] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.357311] D [MSGID: 0] [glusterd-store.c:3802:glusterd_store_create_peer_dir] 0-glusterd: Returning with 0
[2016-03-22 12:25:16.357652] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:16.357815] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 2 times between [2016-03-22 12:25:16.357652] and [2016-03-22 12:25:16.357808]
[2016-03-22 12:25:16.357813] D [MSGID: 0] [glusterd-store.c:3931:glusterd_store_peer_write] 0-glusterd: Returning with 0
[2016-03-22 12:25:16.388652] D [MSGID: 0] [glusterd-store.c:3957:glusterd_store_perform_peer_store] 0-glusterd: Returning 0
[2016-03-22 12:25:16.388710] D [MSGID: 0] [glusterd-store.c:3978:glusterd_store_peerinfo] 0-glusterd: Returning with 0
[2016-03-22 12:25:16.388824] I [MSGID: 106490] [glusterd-handler.c:2539:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:16.389952] E [MSGID: 101076] [common-utils.c:2954:gf_get_hostname_from_ip] 0-common-utils: Could not lookup hostname of 10.32.1.144 : Temporary failure in name resolution
[2016-03-22 12:25:16.389999] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.390234] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_RCVD_FRIEND_REQ'
[2016-03-22 12:25:16.390271] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.390305] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_RCVD_FRIEND_REQ'
[2016-03-22 12:25:16.390339] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.390400] D [MSGID: 0] [glusterd-utils.c:2830:import_prdict_dict] 0-glusterd: Returning with 0
[2016-03-22 12:25:16.390456] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:16.390490] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:16.390539] D [MSGID: 0] [glusterd-utils.c:2757:glusterd_compare_friend_volume] 0-management: quota checksum absent for volume c_glusterfs in peer 10.32.1.144's response
[2016-03-22 12:25:16.390569] D [MSGID: 0] [glusterd-utils.c:2776:glusterd_compare_friend_volume] 0-management: Returning with ret: 0, status: 1
[2016-03-22 12:25:16.390596] D [MSGID: 0] [glusterd-utils.c:4078:glusterd_compare_friend_data] 0-management: Returning with ret: 0, status: 1
[2016-03-22 12:25:16.390684] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:25:16.412397] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_LOCAL_ACC'
[2016-03-22 12:25:16.412548] I [MSGID: 106493] [glusterd-handler.c:3780:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 10.32.1.144 (0), ret: 0
[2016-03-22 12:25:16.412581] D [MSGID: 0] [glusterd-sm.c:911:glusterd_ac_handle_friend_add_req] 0-glusterd: Returning with 0
[2016-03-22 12:25:16.412615] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.412649] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Peer in Cluster' to 'Peer in Cluster' due to event 'GD_FRIEND_EVENT_RCVD_FRIEND_REQ'
[2016-03-22 12:25:16.412687] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:16.412717] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.412758] D [MSGID: 0] [glusterd-store.c:3802:glusterd_store_create_peer_dir] 0-glusterd: Returning with 0
[2016-03-22 12:25:16.413063] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:16.413223] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 2 times between [2016-03-22 12:25:16.413063] and [2016-03-22 12:25:16.413218]
[2016-03-22 12:25:16.413222] D [MSGID: 0] [glusterd-store.c:3931:glusterd_store_peer_write] 0-glusterd: Returning with 0
[2016-03-22 12:25:16.429485] D [MSGID: 0] [glusterd-store.c:3957:glusterd_store_perform_peer_store] 0-glusterd: Returning 0
[2016-03-22 12:25:16.429549] D [MSGID: 0] [glusterd-store.c:3978:glusterd_store_peerinfo] 0-glusterd: Returning with 0
[2016-03-22 12:25:16.430154] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.430275] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_LOCAL_ACC'
[2016-03-22 12:25:16.430311] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.430556] D [MSGID: 0] [glusterd-peer-utils.c:615:gd_add_friend_to_dict] 0-management: Returning 0
[2016-03-22 12:25:16.430919] D [MSGID: 0] [glusterd-rpc-ops.c:1674:glusterd_rpc_friend_update] 0-glusterd: Returning 0
[2016-03-22 12:25:16.430967] D [MSGID: 0] [glusterd-sm.c:590:glusterd_ac_send_friend_update] 0-glusterd: Returning with 0
[2016-03-22 12:25:16.431120] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.431152] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Peer in Cluster' to 'Peer in Cluster' due to event 'GD_FRIEND_EVENT_LOCAL_ACC'
[2016-03-22 12:25:16.431180] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:16.431207] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.431249] D [MSGID: 0] [glusterd-store.c:3802:glusterd_store_create_peer_dir] 0-glusterd: Returning with 0
[2016-03-22 12:25:16.433130] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:16.433451] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 2 times between [2016-03-22 12:25:16.433130] and [2016-03-22 12:25:16.433446]
[2016-03-22 12:25:16.433450] D [MSGID: 0] [glusterd-store.c:3931:glusterd_store_peer_write] 0-glusterd: Returning with 0
[2016-03-22 12:25:16.451224] D [MSGID: 0] [glusterd-store.c:3957:glusterd_store_perform_peer_store] 0-glusterd: Returning 0
[2016-03-22 12:25:16.451293] D [MSGID: 0] [glusterd-store.c:3978:glusterd_store_peerinfo] 0-glusterd: Returning with 0
[2016-03-22 12:25:16.451391] I [MSGID: 106493] [glusterd-rpc-ops.c:480:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2, host: 10.32.1.144, port: 0
[2016-03-22 12:25:16.451455] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.451498] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_RCVD_ACC'
[2016-03-22 12:25:16.451528] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.451556] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_RCVD_ACC'
[2016-03-22 12:25:16.451583] D [MSGID: 0] [glusterd-sm.c:203:glusterd_ac_none] 0-glusterd: Returning with 0
[2016-03-22 12:25:16.451623] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.451653] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Peer in Cluster' to 'Peer in Cluster' due to event 'GD_FRIEND_EVENT_RCVD_ACC'
[2016-03-22 12:25:16.451680] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:16.451708] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.451753] D [MSGID: 0] [glusterd-store.c:3802:glusterd_store_create_peer_dir] 0-glusterd: Returning with 0
[2016-03-22 12:25:16.452200] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:16.452391] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 2 times between [2016-03-22 12:25:16.452200] and [2016-03-22 12:25:16.452385]
[2016-03-22 12:25:16.452390] D [MSGID: 0] [glusterd-store.c:3931:glusterd_store_peer_write] 0-glusterd: Returning with 0
[2016-03-22 12:25:16.466870] D [MSGID: 0] [glusterd-store.c:3957:glusterd_store_perform_peer_store] 0-glusterd: Returning 0
[2016-03-22 12:25:16.466944] D [MSGID: 0] [glusterd-store.c:3978:glusterd_store_peerinfo] 0-glusterd: Returning with 0
[2016-03-22 12:25:16.467593] I [rpc-clnt.c:984:rpc_clnt_connection_init] 0-nfs: setting frame-timeout to 600
[2016-03-22 12:25:16.467625] D [rpc-clnt.c:1001:rpc_clnt_connection_init] 0-nfs: disable ping-timeout
[2016-03-22 12:25:16.467662] D [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3.7.6/rpc-transport/socket.so
[2016-03-22 12:25:16.467786] D [socket.c:3794:socket_init] 0-nfs: disabling nodelay
[2016-03-22 12:25:16.467815] D [socket.c:3845:socket_init] 0-nfs: Configued transport.tcp-user-timeout=0
[2016-03-22 12:25:16.467845] D [socket.c:3928:socket_init] 0-nfs: SSL support on the I/O path is NOT enabled
[2016-03-22 12:25:16.467873] D [socket.c:3931:socket_init] 0-nfs: SSL support for glusterd is NOT enabled
[2016-03-22 12:25:16.467895] D [socket.c:3948:socket_init] 0-nfs: using system polling thread
[2016-03-22 12:25:16.467998] D [MSGID: 0] [glusterd-svc-mgmt.c:109:glusterd_svc_init_common] 0-management: Returning 0
[2016-03-22 12:25:16.468036] D [MSGID: 0] [glusterd-nfs-svc.c:84:glusterd_nfssvc_manager] 0-management: nfs service initialized
[2016-03-22 12:25:16.468331] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already stopped
[2016-03-22 12:25:16.468966] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff79d1e430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff7678eab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff7678f8b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:25:16.469047] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:16.469079] D [MSGID: 0] [glusterd-nfs-svc.c:135:glusterd_nfssvc_stop] 0-management: Returning 0
[2016-03-22 12:25:16.469302] D [MSGID: 0] [glusterd-volgen.c:4809:build_nfs_graph] 0-glusterd: Returning 0
[2016-03-22 12:25:16.469829] D [MSGID: 0] [glusterd-nfs-svc.c:106:glusterd_nfssvc_manager] 0-management: Returning 0
[2016-03-22 12:25:16.470741] I [rpc-clnt.c:984:rpc_clnt_connection_init] 0-glustershd: setting frame-timeout to 600
[2016-03-22 12:25:16.470763] D [rpc-clnt.c:1001:rpc_clnt_connection_init] 0-glustershd: disable ping-timeout
[2016-03-22 12:25:16.470798] D [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3.7.6/rpc-transport/socket.so
[2016-03-22 12:25:16.470893] D [socket.c:3794:socket_init] 0-glustershd: disabling nodelay
[2016-03-22 12:25:16.470920] D [socket.c:3845:socket_init] 0-glustershd: Configued transport.tcp-user-timeout=0
[2016-03-22 12:25:16.470961] D [socket.c:3928:socket_init] 0-glustershd: SSL support on the I/O path is NOT enabled
[2016-03-22 12:25:16.470984] D [socket.c:3931:socket_init] 0-glustershd: SSL support for glusterd is NOT enabled
[2016-03-22 12:25:16.471004] D [socket.c:3948:socket_init] 0-glustershd: using system polling thread
[2016-03-22 12:25:16.471070] D [MSGID: 0] [glusterd-svc-mgmt.c:109:glusterd_svc_init_common] 0-management: Returning 0
[2016-03-22 12:25:16.471099] D [MSGID: 0] [glusterd-shd-svc.c:98:glusterd_shdsvc_manager] 0-management: shd service initialized
[2016-03-22 12:25:16.471434] D [MSGID: 0] [glusterd-volgen.c:881:_xl_link_children] 0-management: glustershd:c_glusterfs-replicate-0
[2016-03-22 12:25:16.472562] D [MSGID: 0] [glusterd-shd-svc.c:78:glusterd_shdsvc_create_volfile] 0-management: Returning 0
[2016-03-22 12:25:16.473724] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: glustershd already stopped
[2016-03-22 12:25:16.474150] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff79d1e430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff7678eab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff7678f8b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:25:16.474252] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:16.474345] D [MSGID: 0] [glusterd-svc-mgmt.c:195:glusterd_svc_start] 0-management: Starting glustershd service
[2016-03-22 12:25:16.482368] D [MSGID: 0] [glusterd-svc-mgmt.c:208:glusterd_svc_start] 0-management: Returning 0
[2016-03-22 12:25:16.482527] D [MSGID: 0] [glusterd-shd-svc.c:177:glusterd_shdsvc_start] 0-management: Returning 0
[2016-03-22 12:25:16.482659] W [socket.c:3009:socket_connect] 0-glustershd: Ignore failed connection attempt on , (No such file or directory) 
[2016-03-22 12:25:16.482770] D [MSGID: 0] [glusterd-shd-svc.c:139:glusterd_shdsvc_manager] 0-management: Returning 0
[2016-03-22 12:25:16.483457] I [rpc-clnt.c:984:rpc_clnt_connection_init] 0-quotad: setting frame-timeout to 600
[2016-03-22 12:25:16.483489] D [rpc-clnt.c:1001:rpc_clnt_connection_init] 0-quotad: disable ping-timeout
[2016-03-22 12:25:16.483523] D [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3.7.6/rpc-transport/socket.so
[2016-03-22 12:25:16.483662] D [socket.c:3794:socket_init] 0-quotad: disabling nodelay
[2016-03-22 12:25:16.483687] D [socket.c:3845:socket_init] 0-quotad: Configued transport.tcp-user-timeout=0
[2016-03-22 12:25:16.483712] D [socket.c:3928:socket_init] 0-quotad: SSL support on the I/O path is NOT enabled
[2016-03-22 12:25:16.483730] D [socket.c:3931:socket_init] 0-quotad: SSL support for glusterd is NOT enabled
[2016-03-22 12:25:16.483748] D [socket.c:3948:socket_init] 0-quotad: using system polling thread
[2016-03-22 12:25:16.483895] D [MSGID: 0] [glusterd-svc-mgmt.c:109:glusterd_svc_init_common] 0-management: Returning 0
[2016-03-22 12:25:16.483946] D [MSGID: 0] [glusterd-quotad-svc.c:72:glusterd_quotadsvc_manager] 0-management: quotad service initialized
[2016-03-22 12:25:16.484410] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: quotad already stopped
[2016-03-22 12:25:16.485112] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff79d1e430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff7678eab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff7678f8b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:25:16.485195] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:16.485227] D [MSGID: 0] [glusterd-quotad-svc.c:111:glusterd_quotadsvc_manager] 0-management: Returning 0
[2016-03-22 12:25:16.485735] I [rpc-clnt.c:984:rpc_clnt_connection_init] 0-bitd: setting frame-timeout to 600
[2016-03-22 12:25:16.485762] D [rpc-clnt.c:1001:rpc_clnt_connection_init] 0-bitd: disable ping-timeout
[2016-03-22 12:25:16.485796] D [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3.7.6/rpc-transport/socket.so
[2016-03-22 12:25:16.485983] D [socket.c:3794:socket_init] 0-bitd: disabling nodelay
[2016-03-22 12:25:16.486013] D [socket.c:3845:socket_init] 0-bitd: Configued transport.tcp-user-timeout=0
[2016-03-22 12:25:16.486041] D [socket.c:3928:socket_init] 0-bitd: SSL support on the I/O path is NOT enabled
[2016-03-22 12:25:16.486060] D [socket.c:3931:socket_init] 0-bitd: SSL support for glusterd is NOT enabled
[2016-03-22 12:25:16.486080] D [socket.c:3948:socket_init] 0-bitd: using system polling thread
[2016-03-22 12:25:16.486171] D [MSGID: 0] [glusterd-svc-mgmt.c:109:glusterd_svc_init_common] 0-management: Returning 0
[2016-03-22 12:25:16.486212] D [MSGID: 0] [glusterd-bitd-svc.c:84:glusterd_bitdsvc_manager] 0-management: BitD service initialized
[2016-03-22 12:25:16.486675] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: bitd already stopped
[2016-03-22 12:25:16.487420] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff79d1e430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff7678eab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff7678f8b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:25:16.487564] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:16.487604] D [MSGID: 0] [glusterd-bitd-svc.c:109:glusterd_bitdsvc_manager] 0-management: Returning 0
[2016-03-22 12:25:16.488181] I [rpc-clnt.c:984:rpc_clnt_connection_init] 0-scrub: setting frame-timeout to 600
[2016-03-22 12:25:16.488217] D [rpc-clnt.c:1001:rpc_clnt_connection_init] 0-scrub: disable ping-timeout
[2016-03-22 12:25:16.488250] D [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3.7.6/rpc-transport/socket.so
[2016-03-22 12:25:16.488374] D [socket.c:3794:socket_init] 0-scrub: disabling nodelay
[2016-03-22 12:25:16.488398] D [socket.c:3845:socket_init] 0-scrub: Configued transport.tcp-user-timeout=0
[2016-03-22 12:25:16.488424] D [socket.c:3928:socket_init] 0-scrub: SSL support on the I/O path is NOT enabled
[2016-03-22 12:25:16.488451] D [socket.c:3931:socket_init] 0-scrub: SSL support for glusterd is NOT enabled
[2016-03-22 12:25:16.488471] D [socket.c:3948:socket_init] 0-scrub: using system polling thread
[2016-03-22 12:25:16.488544] D [MSGID: 0] [glusterd-svc-mgmt.c:109:glusterd_svc_init_common] 0-management: Returning 0
[2016-03-22 12:25:16.488578] D [MSGID: 0] [glusterd-scrub-svc.c:79:glusterd_scrubsvc_manager] 0-management: scrub service initialized
[2016-03-22 12:25:16.489108] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: scrub already stopped
[2016-03-22 12:25:16.489893] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff79d1e430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff7678eab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff7678f8b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:25:16.490001] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:16.490035] D [MSGID: 0] [glusterd-scrub-svc.c:104:glusterd_scrubsvc_manager] 0-management: Returning 0
[2016-03-22 12:25:16.490064] D [MSGID: 0] [glusterd-utils.c:4564:glusterd_restart_bricks] 0-management: starting the volume c_glusterfs
[2016-03-22 12:25:16.499342] I [rpc-clnt.c:984:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2016-03-22 12:25:16.499410] D [rpc-clnt.c:1001:rpc_clnt_connection_init] 0-management: disable ping-timeout
[2016-03-22 12:25:16.499457] D [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3.7.6/rpc-transport/socket.so
[2016-03-22 12:25:16.499610] D [socket.c:3794:socket_init] 0-management: disabling nodelay
[2016-03-22 12:25:16.499635] D [socket.c:3845:socket_init] 0-management: Configued transport.tcp-user-timeout=0
[2016-03-22 12:25:16.499660] D [socket.c:3928:socket_init] 0-management: SSL support on the I/O path is NOT enabled
[2016-03-22 12:25:16.499678] D [socket.c:3931:socket_init] 0-management: SSL support for glusterd is NOT enabled
[2016-03-22 12:25:16.499696] D [socket.c:3948:socket_init] 0-management: using system polling thread
[2016-03-22 12:25:16.499911] D [MSGID: 0] [glusterd-handler.c:3323:glusterd_rpc_create] 0-management: returning 0
[2016-03-22 12:25:16.499962] D [MSGID: 0] [glusterd-utils.c:1572:glusterd_brick_connect] 0-glusterd: Returning 0
[2016-03-22 12:25:16.500005] D [MSGID: 0] [glusterd-utils.c:4539:glusterd_brick_start] 0-management: returning 0 
[2016-03-22 12:25:16.500076] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:25:16.500046] D [MSGID: 0] [glusterd-utils.c:4539:glusterd_brick_start] 0-management: returning 0 
[2016-03-22 12:25:16.500075] D [MSGID: 0] [glusterd-snapshot-utils.c:3595:glusterd_is_snapd_enabled] 0-management: Key features.uss not present in the dict for volume c_glusterfs
[2016-03-22 12:25:16.500313] I [MSGID: 106493] [glusterd-rpc-ops.c:694:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:16.500607] D [MSGID: 0] [glusterd-utils.c:1400:glusterd_volinfo_find_by_volume_id] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:16.500699] D [MSGID: 0] [glusterd-utils.c:884:glusterd_brickinfo_new] 0-management: Returning 0
[2016-03-22 12:25:16.500736] D [MSGID: 0] [glusterd-utils.c:1041:glusterd_brickinfo_new_from_brick] 0-management: Returning 0
[2016-03-22 12:25:16.501262] D [MSGID: 0] [glusterd-peer-utils.c:133:glusterd_peerinfo_find_by_hostname] 0-management: Unable to find friend: 10.32.0.48
[2016-03-22 12:25:16.501375] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.0.48 
[2016-03-22 12:25:16.502016] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.0.48 is local address at interface efd
[2016-03-22 12:25:16.502054] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:16.502084] D [MSGID: 0] [glusterd-utils.c:1326:glusterd_volume_brickinfo_get] 0-management: Found brick 10.32.0.48:/opt/lvmdir/c2/brick in volume c_glusterfs
[2016-03-22 12:25:16.502111] D [MSGID: 0] [glusterd-utils.c:1335:glusterd_volume_brickinfo_get] 0-management: Returning 0
[2016-03-22 12:25:16.502140] D [MSGID: 0] [glusterd-utils.c:1359:glusterd_volume_brickinfo_get_by_brick] 0-glusterd: Returning 0
[2016-03-22 12:25:16.502176] D [MSGID: 0] [glusterd-handler.c:4897:__glusterd_brick_rpc_notify] 0-management: Connected to 10.32.0.48:/opt/lvmdir/c2/brick
[2016-03-22 12:25:16.502205] D [MSGID: 0] [glusterd-utils.c:4869:glusterd_set_brick_status] 0-glusterd: Setting brick 10.32.0.48:/opt/lvmdir/c2/brick status to started
[2016-03-22 12:25:16.502248] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:16.502315] D [MSGID: 0] [glusterd-svc-mgmt.c:315:glusterd_svc_common_rpc_notify] 0-management: glustershd has connected with glusterd.
[2016-03-22 12:25:16.502419] W [socket.c:588:__socket_rwv] 0-glustershd: readv on /var/run/gluster/517bcdf9740ac943aa9615504f526f12.socket failed (Invalid argument)
[2016-03-22 12:25:16.502446] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:16.502557] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup-0x19b20)[0x3fff79d1eaf0] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_notify-0x18e68)[0x3fff79d1f808] (--> /usr/lib64/libgfrpc.so.0(rpc_transport_notify-0x1d8bc)[0x3fff79d1a85c] ))))) 0-: /var/run/gluster/676dea2d0a76d84a15022e8ceef3a57c.socket: ping timer event already removed
[2016-03-22 12:25:16.502610] D [MSGID: 0] [glusterd-utils.c:1400:glusterd_volinfo_find_by_volume_id] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:16.502645] D [MSGID: 0] [glusterd-utils.c:884:glusterd_brickinfo_new] 0-management: Returning 0
[2016-03-22 12:25:16.502686] D [MSGID: 0] [glusterd-utils.c:1041:glusterd_brickinfo_new_from_brick] 0-management: Returning 0
[2016-03-22 12:25:16.502698] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup-0x19b20)[0x3fff79d1eaf0] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_notify-0x18e68)[0x3fff79d1f808] (--> /usr/lib64/libgfrpc.so.0(rpc_transport_notify-0x1d8bc)[0x3fff79d1a85c] ))))) 0-: /var/run/gluster/517bcdf9740ac943aa9615504f526f12.socket: ping timer event already removed
[2016-03-22 12:25:16.502998] D [MSGID: 0] [glusterd-peer-utils.c:133:glusterd_peerinfo_find_by_hostname] 0-management: Unable to find friend: 10.32.0.48
[2016-03-22 12:25:16.503098] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.0.48 
[2016-03-22 12:25:16.503768] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.0.48 is local address at interface efd
[2016-03-22 12:25:16.503821] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:16.503858] D [MSGID: 0] [glusterd-utils.c:1326:glusterd_volume_brickinfo_get] 0-management: Found brick 10.32.0.48:/opt/lvmdir/c2/brick in volume c_glusterfs
[2016-03-22 12:25:16.503893] D [MSGID: 0] [glusterd-utils.c:1335:glusterd_volume_brickinfo_get] 0-management: Returning 0
[2016-03-22 12:25:16.504333] D [MSGID: 0] [glusterd-utils.c:1359:glusterd_volume_brickinfo_get_by_brick] 0-glusterd: Returning 0
[2016-03-22 12:25:16.504468] I [MSGID: 106005] [glusterd-handler.c:4908:__glusterd_brick_rpc_notify] 0-management: Brick 10.32.0.48:/opt/lvmdir/c2/brick has disconnected from glusterd.
[2016-03-22 12:25:16.504509] D [MSGID: 0] [glusterd-utils.c:4872:glusterd_set_brick_status] 0-glusterd: Setting brick 10.32.0.48:/opt/lvmdir/c2/brick status to stopped
[2016-03-22 12:25:16.504575] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=6 from gen=1 to slot->gen=2, fd=15, slot->fd=15
[2016-03-22 12:25:16.504682] I [MSGID: 106006] [glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: glustershd has disconnected from glusterd.
[2016-03-22 12:25:16.504739] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=5 from gen=1 to slot->gen=2, fd=7, slot->fd=7
[2016-03-22 12:25:16.504828] D [MSGID: 0] [glusterd-utils.c:173:glusterd_lock] 0-management: Cluster lock held by 356dabed-4314-48be-9eef-d2711d01270d
[2016-03-22 12:25:16.505210] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.505273] I [MSGID: 106492] [glusterd-handler.c:2715:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:16.505354] I [MSGID: 106502] [glusterd-handler.c:2760:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2016-03-22 12:25:16.507193] I [MSGID: 106488] [glusterd-handler.c:1472:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req
[2016-03-22 12:25:16.507306] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:16.507340] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:16.510843] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:16.510909] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:16.511003] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=4 from gen=1 to slot->gen=2, fd=8, slot->fd=8
[2016-03-22 12:25:16.518275] D [MSGID: 0] [glusterd-handshake.c:409:glusterd_get_args_from_dict] 0-management: No brick name present
[2016-03-22 12:25:16.518329] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning -1
[2016-03-22 12:25:16.518495] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 127.0.0.1 
[2016-03-22 12:25:16.530885] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:16.530915] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:16.530980] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=5 from gen=4 to slot->gen=5, fd=8, slot->fd=8
[2016-03-22 12:25:16.531863] D [MSGID: 0] [glusterd-handshake.c:414:glusterd_get_args_from_dict] 0-management: brick_name = /opt/lvmdir/c2/brick
[2016-03-22 12:25:16.531902] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning -1
[2016-03-22 12:25:16.532068] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.0.48 
[2016-03-22 12:25:16.532738] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.0.48 is local address at interface efd
[2016-03-22 12:25:16.532782] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning -1
[2016-03-22 12:25:16.532813] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:16.532840] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:16.533268] D [MSGID: 0] [glusterd-handshake.c:841:__server_getspec] 0-management: Look for missing snap creates for /opt/lvmdir/c2/brick
[2016-03-22 12:25:16.565829] I [MSGID: 106143] [glusterd-pmap.c:229:pmap_registry_bind] 0-pmap: adding brick /opt/lvmdir/c2/brick on port 49729
[2016-03-22 12:25:16.566040] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.0.48 
[2016-03-22 12:25:16.566738] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.0.48 is local address at interface efd
[2016-03-22 12:25:16.571674] I [MSGID: 106488] [glusterd-handler.c:1472:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req
[2016-03-22 12:25:16.571739] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:16.571766] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:16.575401] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:16.575467] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:16.575516] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=6 from gen=4 to slot->gen=5, fd=8, slot->fd=8
[2016-03-22 12:25:20.351004] D [MSGID: 0] [glusterd-handler.c:910:glusterd_req_ctx_create] 0-management: Received op from uuid 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:20.351106] D [MSGID: 0] [glusterd-handler.c:985:__glusterd_handle_stage_op] 0-management: transaction ID = e739fff3-8668-4c1a-9fdc-6d71196119bd
[2016-03-22 12:25:20.351183] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:20.351222] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning -2
[2016-03-22 12:25:20.351253] D [MSGID: 0] [glusterd-handler.c:1005:__glusterd_handle_stage_op] 0-management: No transaction's opinfo set
[2016-03-22 12:25:20.351317] D [MSGID: 0] [glusterd-op-sm.c:311:glusterd_set_txn_opinfo] 0-management: Successfully set opinfo for transaction ID : e739fff3-8668-4c1a-9fdc-6d71196119bd
[2016-03-22 12:25:20.351353] D [MSGID: 0] [glusterd-op-sm.c:318:glusterd_set_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.351394] D [MSGID: 0] [glusterd-op-sm.c:7346:glusterd_op_sm_inject_event] 0-management: Enqueue event: 'GD_OP_EVENT_STAGE_OP'
[2016-03-22 12:25:20.351429] D [MSGID: 0] [glusterd-op-sm.c:7428:glusterd_op_sm] 0-management: Dequeued event of type: 'GD_OP_EVENT_STAGE_OP'
[2016-03-22 12:25:20.351468] D [MSGID: 0] [glusterd-op-sm.c:7431:glusterd_op_sm] 0-management: transaction ID = e739fff3-8668-4c1a-9fdc-6d71196119bd
[2016-03-22 12:25:20.351507] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : e739fff3-8668-4c1a-9fdc-6d71196119bd
[2016-03-22 12:25:20.351538] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.351586] D [MSGID: 0] [glusterd-op-sm.c:1695:glusterd_op_stage_status_volume] 0-management: Returning: 0
[2016-03-22 12:25:20.351619] D [MSGID: 0] [glusterd-op-sm.c:5515:glusterd_op_stage_validate] 0-management: OP = 18. Returning 0
[2016-03-22 12:25:20.351777] D [MSGID: 0] [glusterd-handler.c:2469:glusterd_op_stage_send_resp] 0-management: Responded to stage, ret: 0
[2016-03-22 12:25:20.351810] D [MSGID: 0] [glusterd-op-sm.c:5224:glusterd_op_ac_stage_op] 0-management: Returning with 0
[2016-03-22 12:25:20.351846] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Locked' to 'Staged' due to event 'GD_OP_EVENT_STAGE_OP'
[2016-03-22 12:25:20.351879] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:20.351928] D [MSGID: 0] [glusterd-op-sm.c:311:glusterd_set_txn_opinfo] 0-management: Successfully set opinfo for transaction ID : e739fff3-8668-4c1a-9fdc-6d71196119bd
[2016-03-22 12:25:20.351962] D [MSGID: 0] [glusterd-op-sm.c:318:glusterd_set_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.354690] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:20.354764] D [MSGID: 0] [glusterd-handler.c:910:glusterd_req_ctx_create] 0-management: Received op from uuid 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:20.354821] D [MSGID: 0] [glusterd-handler.c:1092:__glusterd_handle_commit_op] 0-management: transaction ID = e739fff3-8668-4c1a-9fdc-6d71196119bd
[2016-03-22 12:25:20.354852] D [MSGID: 0] [glusterd-op-sm.c:7346:glusterd_op_sm_inject_event] 0-management: Enqueue event: 'GD_OP_EVENT_COMMIT_OP'
[2016-03-22 12:25:20.354883] D [MSGID: 0] [glusterd-op-sm.c:7428:glusterd_op_sm] 0-management: Dequeued event of type: 'GD_OP_EVENT_COMMIT_OP'
[2016-03-22 12:25:20.354931] D [MSGID: 0] [glusterd-op-sm.c:7431:glusterd_op_sm] 0-management: transaction ID = e739fff3-8668-4c1a-9fdc-6d71196119bd
[2016-03-22 12:25:20.354986] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : e739fff3-8668-4c1a-9fdc-6d71196119bd
[2016-03-22 12:25:20.355019] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.355074] D [MSGID: 0] [glusterd-rpc-ops.c:2220:glusterd_brick_op] 0-management: transaction ID = e739fff3-8668-4c1a-9fdc-6d71196119bd
[2016-03-22 12:25:20.355117] D [MSGID: 0] [glusterd-op-sm.c:7010:glusterd_op_bricks_select] 0-management: Returning 0
[2016-03-22 12:25:20.355148] D [MSGID: 0] [glusterd-rpc-ops.c:2323:glusterd_brick_op] 0-management: Returning 0
[2016-03-22 12:25:20.355180] D [MSGID: 0] [glusterd-op-sm.c:7346:glusterd_op_sm_inject_event] 0-management: Enqueue event: 'GD_OP_EVENT_ALL_ACK'
[2016-03-22 12:25:20.355210] D [MSGID: 0] [glusterd-op-sm.c:6896:glusterd_op_ac_send_brick_op] 0-management: Returning with 0
[2016-03-22 12:25:20.355243] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Staged' to 'Brick op Committed' due to event 'GD_OP_EVENT_COMMIT_OP'
[2016-03-22 12:25:20.355277] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:20.355318] D [MSGID: 0] [glusterd-op-sm.c:311:glusterd_set_txn_opinfo] 0-management: Successfully set opinfo for transaction ID : e739fff3-8668-4c1a-9fdc-6d71196119bd
[2016-03-22 12:25:20.355350] D [MSGID: 0] [glusterd-op-sm.c:318:glusterd_set_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.355384] D [MSGID: 0] [glusterd-op-sm.c:7428:glusterd_op_sm] 0-management: Dequeued event of type: 'GD_OP_EVENT_ALL_ACK'
[2016-03-22 12:25:20.355419] D [MSGID: 0] [glusterd-op-sm.c:7431:glusterd_op_sm] 0-management: transaction ID = e739fff3-8668-4c1a-9fdc-6d71196119bd
[2016-03-22 12:25:20.355458] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : e739fff3-8668-4c1a-9fdc-6d71196119bd
[2016-03-22 12:25:20.355492] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.355578] D [MSGID: 0] [glusterd-op-sm.c:3360:glusterd_op_status_volume] 0-management: Returning 0
[2016-03-22 12:25:20.355612] D [MSGID: 0] [glusterd-op-sm.c:5642:glusterd_op_commit_perform] 0-management: Returning 0
[2016-03-22 12:25:20.355765] D [MSGID: 0] [glusterd-handler.c:2511:glusterd_op_commit_send_resp] 0-management: Responded to commit, ret: 0
[2016-03-22 12:25:20.355802] D [MSGID: 0] [glusterd-op-sm.c:5341:glusterd_op_ac_commit_op] 0-management: Returning with 0
[2016-03-22 12:25:20.355836] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Brick op Committed' to 'Committed' due to event 'GD_OP_EVENT_ALL_ACK'
[2016-03-22 12:25:20.355869] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:20.355908] D [MSGID: 0] [glusterd-op-sm.c:311:glusterd_set_txn_opinfo] 0-management: Successfully set opinfo for transaction ID : e739fff3-8668-4c1a-9fdc-6d71196119bd
[2016-03-22 12:25:20.355957] D [MSGID: 0] [glusterd-op-sm.c:318:glusterd_set_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.363005] D [MSGID: 0] [glusterd-mgmt-handler.c:151:glusterd_handle_mgmt_v3_lock_fn] 0-management: Received mgmt_v3 lock req from uuid: 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:20.363078] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:20.363158] D [MSGID: 0] [glusterd-op-sm.c:311:glusterd_set_txn_opinfo] 0-management: Successfully set opinfo for transaction ID : 00999cc1-70ab-410c-b2dd-b5e197de6f90
[2016-03-22 12:25:20.363187] D [MSGID: 0] [glusterd-op-sm.c:318:glusterd_set_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.363218] D [MSGID: 0] [glusterd-op-sm.c:7346:glusterd_op_sm_inject_event] 0-management: Enqueue event: 'GD_OP_EVENT_LOCK'
[2016-03-22 12:25:20.363264] D [MSGID: 0] [glusterd-op-sm.c:7428:glusterd_op_sm] 0-management: Dequeued event of type: 'GD_OP_EVENT_LOCK'
[2016-03-22 12:25:20.363296] D [MSGID: 0] [glusterd-op-sm.c:7431:glusterd_op_sm] 0-management: transaction ID = 00999cc1-70ab-410c-b2dd-b5e197de6f90
[2016-03-22 12:25:20.363334] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : 00999cc1-70ab-410c-b2dd-b5e197de6f90
[2016-03-22 12:25:20.363363] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.363403] D [MSGID: 0] [glusterd-locks.c:562:glusterd_mgmt_v3_lock] 0-management: Trying to acquire lock of vol c_glusterfs for 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2 as c_glusterfs_vol
[2016-03-22 12:25:20.364009] D [MSGID: 0] [glusterd-locks.c:618:glusterd_mgmt_v3_lock] 0-management: Lock for vol c_glusterfs successfully held by 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:20.364135] D [MSGID: 0] [glusterd-handler.c:2334:glusterd_op_mgmt_v3_lock_send_resp] 0-management: Responded to mgmt_v3 lock, ret: 0
[2016-03-22 12:25:20.364166] D [MSGID: 0] [glusterd-op-sm.c:3676:glusterd_op_ac_lock] 0-management: Lock Returned 0
[2016-03-22 12:25:20.364196] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Default' to 'Locked' due to event 'GD_OP_EVENT_LOCK'
[2016-03-22 12:25:20.364225] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:20.364260] D [MSGID: 0] [glusterd-op-sm.c:311:glusterd_set_txn_opinfo] 0-management: Successfully set opinfo for transaction ID : 00999cc1-70ab-410c-b2dd-b5e197de6f90
[2016-03-22 12:25:20.364289] D [MSGID: 0] [glusterd-op-sm.c:318:glusterd_set_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.367955] D [MSGID: 0] [glusterd-handler.c:910:glusterd_req_ctx_create] 0-management: Received op from uuid 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:20.368024] D [MSGID: 0] [glusterd-handler.c:985:__glusterd_handle_stage_op] 0-management: transaction ID = 00999cc1-70ab-410c-b2dd-b5e197de6f90
[2016-03-22 12:25:20.368056] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:20.368094] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : 00999cc1-70ab-410c-b2dd-b5e197de6f90
[2016-03-22 12:25:20.368123] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.368153] D [MSGID: 0] [glusterd-op-sm.c:7346:glusterd_op_sm_inject_event] 0-management: Enqueue event: 'GD_OP_EVENT_STAGE_OP'
[2016-03-22 12:25:20.368183] D [MSGID: 0] [glusterd-op-sm.c:7428:glusterd_op_sm] 0-management: Dequeued event of type: 'GD_OP_EVENT_STAGE_OP'
[2016-03-22 12:25:20.368216] D [MSGID: 0] [glusterd-op-sm.c:7431:glusterd_op_sm] 0-management: transaction ID = 00999cc1-70ab-410c-b2dd-b5e197de6f90
[2016-03-22 12:25:20.368251] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : 00999cc1-70ab-410c-b2dd-b5e197de6f90
[2016-03-22 12:25:20.368280] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.368317] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:20.368347] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:20.368394] D [MSGID: 0] [glusterd-op-sm.c:1695:glusterd_op_stage_status_volume] 0-management: Returning: 0
[2016-03-22 12:25:20.368424] D [MSGID: 0] [glusterd-op-sm.c:5515:glusterd_op_stage_validate] 0-management: OP = 18. Returning 0
[2016-03-22 12:25:20.368520] D [MSGID: 0] [glusterd-handler.c:2469:glusterd_op_stage_send_resp] 0-management: Responded to stage, ret: 0
[2016-03-22 12:25:20.368551] D [MSGID: 0] [glusterd-op-sm.c:5224:glusterd_op_ac_stage_op] 0-management: Returning with 0
[2016-03-22 12:25:20.368595] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Locked' to 'Staged' due to event 'GD_OP_EVENT_STAGE_OP'
[2016-03-22 12:25:20.368625] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:20.368662] D [MSGID: 0] [glusterd-op-sm.c:311:glusterd_set_txn_opinfo] 0-management: Successfully set opinfo for transaction ID : 00999cc1-70ab-410c-b2dd-b5e197de6f90
[2016-03-22 12:25:20.368690] D [MSGID: 0] [glusterd-op-sm.c:318:glusterd_set_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.372951] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:20.372992] D [MSGID: 0] [glusterd-handler.c:910:glusterd_req_ctx_create] 0-management: Received op from uuid 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:20.373050] D [MSGID: 0] [glusterd-handler.c:1092:__glusterd_handle_commit_op] 0-management: transaction ID = 00999cc1-70ab-410c-b2dd-b5e197de6f90
[2016-03-22 12:25:20.373085] D [MSGID: 0] [glusterd-op-sm.c:7346:glusterd_op_sm_inject_event] 0-management: Enqueue event: 'GD_OP_EVENT_COMMIT_OP'
[2016-03-22 12:25:20.373116] D [MSGID: 0] [glusterd-op-sm.c:7428:glusterd_op_sm] 0-management: Dequeued event of type: 'GD_OP_EVENT_COMMIT_OP'
[2016-03-22 12:25:20.373146] D [MSGID: 0] [glusterd-op-sm.c:7431:glusterd_op_sm] 0-management: transaction ID = 00999cc1-70ab-410c-b2dd-b5e197de6f90
[2016-03-22 12:25:20.373182] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : 00999cc1-70ab-410c-b2dd-b5e197de6f90
[2016-03-22 12:25:20.373211] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.373244] D [MSGID: 0] [glusterd-rpc-ops.c:2220:glusterd_brick_op] 0-management: transaction ID = 00999cc1-70ab-410c-b2dd-b5e197de6f90
[2016-03-22 12:25:20.373278] D [MSGID: 0] [glusterd-op-sm.c:7010:glusterd_op_bricks_select] 0-management: Returning 0
[2016-03-22 12:25:20.373307] D [MSGID: 0] [glusterd-rpc-ops.c:2323:glusterd_brick_op] 0-management: Returning 0
[2016-03-22 12:25:20.373337] D [MSGID: 0] [glusterd-op-sm.c:7346:glusterd_op_sm_inject_event] 0-management: Enqueue event: 'GD_OP_EVENT_ALL_ACK'
[2016-03-22 12:25:20.373365] D [MSGID: 0] [glusterd-op-sm.c:6896:glusterd_op_ac_send_brick_op] 0-management: Returning with 0
[2016-03-22 12:25:20.373395] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Staged' to 'Brick op Committed' due to event 'GD_OP_EVENT_COMMIT_OP'
[2016-03-22 12:25:20.373424] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:20.373464] D [MSGID: 0] [glusterd-op-sm.c:311:glusterd_set_txn_opinfo] 0-management: Successfully set opinfo for transaction ID : 00999cc1-70ab-410c-b2dd-b5e197de6f90
[2016-03-22 12:25:20.373493] D [MSGID: 0] [glusterd-op-sm.c:318:glusterd_set_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.373522] D [MSGID: 0] [glusterd-op-sm.c:7428:glusterd_op_sm] 0-management: Dequeued event of type: 'GD_OP_EVENT_ALL_ACK'
[2016-03-22 12:25:20.373553] D [MSGID: 0] [glusterd-op-sm.c:7431:glusterd_op_sm] 0-management: transaction ID = 00999cc1-70ab-410c-b2dd-b5e197de6f90
[2016-03-22 12:25:20.373589] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : 00999cc1-70ab-410c-b2dd-b5e197de6f90
[2016-03-22 12:25:20.373621] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.373669] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:20.373698] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:20.373793] D [MSGID: 0] [glusterd-snapshot-utils.c:3595:glusterd_is_snapd_enabled] 0-management: Key features.uss not present in the dict for volume c_glusterfs
[2016-03-22 12:25:20.373889] D [MSGID: 0] [glusterd-utils.c:4342:glusterd_add_node_to_dict] 0-management: Returning 0
[2016-03-22 12:25:20.374244] D [MSGID: 0] [glusterd-op-sm.c:3360:glusterd_op_status_volume] 0-management: Returning 0
[2016-03-22 12:25:20.374282] D [MSGID: 0] [glusterd-op-sm.c:5642:glusterd_op_commit_perform] 0-management: Returning 0
[2016-03-22 12:25:20.374403] D [MSGID: 0] [glusterd-handler.c:2511:glusterd_op_commit_send_resp] 0-management: Responded to commit, ret: 0
[2016-03-22 12:25:20.374444] D [MSGID: 0] [glusterd-op-sm.c:5341:glusterd_op_ac_commit_op] 0-management: Returning with 0
[2016-03-22 12:25:20.374482] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Brick op Committed' to 'Committed' due to event 'GD_OP_EVENT_ALL_ACK'
[2016-03-22 12:25:20.374515] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:20.374557] D [MSGID: 0] [glusterd-op-sm.c:311:glusterd_set_txn_opinfo] 0-management: Successfully set opinfo for transaction ID : 00999cc1-70ab-410c-b2dd-b5e197de6f90
[2016-03-22 12:25:20.374587] D [MSGID: 0] [glusterd-op-sm.c:318:glusterd_set_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.377615] D [MSGID: 0] [glusterd-mgmt-handler.c:874:glusterd_handle_mgmt_v3_unlock_fn] 0-management: Received volume unlock req from uuid: 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:20.377650] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:20.377819] D [MSGID: 0] [glusterd-op-sm.c:7346:glusterd_op_sm_inject_event] 0-management: Enqueue event: 'GD_OP_EVENT_UNLOCK'
[2016-03-22 12:25:20.377851] D [MSGID: 0] [glusterd-op-sm.c:7428:glusterd_op_sm] 0-management: Dequeued event of type: 'GD_OP_EVENT_UNLOCK'
[2016-03-22 12:25:20.377883] D [MSGID: 0] [glusterd-op-sm.c:7431:glusterd_op_sm] 0-management: transaction ID = 00999cc1-70ab-410c-b2dd-b5e197de6f90
[2016-03-22 12:25:20.377931] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : 00999cc1-70ab-410c-b2dd-b5e197de6f90
[2016-03-22 12:25:20.377963] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.378002] D [MSGID: 0] [glusterd-locks.c:669:glusterd_mgmt_v3_unlock] 0-management: Trying to release lock of vol c_glusterfs for 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2 as c_glusterfs_vol
[2016-03-22 12:25:20.378046] D [MSGID: 0] [glusterd-locks.c:714:glusterd_mgmt_v3_unlock] 0-management: Lock for vol c_glusterfs successfully released
[2016-03-22 12:25:20.378124] D [MSGID: 0] [glusterd-handler.c:2359:glusterd_op_mgmt_v3_unlock_send_resp] 0-management: Responded to mgmt_v3 unlock, ret: 0
[2016-03-22 12:25:20.378213] D [MSGID: 0] [glusterd-op-sm.c:3740:glusterd_op_ac_unlock] 0-management: Unlock Returned 0
[2016-03-22 12:25:20.378242] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Committed' to 'Default' due to event 'GD_OP_EVENT_UNLOCK'
[2016-03-22 12:25:20.378271] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:20.378307] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : 00999cc1-70ab-410c-b2dd-b5e197de6f90
[2016-03-22 12:25:20.378335] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.378406] D [MSGID: 0] [glusterd-op-sm.c:360:glusterd_clear_txn_opinfo] 0-management: Successfully cleared opinfo for transaction ID : 00999cc1-70ab-410c-b2dd-b5e197de6f90
[2016-03-22 12:25:20.378435] D [MSGID: 0] [glusterd-op-sm.c:364:glusterd_clear_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.436195] D [MSGID: 0] [glusterd-mgmt-handler.c:151:glusterd_handle_mgmt_v3_lock_fn] 0-management: Received mgmt_v3 lock req from uuid: 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:20.436287] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:20.436407] D [MSGID: 0] [glusterd-op-sm.c:311:glusterd_set_txn_opinfo] 0-management: Successfully set opinfo for transaction ID : 1d126f79-deb2-4cf1-bbd8-1651ee9183c3
[2016-03-22 12:25:20.436446] D [MSGID: 0] [glusterd-op-sm.c:318:glusterd_set_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.436486] D [MSGID: 0] [glusterd-op-sm.c:7346:glusterd_op_sm_inject_event] 0-management: Enqueue event: 'GD_OP_EVENT_LOCK'
[2016-03-22 12:25:20.436521] D [MSGID: 0] [glusterd-op-sm.c:7428:glusterd_op_sm] 0-management: Dequeued event of type: 'GD_OP_EVENT_LOCK'
[2016-03-22 12:25:20.436560] D [MSGID: 0] [glusterd-op-sm.c:7431:glusterd_op_sm] 0-management: transaction ID = 1d126f79-deb2-4cf1-bbd8-1651ee9183c3
[2016-03-22 12:25:20.436599] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : 1d126f79-deb2-4cf1-bbd8-1651ee9183c3
[2016-03-22 12:25:20.436631] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.436675] D [MSGID: 0] [glusterd-locks.c:562:glusterd_mgmt_v3_lock] 0-management: Trying to acquire lock of vol c_glusterfs for 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2 as c_glusterfs_vol
[2016-03-22 12:25:20.437345] D [MSGID: 0] [glusterd-locks.c:618:glusterd_mgmt_v3_lock] 0-management: Lock for vol c_glusterfs successfully held by 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:20.437475] D [MSGID: 0] [glusterd-handler.c:2334:glusterd_op_mgmt_v3_lock_send_resp] 0-management: Responded to mgmt_v3 lock, ret: 0
[2016-03-22 12:25:20.437510] D [MSGID: 0] [glusterd-op-sm.c:3676:glusterd_op_ac_lock] 0-management: Lock Returned 0
[2016-03-22 12:25:20.437542] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Default' to 'Locked' due to event 'GD_OP_EVENT_LOCK'
[2016-03-22 12:25:20.437574] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:20.437616] D [MSGID: 0] [glusterd-op-sm.c:311:glusterd_set_txn_opinfo] 0-management: Successfully set opinfo for transaction ID : 1d126f79-deb2-4cf1-bbd8-1651ee9183c3
[2016-03-22 12:25:20.437651] D [MSGID: 0] [glusterd-op-sm.c:318:glusterd_set_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.440743] D [MSGID: 0] [glusterd-handler.c:910:glusterd_req_ctx_create] 0-management: Received op from uuid 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:20.440824] D [MSGID: 0] [glusterd-handler.c:985:__glusterd_handle_stage_op] 0-management: transaction ID = 1d126f79-deb2-4cf1-bbd8-1651ee9183c3
[2016-03-22 12:25:20.440876] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:20.440928] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : 1d126f79-deb2-4cf1-bbd8-1651ee9183c3
[2016-03-22 12:25:20.440960] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.440993] D [MSGID: 0] [glusterd-op-sm.c:7346:glusterd_op_sm_inject_event] 0-management: Enqueue event: 'GD_OP_EVENT_STAGE_OP'
[2016-03-22 12:25:20.441026] D [MSGID: 0] [glusterd-op-sm.c:7428:glusterd_op_sm] 0-management: Dequeued event of type: 'GD_OP_EVENT_STAGE_OP'
[2016-03-22 12:25:20.441061] D [MSGID: 0] [glusterd-op-sm.c:7431:glusterd_op_sm] 0-management: transaction ID = 1d126f79-deb2-4cf1-bbd8-1651ee9183c3
[2016-03-22 12:25:20.441099] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : 1d126f79-deb2-4cf1-bbd8-1651ee9183c3
[2016-03-22 12:25:20.441130] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.441176] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:20.441215] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:20.441258] D [MSGID: 0] [glusterd-brick-ops.c:2084:glusterd_op_stage_remove_brick] 0-management: Returning 0
[2016-03-22 12:25:20.441290] D [MSGID: 0] [glusterd-op-sm.c:5515:glusterd_op_stage_validate] 0-management: OP = 9. Returning 0
[2016-03-22 12:25:20.441398] D [MSGID: 0] [glusterd-handler.c:2469:glusterd_op_stage_send_resp] 0-management: Responded to stage, ret: 0
[2016-03-22 12:25:20.441431] D [MSGID: 0] [glusterd-op-sm.c:5224:glusterd_op_ac_stage_op] 0-management: Returning with 0
[2016-03-22 12:25:20.441465] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Locked' to 'Staged' due to event 'GD_OP_EVENT_STAGE_OP'
[2016-03-22 12:25:20.441506] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:20.441545] D [MSGID: 0] [glusterd-op-sm.c:311:glusterd_set_txn_opinfo] 0-management: Successfully set opinfo for transaction ID : 1d126f79-deb2-4cf1-bbd8-1651ee9183c3
[2016-03-22 12:25:20.441576] D [MSGID: 0] [glusterd-op-sm.c:318:glusterd_set_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.535001] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:20.535082] D [MSGID: 0] [glusterd-handler.c:910:glusterd_req_ctx_create] 0-management: Received op from uuid 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:20.535166] D [MSGID: 0] [glusterd-handler.c:1092:__glusterd_handle_commit_op] 0-management: transaction ID = 1d126f79-deb2-4cf1-bbd8-1651ee9183c3
[2016-03-22 12:25:20.535202] D [MSGID: 0] [glusterd-op-sm.c:7346:glusterd_op_sm_inject_event] 0-management: Enqueue event: 'GD_OP_EVENT_COMMIT_OP'
[2016-03-22 12:25:20.535237] D [MSGID: 0] [glusterd-op-sm.c:7428:glusterd_op_sm] 0-management: Dequeued event of type: 'GD_OP_EVENT_COMMIT_OP'
[2016-03-22 12:25:20.535272] D [MSGID: 0] [glusterd-op-sm.c:7431:glusterd_op_sm] 0-management: transaction ID = 1d126f79-deb2-4cf1-bbd8-1651ee9183c3
[2016-03-22 12:25:20.535312] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : 1d126f79-deb2-4cf1-bbd8-1651ee9183c3
[2016-03-22 12:25:20.535344] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.535381] D [MSGID: 0] [glusterd-rpc-ops.c:2220:glusterd_brick_op] 0-management: transaction ID = 1d126f79-deb2-4cf1-bbd8-1651ee9183c3
[2016-03-22 12:25:20.535416] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:20.535448] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:20.535483] I [MSGID: 106062] [glusterd-op-sm.c:5745:glusterd_bricks_select_remove_brick] 0-management: force flag is not set
[2016-03-22 12:25:20.535515] D [MSGID: 0] [glusterd-op-sm.c:7010:glusterd_op_bricks_select] 0-management: Returning 0
[2016-03-22 12:25:20.535546] D [MSGID: 0] [glusterd-rpc-ops.c:2323:glusterd_brick_op] 0-management: Returning 0
[2016-03-22 12:25:20.535579] D [MSGID: 0] [glusterd-op-sm.c:7346:glusterd_op_sm_inject_event] 0-management: Enqueue event: 'GD_OP_EVENT_ALL_ACK'
[2016-03-22 12:25:20.535609] D [MSGID: 0] [glusterd-op-sm.c:6896:glusterd_op_ac_send_brick_op] 0-management: Returning with 0
[2016-03-22 12:25:20.535642] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Staged' to 'Brick op Committed' due to event 'GD_OP_EVENT_COMMIT_OP'
[2016-03-22 12:25:20.535674] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:20.535713] D [MSGID: 0] [glusterd-op-sm.c:311:glusterd_set_txn_opinfo] 0-management: Successfully set opinfo for transaction ID : 1d126f79-deb2-4cf1-bbd8-1651ee9183c3
[2016-03-22 12:25:20.535744] D [MSGID: 0] [glusterd-op-sm.c:318:glusterd_set_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.535789] D [MSGID: 0] [glusterd-op-sm.c:7428:glusterd_op_sm] 0-management: Dequeued event of type: 'GD_OP_EVENT_ALL_ACK'
[2016-03-22 12:25:20.535824] D [MSGID: 0] [glusterd-op-sm.c:7431:glusterd_op_sm] 0-management: transaction ID = 1d126f79-deb2-4cf1-bbd8-1651ee9183c3
[2016-03-22 12:25:20.535862] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : 1d126f79-deb2-4cf1-bbd8-1651ee9183c3
[2016-03-22 12:25:20.535894] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.536358] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:20.536391] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:20.536894] W [dict.c:592:dict_unref] (-->/usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_op_commit_perform-0x1350d4) [0x3fff766b575c] -->/usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_op_remove_brick-0xa98b0) [0x3fff767456c0] -->/usr/lib64/libglusterfs.so.0(dict_unref-0xc08c0) [0x3fff79d6e520] ) 0-dict: dict is NULL [Invalid argument]
[2016-03-22 12:25:20.536999] D [MSGID: 0] [glusterd-utils.c:884:glusterd_brickinfo_new] 0-management: Returning 0
[2016-03-22 12:25:20.537039] D [MSGID: 0] [glusterd-utils.c:1041:glusterd_brickinfo_new_from_brick] 0-management: Returning 0
[2016-03-22 12:25:20.537426] D [MSGID: 0] [glusterd-peer-utils.c:133:glusterd_peerinfo_find_by_hostname] 0-management: Unable to find friend: 10.32.0.48
[2016-03-22 12:25:20.537532] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.0.48 
[2016-03-22 12:25:20.538261] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.0.48 is local address at interface efd
[2016-03-22 12:25:20.538298] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:20.538331] D [MSGID: 0] [glusterd-utils.c:1326:glusterd_volume_brickinfo_get] 0-management: Found brick 10.32.0.48:/opt/lvmdir/c2/brick in volume c_glusterfs
[2016-03-22 12:25:20.538362] D [MSGID: 0] [glusterd-utils.c:1335:glusterd_volume_brickinfo_get] 0-management: Returning 0
[2016-03-22 12:25:20.538395] D [MSGID: 0] [glusterd-utils.c:1359:glusterd_volume_brickinfo_get_by_brick] 0-glusterd: Returning 0
[2016-03-22 12:25:20.538706] D [MSGID: 0] [glusterd-peer-utils.c:133:glusterd_peerinfo_find_by_hostname] 0-management: Unable to find friend: 10.32.0.48
[2016-03-22 12:25:20.538811] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.0.48 
[2016-03-22 12:25:20.539342] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.0.48 is local address at interface efd
[2016-03-22 12:25:20.539378] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:20.539409] D [MSGID: 0] [glusterd-utils.c:925:glusterd_resolve_brick] 0-management: Returning 0
[2016-03-22 12:25:20.539441] D [MSGID: 0] [glusterd-utils.c:5531:glusterd_brick_stop] 0-management: About to stop glusterfs for brick 10.32.0.48:/opt/lvmdir/c2/brick
[2016-03-22 12:25:20.539878] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff79d1e430] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_unref-0x1a098)[0x3fff79d1e548] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_rpc_clnt_unref-0x10cba4)[0x3fff766df294] ))))) 0-: /var/run/gluster/676dea2d0a76d84a15022e8ceef3a57c.socket: ping timer event already removed
[2016-03-22 12:25:20.539947] D [MSGID: 0] [glusterd-utils.c:1400:glusterd_volinfo_find_by_volume_id] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:20.539986] D [MSGID: 0] [glusterd-utils.c:884:glusterd_brickinfo_new] 0-management: Returning 0
[2016-03-22 12:25:20.540031] D [MSGID: 0] [glusterd-utils.c:1041:glusterd_brickinfo_new_from_brick] 0-management: Returning 0
[2016-03-22 12:25:20.540349] D [MSGID: 0] [glusterd-peer-utils.c:133:glusterd_peerinfo_find_by_hostname] 0-management: Unable to find friend: 10.32.0.48
[2016-03-22 12:25:20.540454] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.0.48 
[2016-03-22 12:25:20.541009] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.0.48 is local address at interface efd
[2016-03-22 12:25:20.541045] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:20.541077] D [MSGID: 0] [glusterd-utils.c:1335:glusterd_volume_brickinfo_get] 0-management: Returning -1
[2016-03-22 12:25:20.541109] D [MSGID: 0] [glusterd-utils.c:1359:glusterd_volume_brickinfo_get_by_brick] 0-glusterd: Returning -1
[2016-03-22 12:25:20.541153] I [MSGID: 101053] [mem-pool.c:616:mem_pool_destroy] 0-management: size=588 max=0 total=0
[2016-03-22 12:25:20.541187] I [MSGID: 101053] [mem-pool.c:616:mem_pool_destroy] 0-management: size=124 max=0 total=0
[2016-03-22 12:25:20.541466] D [MSGID: 0] [glusterd-utils.c:1453:glusterd_service_stop] 0-management: Stopping gluster brick running in pid: 2586
[2016-03-22 12:25:20.541513] D [MSGID: 0] [glusterd-utils.c:4872:glusterd_set_brick_status] 0-glusterd: Setting brick 10.32.0.48:/opt/lvmdir/c2/brick status to stopped
[2016-03-22 12:25:20.541860] D [MSGID: 0] [store.c:460:gf_store_handle_destroy] 0-: Returning 0
[2016-03-22 12:25:20.541892] D [MSGID: 0] [glusterd-store.c:620:glusterd_store_delete_brick] 0-management: Returning with 0
[2016-03-22 12:25:20.541953] D [MSGID: 0] [glusterd-utils.c:5542:glusterd_brick_stop] 0-management: returning 0 
[2016-03-22 12:25:20.541985] D [MSGID: 0] [glusterd-brick-ops.c:1486:glusterd_op_perform_remove_brick] 0-glusterd: Returning 0
[2016-03-22 12:25:20.542022] I [MSGID: 106062] [glusterd-brick-ops.c:2556:glusterd_op_remove_brick] 0-management: changing replica count 2 to 1 on volume c_glusterfs
[2016-03-22 12:25:20.542214] D [MSGID: 0] [glusterd-volgen.c:5109:generate_brick_volfiles] 0-management: Found a brick - 10.32.1.144:/opt/lvmdir/c2/brick
[2016-03-22 12:25:20.543954] D [MSGID: 0] [glusterd-volgen.c:1268:server_check_marker_off] 0-glusterd: Returning 0
[2016-03-22 12:25:20.545036] D [MSGID: 0] [glusterd-volgen.c:5120:generate_brick_volfiles] 0-management: Returning 0
[2016-03-22 12:25:20.545186] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.1.144 found.. state: 3
[2016-03-22 12:25:20.545219] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:20.545254] D [MSGID: 0] [glusterd-utils.c:1326:glusterd_volume_brickinfo_get] 0-management: Found brick 10.32.1.144:/opt/lvmdir/c2/brick in volume c_glusterfs
[2016-03-22 12:25:20.545284] D [MSGID: 0] [glusterd-utils.c:1335:glusterd_volume_brickinfo_get] 0-management: Returning 0
[2016-03-22 12:25:20.546827] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.1.144 found.. state: 3
[2016-03-22 12:25:20.546861] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:20.546894] D [MSGID: 0] [glusterd-utils.c:1326:glusterd_volume_brickinfo_get] 0-management: Found brick 10.32.1.144:/opt/lvmdir/c2/brick in volume c_glusterfs
[2016-03-22 12:25:20.547001] D [MSGID: 0] [glusterd-utils.c:1335:glusterd_volume_brickinfo_get] 0-management: Returning 0
[2016-03-22 12:25:20.548689] D [MSGID: 0] [glusterd-store.c:1058:glusterd_store_create_volume_dir] 0-management: Returning with 0
[2016-03-22 12:25:20.548872] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:20.561090] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 18 times between [2016-03-22 12:25:20.548872] and [2016-03-22 12:25:20.550266]
[2016-03-22 12:25:20.561087] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:25:20.561405] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:20.561438] D [MSGID: 0] [glusterd-store.c:440:glusterd_store_snapd_write] 0-management: Returning 0
[2016-03-22 12:25:20.580013] D [MSGID: 0] [glusterd-store.c:503:glusterd_store_perform_snapd_store] 0-management: Returning 0
[2016-03-22 12:25:20.580077] D [MSGID: 0] [glusterd-store.c:568:glusterd_store_snapd_info] 0-management: Returning with 0
[2016-03-22 12:25:20.580145] D [MSGID: 0] [glusterd-store.c:753:_storeopts] 0-management: Storing in volinfo:key= performance.readdir-ahead, val=on
[2016-03-22 12:25:20.580277] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:20.580314] D [MSGID: 0] [glusterd-store.c:753:_storeopts] 0-management: Storing in volinfo:key= network.ping-timeout, val=4
[2016-03-22 12:25:20.580397] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:20.580449] D [MSGID: 0] [glusterd-store.c:753:_storeopts] 0-management: Storing in volinfo:key= nfs.disable, val=on
[2016-03-22 12:25:20.580532] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:20.580565] D [MSGID: 0] [glusterd-store.c:1104:glusterd_store_volinfo_write] 0-management: Returning 0
[2016-03-22 12:25:20.580659] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:20.581342] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 8 times between [2016-03-22 12:25:20.580659] and [2016-03-22 12:25:20.581337]
[2016-03-22 12:25:20.581341] D [MSGID: 0] [glusterd-store.c:414:glusterd_store_brickinfo_write] 0-management: Returning 0
[2016-03-22 12:25:20.581431] D [MSGID: 0] [glusterd-store.c:464:glusterd_store_perform_brick_store] 0-management: Returning 0
[2016-03-22 12:25:20.581465] D [MSGID: 0] [glusterd-store.c:533:glusterd_store_brickinfo] 0-management: Returning with 0
[2016-03-22 12:25:20.581504] D [MSGID: 0] [glusterd-store.c:1325:glusterd_store_brickinfos] 0-management: Returning 0
[2016-03-22 12:25:20.581538] D [MSGID: 0] [glusterd-store.c:1439:glusterd_store_perform_volume_store] 0-management: Returning 0
[2016-03-22 12:25:20.616459] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:20.616758] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 3 times between [2016-03-22 12:25:20.616459] and [2016-03-22 12:25:20.616753]
[2016-03-22 12:25:20.616757] D [MSGID: 0] [glusterd-store.c:1383:glusterd_store_node_state_write] 0-management: Returning 0
[2016-03-22 12:25:20.633029] D [MSGID: 0] [glusterd-store.c:1411:glusterd_store_perform_node_state_store] 0-management: Returning 0
[2016-03-22 12:25:20.633812] D [MSGID: 0] [glusterd-utils.c:2052:glusterd_volume_compute_cksum] 0-management: Returning with 0
[2016-03-22 12:25:20.633852] D [MSGID: 0] [glusterd-store.c:1645:glusterd_store_volinfo] 0-management: Returning 0
[2016-03-22 12:25:20.633930] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already stopped
[2016-03-22 12:25:20.634611] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff79d1e430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff7678eab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff7678f8b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:25:20.634716] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:20.634752] D [MSGID: 0] [glusterd-nfs-svc.c:135:glusterd_nfssvc_stop] 0-management: Returning 0
[2016-03-22 12:25:20.635028] D [MSGID: 0] [glusterd-volgen.c:4809:build_nfs_graph] 0-glusterd: Returning 0
[2016-03-22 12:25:20.635555] D [MSGID: 0] [glusterd-nfs-svc.c:106:glusterd_nfssvc_manager] 0-management: Returning 0
[2016-03-22 12:25:20.635592] D [MSGID: 0] [glusterd-shd-svc.c:139:glusterd_shdsvc_manager] 0-management: Returning 0
[2016-03-22 12:25:20.635838] D [MSGID: 0] [glusterd-quotad-svc.c:111:glusterd_quotadsvc_manager] 0-management: Returning 0
[2016-03-22 12:25:20.635969] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: bitd already stopped
[2016-03-22 12:25:20.636387] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff79d1e430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff7678eab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff7678f8b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:25:20.636464] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:20.636545] D [MSGID: 0] [glusterd-bitd-svc.c:109:glusterd_bitdsvc_manager] 0-management: Returning 0
[2016-03-22 12:25:20.636666] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: scrub already stopped
[2016-03-22 12:25:20.637018] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff79d1e430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff7678eab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff7678f8b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:25:20.637095] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:20.637130] D [MSGID: 0] [glusterd-scrub-svc.c:104:glusterd_scrubsvc_manager] 0-management: Returning 0
[2016-03-22 12:25:20.637353] D [MSGID: 0] [glusterd-op-sm.c:5642:glusterd_op_commit_perform] 0-management: Returning 0
[2016-03-22 12:25:20.637504] D [MSGID: 0] [glusterd-handler.c:2511:glusterd_op_commit_send_resp] 0-management: Responded to commit, ret: 0
[2016-03-22 12:25:20.637540] D [MSGID: 0] [glusterd-op-sm.c:5341:glusterd_op_ac_commit_op] 0-management: Returning with 0
[2016-03-22 12:25:20.637575] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Brick op Committed' to 'Committed' due to event 'GD_OP_EVENT_ALL_ACK'
[2016-03-22 12:25:20.637611] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:20.637763] D [MSGID: 0] [glusterd-op-sm.c:311:glusterd_set_txn_opinfo] 0-management: Successfully set opinfo for transaction ID : 1d126f79-deb2-4cf1-bbd8-1651ee9183c3
[2016-03-22 12:25:20.637795] D [MSGID: 0] [glusterd-op-sm.c:318:glusterd_set_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.638529] I [MSGID: 106144] [glusterd-pmap.c:274:pmap_registry_remove] 0-pmap: removing brick /opt/lvmdir/c2/brick on port 49729
[2016-03-22 12:25:20.638733] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.1.144 
[2016-03-22 12:25:20.641088] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.1.144 " repeated 2 times between [2016-03-22 12:25:20.638733] and [2016-03-22 12:25:20.640435]
[2016-03-22 12:25:20.641086] D [MSGID: 0] [common-utils.c:3149:gf_is_local_addr] 0-management: 10.32.1.144 is not local
[2016-03-22 12:25:20.641264] W [socket.c:588:__socket_rwv] 0-socket.management: writev on 10.32.0.48:65531 failed (Connection reset by peer)
[2016-03-22 12:25:20.641346] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:20.641370] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:20.641462] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=5 from gen=7 to slot->gen=8, fd=15, slot->fd=15
[2016-03-22 12:25:20.641748] D [MSGID: 0] [glusterd-handshake.c:409:glusterd_get_args_from_dict] 0-management: No brick name present
[2016-03-22 12:25:20.641794] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning -1
[2016-03-22 12:25:20.641985] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 127.0.0.1 
[2016-03-22 12:25:20.642364] D [MSGID: 0] [glusterd-mgmt-handler.c:874:glusterd_handle_mgmt_v3_unlock_fn] 0-management: Received volume unlock req from uuid: 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:20.642412] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:20.642487] D [MSGID: 0] [glusterd-op-sm.c:7346:glusterd_op_sm_inject_event] 0-management: Enqueue event: 'GD_OP_EVENT_UNLOCK'
[2016-03-22 12:25:20.642521] D [MSGID: 0] [glusterd-op-sm.c:7428:glusterd_op_sm] 0-management: Dequeued event of type: 'GD_OP_EVENT_UNLOCK'
[2016-03-22 12:25:20.642555] D [MSGID: 0] [glusterd-op-sm.c:7431:glusterd_op_sm] 0-management: transaction ID = 1d126f79-deb2-4cf1-bbd8-1651ee9183c3
[2016-03-22 12:25:20.642595] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : 1d126f79-deb2-4cf1-bbd8-1651ee9183c3
[2016-03-22 12:25:20.642627] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.642675] D [MSGID: 0] [glusterd-locks.c:669:glusterd_mgmt_v3_unlock] 0-management: Trying to release lock of vol c_glusterfs for 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2 as c_glusterfs_vol
[2016-03-22 12:25:20.642719] D [MSGID: 0] [glusterd-locks.c:714:glusterd_mgmt_v3_unlock] 0-management: Lock for vol c_glusterfs successfully released
[2016-03-22 12:25:20.642834] D [MSGID: 0] [glusterd-handler.c:2359:glusterd_op_mgmt_v3_unlock_send_resp] 0-management: Responded to mgmt_v3 unlock, ret: 0
[2016-03-22 12:25:20.642891] D [MSGID: 0] [glusterd-op-sm.c:3740:glusterd_op_ac_unlock] 0-management: Unlock Returned 0
[2016-03-22 12:25:20.642943] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Committed' to 'Default' due to event 'GD_OP_EVENT_UNLOCK'
[2016-03-22 12:25:20.642978] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:20.643025] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : 1d126f79-deb2-4cf1-bbd8-1651ee9183c3
[2016-03-22 12:25:20.643063] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.643134] D [MSGID: 0] [glusterd-op-sm.c:360:glusterd_clear_txn_opinfo] 0-management: Successfully cleared opinfo for transaction ID : 1d126f79-deb2-4cf1-bbd8-1651ee9183c3
[2016-03-22 12:25:20.643169] D [MSGID: 0] [glusterd-op-sm.c:364:glusterd_clear_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.748224] I [MSGID: 106491] [glusterd-handler.c:2589:__glusterd_handle_incoming_unfriend_req] 0-glusterd: Received unfriend from uuid: 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:20.748682] E [MSGID: 101076] [common-utils.c:2954:gf_get_hostname_from_ip] 0-common-utils: Could not lookup hostname of 10.32.1.144 : Temporary failure in name resolution
[2016-03-22 12:25:20.748736] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:20.748772] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_RCVD_REMOVE_FRIEND'
[2016-03-22 12:25:20.748814] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:20.748843] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_RCVD_REMOVE_FRIEND'
[2016-03-22 12:25:20.748974] I [MSGID: 106493] [glusterd-handler.c:3747:glusterd_xfer_friend_remove_resp] 0-glusterd: Responded to 10.32.1.144 (0), ret: 0
[2016-03-22 12:25:20.749011] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_REMOVE_FRIEND'
[2016-03-22 12:25:20.749047] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management:  already stopped
[2016-03-22 12:25:20.749079] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:20.749107] D [MSGID: 0] [glusterd-utils.c:6763:glusterd_friend_contains_vol_bricks] 0-management: Returning 0
[2016-03-22 12:25:20.749135] I [MSGID: 106510] [glusterd-sm.c:637:glusterd_peer_detach_cleanup] 0-management: Deleting stale volume c_glusterfs
[2016-03-22 12:25:20.750050] D [MSGID: 0] [common-utils.c:3874:recursive_rmdir] 0-management: Removed quota.cksum
[2016-03-22 12:25:20.750243] D [MSGID: 0] [common-utils.c:3874:recursive_rmdir] 0-management: Removed run
[2016-03-22 12:25:20.750363] D [MSGID: 0] [common-utils.c:3874:recursive_rmdir] 0-management: Removed c_glusterfs.10.32.1.144.opt-lvmdir-c2-brick.vol
[2016-03-22 12:25:20.750478] D [MSGID: 0] [common-utils.c:3874:recursive_rmdir] 0-management: Removed info
[2016-03-22 12:25:20.750593] D [MSGID: 0] [common-utils.c:3874:recursive_rmdir] 0-management: Removed c_glusterfs.tcp-fuse.vol
[2016-03-22 12:25:20.750705] D [MSGID: 0] [common-utils.c:3874:recursive_rmdir] 0-management: Removed snapd.info
[2016-03-22 12:25:20.750793] D [MSGID: 0] [common-utils.c:3874:recursive_rmdir] 0-management: Removed quota.conf
[2016-03-22 12:25:20.750909] D [MSGID: 0] [common-utils.c:3874:recursive_rmdir] 0-management: Removed trusted-c_glusterfs.tcp-fuse.vol
[2016-03-22 12:25:20.751040] D [MSGID: 0] [common-utils.c:3874:recursive_rmdir] 0-management: Removed cksum
[2016-03-22 12:25:20.751202] D [MSGID: 0] [common-utils.c:3874:recursive_rmdir] 0-management: Removed 10.32.1.144:-opt-lvmdir-c2-brick
[2016-03-22 12:25:20.751321] D [MSGID: 0] [common-utils.c:3874:recursive_rmdir] 0-management: Removed bricks
[2016-03-22 12:25:20.751437] D [MSGID: 0] [common-utils.c:3874:recursive_rmdir] 0-management: Removed node_state.info
[2016-03-22 12:25:20.751578] D [MSGID: 0] [common-utils.c:3874:recursive_rmdir] 0-management: Removed 0992c376-e66e-4bc4-a0c7-2ca28f894aa2.deleted
[2016-03-22 12:25:20.751704] D [MSGID: 0] [store.c:460:gf_store_handle_destroy] 0-: Returning 0
[2016-03-22 12:25:20.751734] D [MSGID: 0] [glusterd-store.c:1709:glusterd_store_delete_volume] 0-management: Returning 0
[2016-03-22 12:25:20.751767] D [MSGID: 0] [glusterd-utils.c:815:glusterd_volume_brickinfos_delete] 0-management: Returning 0
[2016-03-22 12:25:20.751833] D [MSGID: 0] [store.c:460:gf_store_handle_destroy] 0-: Returning 0
[2016-03-22 12:25:20.751876] D [MSGID: 0] [glusterd-utils.c:859:glusterd_volinfo_delete] 0-management: Returning 0
[2016-03-22 12:25:20.751904] D [MSGID: 0] [glusterd-utils.c:6077:glusterd_delete_volume] 0-management: returning 0
[2016-03-22 12:25:20.751967] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already stopped
[2016-03-22 12:25:20.752593] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff79d1e430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff7678eab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff7678f8b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:25:20.752679] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:20.752884] D [MSGID: 0] [glusterd-proc-mgmt.c:87:glusterd_proc_stop] 0-management: Stopping glustershd daemon running in pid: 2544
[2016-03-22 12:25:20.759993] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:20.760054] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:20.760136] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=4 from gen=4 to slot->gen=5, fd=7, slot->fd=7
[2016-03-22 12:25:21.753704] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff79d1e430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff7678eab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff7678f8b8] ))))) 0-: /var/run/gluster/517bcdf9740ac943aa9615504f526f12.socket: ping timer event already removed
[2016-03-22 12:25:21.753796] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:21.753846] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: quotad already stopped
[2016-03-22 12:25:21.754166] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff79d1e430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff7678eab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff7678f8b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:25:21.754227] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:21.754267] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: bitd already stopped
[2016-03-22 12:25:21.754564] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff79d1e430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff7678eab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff7678f8b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:25:21.754621] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:21.754660] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: scrub already stopped
[2016-03-22 12:25:21.754965] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff79d1e430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff7678eab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff7678f8b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:25:21.755032] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:21.755062] D [MSGID: 0] [glusterd-sm.c:713:glusterd_ac_handle_friend_remove_req] 0-management: Returning with 0
[2016-03-22 12:25:21.755097] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:21.755130] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Peer in Cluster' to 'Establishing Connection' due to event 'GD_FRIEND_EVENT_RCVD_REMOVE_FRIEND'
[2016-03-22 12:25:21.755158] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:21.755186] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Establishing Connection
[2016-03-22 12:25:21.755226] D [MSGID: 0] [glusterd-store.c:3802:glusterd_store_create_peer_dir] 0-glusterd: Returning with 0
[2016-03-22 12:25:21.755443] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:21.755599] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 2 times between [2016-03-22 12:25:21.755443] and [2016-03-22 12:25:21.755595]
[2016-03-22 12:25:21.755598] D [MSGID: 0] [glusterd-store.c:3931:glusterd_store_peer_write] 0-glusterd: Returning with 0
[2016-03-22 12:25:21.893560] D [MSGID: 0] [glusterd-store.c:3957:glusterd_store_perform_peer_store] 0-glusterd: Returning 0
[2016-03-22 12:25:21.893627] D [MSGID: 0] [glusterd-store.c:3978:glusterd_store_peerinfo] 0-glusterd: Returning with 0
[2016-03-22 12:25:21.893668] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Establishing Connection
[2016-03-22 12:25:21.893698] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_REMOVE_FRIEND'
[2016-03-22 12:25:21.893727] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Establishing Connection
[2016-03-22 12:25:21.893756] D [MSGID: 0] [glusterd-utils.c:6799:glusterd_friend_remove_cleanup_vols] 0-management: Returning 0
[2016-03-22 12:25:21.893936] D [MSGID: 0] [glusterd-utils.c:173:glusterd_lock] 0-management: Cluster lock held by 356dabed-4314-48be-9eef-d2711d01270d
[2016-03-22 12:25:21.894215] D [MSGID: 0] [store.c:460:gf_store_handle_destroy] 0-: Returning 0
[2016-03-22 12:25:21.894272] D [MSGID: 0] [glusterd-store.c:3777:glusterd_store_delete_peerinfo] 0-management: Returning with 0
[2016-03-22 12:25:21.894832] I [MSGID: 106163] [glusterd-handshake.c:1193:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30706
[2016-03-22 12:25:21.895101] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:21.910738] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:25:21.895191] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:21.910737] I [MSGID: 106490] [glusterd-handler.c:2884:__glusterd_handle_probe_query] 0-glusterd: Received probe from uuid: 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:21.911186] E [MSGID: 101076] [common-utils.c:2954:gf_get_hostname_from_ip] 0-common-utils: Could not lookup hostname of 10.32.1.144 : Temporary failure in name resolution
[2016-03-22 12:25:21.911230] D [MSGID: 0] [glusterd-peer-utils.c:209:glusterd_peerinfo_find_by_uuid] 0-management: Friend with uuid: 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2, not found
[2016-03-22 12:25:21.911262] D [MSGID: 0] [glusterd-peer-utils.c:235:glusterd_peerinfo_find] 0-management: Unable to find peer by uuid: 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:21.911408] D [MSGID: 0] [glusterd-peer-utils.c:133:glusterd_peerinfo_find_by_hostname] 0-management: Unable to find friend: 10.32.1.144
[2016-03-22 12:25:21.911441] D [MSGID: 0] [glusterd-peer-utils.c:247:glusterd_peerinfo_find] 0-management: Unable to find hostname: 10.32.1.144
[2016-03-22 12:25:21.911469] I [MSGID: 106129] [glusterd-handler.c:2919:__glusterd_handle_probe_query] 0-glusterd: Unable to find peerinfo for host: 10.32.1.144 (24007)
[2016-03-22 12:25:21.911503] D [MSGID: 0] [glusterd-peer-utils.c:458:glusterd_peer_hostname_new] 0-glusterd: Returning 0
[2016-03-22 12:25:21.911536] D [MSGID: 0] [glusterd-utils.c:5974:glusterd_sm_tr_log_init] 0-glusterd: returning 0
[2016-03-22 12:25:21.911578] D [MSGID: 0] [glusterd-store.c:3802:glusterd_store_create_peer_dir] 0-glusterd: Returning with 0
[2016-03-22 12:25:21.918638] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:25:21.918962] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:21.919123] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 2 times between [2016-03-22 12:25:21.918962] and [2016-03-22 12:25:21.919117]
[2016-03-22 12:25:21.919122] D [MSGID: 0] [glusterd-store.c:3931:glusterd_store_peer_write] 0-glusterd: Returning with 0
[2016-03-22 12:25:21.933901] D [MSGID: 0] [glusterd-store.c:3957:glusterd_store_perform_peer_store] 0-glusterd: Returning 0
[2016-03-22 12:25:21.933979] D [MSGID: 0] [glusterd-store.c:3978:glusterd_store_peerinfo] 0-glusterd: Returning with 0
[2016-03-22 12:25:21.934098] D [MSGID: 0] [glusterd-handler.c:3391:glusterd_transport_inet_options_build] 0-glusterd: Returning 0
[2016-03-22 12:25:21.934150] I [rpc-clnt.c:984:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2016-03-22 12:25:21.934173] D [rpc-clnt.c:996:rpc_clnt_connection_init] 0-management: setting ping-timeout to 30
[2016-03-22 12:25:21.934203] D [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3.7.6/rpc-transport/socket.so
[2016-03-22 12:25:21.934334] D [socket.c:3845:socket_init] 0-management: Configued transport.tcp-user-timeout=-1
[2016-03-22 12:25:21.934363] D [socket.c:3928:socket_init] 0-management: SSL support on the I/O path is NOT enabled
[2016-03-22 12:25:21.934382] D [socket.c:3931:socket_init] 0-management: SSL support for glusterd is NOT enabled
[2016-03-22 12:25:21.934401] D [socket.c:3948:socket_init] 0-management: using system polling thread
[2016-03-22 12:25:21.935054] D [MSGID: 0] [common-utils.c:333:gf_resolve_ip6] 0-resolver: returning ip-10.32.1.144 (port-24007) for hostname: 10.32.1.144 and port: 24007
[2016-03-22 12:25:21.935089] D [socket.c:2833:socket_fix_ssl_opts] 0-management: disabling SSL for portmapper connection
[2016-03-22 12:25:21.935146] W [socket.c:869:__socket_keepalive] 0-socket: failed to set TCP_USER_TIMEOUT -1000 on socket 8, Invalid argument
[2016-03-22 12:25:21.935169] E [socket.c:2965:socket_connect] 0-management: Failed to set keep-alive: Invalid argument
[2016-03-22 12:25:21.935553] D [MSGID: 0] [glusterd-handler.c:3323:glusterd_rpc_create] 0-management: returning 0
[2016-03-22 12:25:21.935585] I [MSGID: 106498] [glusterd-handler.c:3528:glusterd_friend_add] 0-management: connect returned 0
[2016-03-22 12:25:21.935692] I [MSGID: 106493] [glusterd-handler.c:2947:__glusterd_handle_probe_query] 0-glusterd: Responded to 10.32.1.144, op_ret: 0, op_errno: 0, ret: 0
[2016-03-22 12:25:21.935839] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Probe Received from peer
[2016-03-22 12:25:21.935872] D [MSGID: 0] [glusterd-handler.c:5042:__glusterd_peer_rpc_notify] 0-management: got RPC_CLNT_CONNECT
[2016-03-22 12:25:21.935928] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Probe Received from peer
[2016-03-22 12:25:21.936545] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(+0x1a188)[0x3fff79d23188] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_submit-0x1a4d8)[0x3fff79d1e0a8] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_submit_request-0x129078)[0x3fff766c1de8] ))))) 0-: 10.32.1.144:24007: ping timer event already removed
[2016-03-22 12:25:21.936700] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:25:21.936686] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Probe Received from peer
[2016-03-22 12:25:21.936700] D [MSGID: 0] [glusterd-handshake.c:2098:__glusterd_peer_dump_version_cbk] 0-management: Proceeding to op-version handshake with peer 10.32.1.144
[2016-03-22 12:25:21.936793] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Probe Received from peer
[2016-03-22 12:25:21.937056] I [MSGID: 106490] [glusterd-handler.c:2539:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:21.937393] E [MSGID: 101076] [common-utils.c:2954:gf_get_hostname_from_ip] 0-common-utils: Could not lookup hostname of 10.32.1.144 : Temporary failure in name resolution
[2016-03-22 12:25:21.937437] D [MSGID: 0] [glusterd-peer-utils.c:209:glusterd_peerinfo_find_by_uuid] 0-management: Friend with uuid: 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2, not found
[2016-03-22 12:25:21.937469] D [MSGID: 0] [glusterd-peer-utils.c:235:glusterd_peerinfo_find] 0-management: Unable to find peer by uuid: 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:21.937499] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.1.144 found.. state: 8
[2016-03-22 12:25:21.937694] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_RCVD_FRIEND_REQ'
[2016-03-22 12:25:21.937730] D [MSGID: 0] [glusterd-peer-utils.c:235:glusterd_peerinfo_find] 0-management: Unable to find peer by uuid: 00000000-0000-0000-0000-000000000000
[2016-03-22 12:25:21.937760] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.1.144 found.. state: 8
[2016-03-22 12:25:21.937790] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_RCVD_FRIEND_REQ'
[2016-03-22 12:25:21.937823] D [MSGID: 0] [glusterd-peer-utils.c:235:glusterd_peerinfo_find] 0-management: Unable to find peer by uuid: 00000000-0000-0000-0000-000000000000
[2016-03-22 12:25:21.937853] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.1.144 found.. state: 8
[2016-03-22 12:25:21.937900] D [MSGID: 0] [glusterd-utils.c:2830:import_prdict_dict] 0-glusterd: Returning with 0
[2016-03-22 12:25:21.937959] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning -1
[2016-03-22 12:25:21.937989] D [MSGID: 0] [glusterd-utils.c:2776:glusterd_compare_friend_volume] 0-management: Returning with ret: 0, status: 2
[2016-03-22 12:25:21.938034] D [MSGID: 0] [glusterd-utils.c:512:glusterd_volinfo_new] 0-management: Returning 0
[2016-03-22 12:25:21.938124] D [MSGID: 0] [glusterd-utils.c:3197:gd_import_friend_volume_rebal_dict] 0-management: Returning with 0
[2016-03-22 12:25:21.938207] D [MSGID: 0] [glusterd-utils.c:2830:import_prdict_dict] 0-glusterd: Returning with 0
[2016-03-22 12:25:21.938245] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:25:21.938242] D [MSGID: 0] [glusterd-utils.c:2830:import_prdict_dict] 0-glusterd: Returning with 0
[2016-03-22 12:25:21.938244] D [MSGID: 0] [glusterd-utils.c:2909:glusterd_import_friend_volume_opts] 0-glusterd: Returning with 0
[2016-03-22 12:25:21.938347] D [MSGID: 0] [glusterd-utils.c:884:glusterd_brickinfo_new] 0-management: Returning 0
[2016-03-22 12:25:21.938393] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.1.144 found.. state: 8
[2016-03-22 12:25:21.938422] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:21.938448] D [MSGID: 0] [glusterd-utils.c:925:glusterd_resolve_brick] 0-management: Returning 0
[2016-03-22 12:25:21.938475] D [MSGID: 0] [glusterd-utils.c:2997:glusterd_import_new_brick] 0-glusterd: Returning with 0
[2016-03-22 12:25:21.938501] D [MSGID: 0] [glusterd-utils.c:3037:glusterd_import_bricks] 0-glusterd: Returning with 0
[2016-03-22 12:25:21.938527] D [MSGID: 0] [glusterd-utils.c:3593:glusterd_import_volinfo] 0-glusterd: Returning with 0
[2016-03-22 12:25:21.938553] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning -1
[2016-03-22 12:25:21.938581] D [MSGID: 0] [glusterd-utils.c:4539:glusterd_brick_start] 0-management: returning 0 
[2016-03-22 12:25:21.938610] D [MSGID: 0] [glusterd-snapshot-utils.c:3595:glusterd_is_snapd_enabled] 0-management: Key features.uss not present in the dict for volume c_glusterfs
[2016-03-22 12:25:21.938793] D [MSGID: 0] [glusterd-store.c:1058:glusterd_store_create_volume_dir] 0-management: Returning with 0
[2016-03-22 12:25:21.944936] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:25:21.951964] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:25:21.951749] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:25:21.951961] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:21.959888] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 18 times between [2016-03-22 12:25:21.951961] and [2016-03-22 12:25:21.953094]
[2016-03-22 12:25:21.959886] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:25:21.960193] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:21.960225] D [MSGID: 0] [glusterd-store.c:440:glusterd_store_snapd_write] 0-management: Returning 0
[2016-03-22 12:25:21.974334] D [MSGID: 0] [glusterd-store.c:503:glusterd_store_perform_snapd_store] 0-management: Returning 0
[2016-03-22 12:25:21.974400] D [MSGID: 0] [glusterd-store.c:568:glusterd_store_snapd_info] 0-management: Returning with 0
[2016-03-22 12:25:21.974466] D [MSGID: 0] [glusterd-store.c:753:_storeopts] 0-management: Storing in volinfo:key= performance.readdir-ahead, val=on
[2016-03-22 12:25:21.974609] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:21.974646] D [MSGID: 0] [glusterd-store.c:753:_storeopts] 0-management: Storing in volinfo:key= network.ping-timeout, val=4
[2016-03-22 12:25:21.974726] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:21.974777] D [MSGID: 0] [glusterd-store.c:753:_storeopts] 0-management: Storing in volinfo:key= nfs.disable, val=on
[2016-03-22 12:25:21.974857] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:21.974888] D [MSGID: 0] [glusterd-store.c:1104:glusterd_store_volinfo_write] 0-management: Returning 0
[2016-03-22 12:25:21.974989] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:21.982025] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:25:21.982293] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:21.982724] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 7 times between [2016-03-22 12:25:21.982293] and [2016-03-22 12:25:21.982719]
[2016-03-22 12:25:21.982722] D [MSGID: 0] [glusterd-store.c:414:glusterd_store_brickinfo_write] 0-management: Returning 0
[2016-03-22 12:25:21.982799] D [MSGID: 0] [glusterd-store.c:464:glusterd_store_perform_brick_store] 0-management: Returning 0
[2016-03-22 12:25:21.982826] D [MSGID: 0] [glusterd-store.c:533:glusterd_store_brickinfo] 0-management: Returning with 0
[2016-03-22 12:25:21.982853] D [MSGID: 0] [glusterd-store.c:1325:glusterd_store_brickinfos] 0-management: Returning 0
[2016-03-22 12:25:21.982880] D [MSGID: 0] [glusterd-store.c:1439:glusterd_store_perform_volume_store] 0-management: Returning 0
[2016-03-22 12:25:22.011019] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:22.011250] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 3 times between [2016-03-22 12:25:22.011019] and [2016-03-22 12:25:22.011245]
[2016-03-22 12:25:22.011249] D [MSGID: 0] [glusterd-store.c:1383:glusterd_store_node_state_write] 0-management: Returning 0
[2016-03-22 12:25:22.024845] D [MSGID: 0] [glusterd-store.c:1411:glusterd_store_perform_node_state_store] 0-management: Returning 0
[2016-03-22 12:25:22.025354] D [MSGID: 0] [glusterd-utils.c:2052:glusterd_volume_compute_cksum] 0-management: Returning with 0
[2016-03-22 12:25:22.025387] D [MSGID: 0] [glusterd-store.c:1645:glusterd_store_volinfo] 0-management: Returning 0
[2016-03-22 12:25:22.025554] D [MSGID: 0] [glusterd-volgen.c:5109:generate_brick_volfiles] 0-management: Found a brick - 10.32.1.144:/opt/lvmdir/c2/brick
[2016-03-22 12:25:22.026253] D [MSGID: 0] [glusterd-volgen.c:1268:server_check_marker_off] 0-glusterd: Returning 0
[2016-03-22 12:25:22.027109] D [MSGID: 0] [glusterd-volgen.c:5120:generate_brick_volfiles] 0-management: Returning 0
[2016-03-22 12:25:22.027218] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.1.144 found.. state: 8
[2016-03-22 12:25:22.027249] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:22.027279] D [MSGID: 0] [glusterd-utils.c:1326:glusterd_volume_brickinfo_get] 0-management: Found brick 10.32.1.144:/opt/lvmdir/c2/brick in volume c_glusterfs
[2016-03-22 12:25:22.027306] D [MSGID: 0] [glusterd-utils.c:1335:glusterd_volume_brickinfo_get] 0-management: Returning 0
[2016-03-22 12:25:22.028759] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.1.144 found.. state: 8
[2016-03-22 12:25:22.028792] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:22.028821] D [MSGID: 0] [glusterd-utils.c:1326:glusterd_volume_brickinfo_get] 0-management: Found brick 10.32.1.144:/opt/lvmdir/c2/brick in volume c_glusterfs
[2016-03-22 12:25:22.028848] D [MSGID: 0] [glusterd-utils.c:1335:glusterd_volume_brickinfo_get] 0-management: Returning 0
[2016-03-22 12:25:22.030349] D [MSGID: 0] [store.c:460:gf_store_handle_destroy] 0-: Returning 0
[2016-03-22 12:25:22.030385] D [MSGID: 0] [glusterd-utils.c:3902:glusterd_import_friend_volume] 0-glusterd: Returning with ret: 0
[2016-03-22 12:25:22.030413] D [MSGID: 0] [glusterd-utils.c:3927:glusterd_import_friend_volumes] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.030460] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already stopped
[2016-03-22 12:25:22.031074] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff79d1e430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff7678eab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff7678f8b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:25:22.031144] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:22.031174] D [MSGID: 0] [glusterd-nfs-svc.c:135:glusterd_nfssvc_stop] 0-management: Returning 0
[2016-03-22 12:25:22.031325] D [MSGID: 0] [glusterd-volgen.c:4809:build_nfs_graph] 0-glusterd: Returning 0
[2016-03-22 12:25:22.031787] D [MSGID: 0] [glusterd-nfs-svc.c:106:glusterd_nfssvc_manager] 0-management: Returning 0
[2016-03-22 12:25:22.031831] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: glustershd already stopped
[2016-03-22 12:25:22.032200] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff79d1e430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff7678eab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff7678f8b8] ))))) 0-: /var/run/gluster/517bcdf9740ac943aa9615504f526f12.socket: ping timer event already removed
[2016-03-22 12:25:22.032258] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:22.032288] D [MSGID: 0] [glusterd-shd-svc.c:139:glusterd_shdsvc_manager] 0-management: Returning 0
[2016-03-22 12:25:22.032409] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: quotad already stopped
[2016-03-22 12:25:22.032707] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff79d1e430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff7678eab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff7678f8b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:25:22.032762] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:22.032791] D [MSGID: 0] [glusterd-quotad-svc.c:111:glusterd_quotadsvc_manager] 0-management: Returning 0
[2016-03-22 12:25:22.032906] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: bitd already stopped
[2016-03-22 12:25:22.033218] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff79d1e430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff7678eab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff7678f8b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:25:22.033274] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:22.033304] D [MSGID: 0] [glusterd-bitd-svc.c:109:glusterd_bitdsvc_manager] 0-management: Returning 0
[2016-03-22 12:25:22.033418] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: scrub already stopped
[2016-03-22 12:25:22.033717] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff79d1e430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff7678eab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff7678f8b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:25:22.033775] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:22.033804] D [MSGID: 0] [glusterd-scrub-svc.c:104:glusterd_scrubsvc_manager] 0-management: Returning 0
[2016-03-22 12:25:22.033833] D [MSGID: 0] [glusterd-utils.c:4078:glusterd_compare_friend_data] 0-management: Returning with ret: 0, status: 2
[2016-03-22 12:25:22.046760] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_LOCAL_ACC'
[2016-03-22 12:25:22.046944] I [MSGID: 106493] [glusterd-handler.c:3780:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 10.32.1.144 (0), ret: 0
[2016-03-22 12:25:22.046976] D [MSGID: 0] [glusterd-sm.c:911:glusterd_ac_handle_friend_add_req] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.047015] D [MSGID: 0] [glusterd-peer-utils.c:235:glusterd_peerinfo_find] 0-management: Unable to find peer by uuid: 00000000-0000-0000-0000-000000000000
[2016-03-22 12:25:22.047046] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.1.144 found.. state: 8
[2016-03-22 12:25:22.047080] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Probe Received from peer' to 'Probe Received from Peer' due to event 'GD_FRIEND_EVENT_RCVD_FRIEND_REQ'
[2016-03-22 12:25:22.047109] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:22.047140] D [MSGID: 0] [glusterd-peer-utils.c:235:glusterd_peerinfo_find] 0-management: Unable to find peer by uuid: 00000000-0000-0000-0000-000000000000
[2016-03-22 12:25:22.047169] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.1.144 found.. state: 2
[2016-03-22 12:25:22.047216] D [MSGID: 0] [glusterd-store.c:3802:glusterd_store_create_peer_dir] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.047262] D [MSGID: 0] [store.c:460:gf_store_handle_destroy] 0-: Returning 0
[2016-03-22 12:25:22.053112] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:25:22.053374] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:22.053528] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 2 times between [2016-03-22 12:25:22.053374] and [2016-03-22 12:25:22.053522]
[2016-03-22 12:25:22.053526] D [MSGID: 0] [glusterd-store.c:3931:glusterd_store_peer_write] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.067795] D [MSGID: 0] [glusterd-store.c:3957:glusterd_store_perform_peer_store] 0-glusterd: Returning 0
[2016-03-22 12:25:22.067852] D [MSGID: 0] [glusterd-store.c:3978:glusterd_store_peerinfo] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.068035] D [MSGID: 0] [glusterd-peer-utils.c:235:glusterd_peerinfo_find] 0-management: Unable to find peer by uuid: 00000000-0000-0000-0000-000000000000
[2016-03-22 12:25:22.068068] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.1.144 found.. state: 2
[2016-03-22 12:25:22.068098] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_LOCAL_ACC'
[2016-03-22 12:25:22.068126] D [MSGID: 0] [glusterd-sm.c:203:glusterd_ac_none] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.068169] D [MSGID: 0] [glusterd-peer-utils.c:235:glusterd_peerinfo_find] 0-management: Unable to find peer by uuid: 00000000-0000-0000-0000-000000000000
[2016-03-22 12:25:22.068198] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.1.144 found.. state: 2
[2016-03-22 12:25:22.068231] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Probe Received from Peer' to 'Accepted peer request' due to event 'GD_FRIEND_EVENT_LOCAL_ACC'
[2016-03-22 12:25:22.068258] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:22.068290] D [MSGID: 0] [glusterd-peer-utils.c:235:glusterd_peerinfo_find] 0-management: Unable to find peer by uuid: 00000000-0000-0000-0000-000000000000
[2016-03-22 12:25:22.068318] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.1.144 found.. state: 4
[2016-03-22 12:25:22.068360] D [MSGID: 0] [glusterd-store.c:3802:glusterd_store_create_peer_dir] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.068602] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:22.068753] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 2 times between [2016-03-22 12:25:22.068602] and [2016-03-22 12:25:22.068748]
[2016-03-22 12:25:22.068751] D [MSGID: 0] [glusterd-store.c:3931:glusterd_store_peer_write] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.083444] D [MSGID: 0] [glusterd-store.c:3957:glusterd_store_perform_peer_store] 0-glusterd: Returning 0
[2016-03-22 12:25:22.083508] D [MSGID: 0] [glusterd-store.c:3978:glusterd_store_peerinfo] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.083589] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Accepted peer request
[2016-03-22 12:25:22.083653] D [MSGID: 0] [glusterd-handshake.c:1660:gd_validate_peer_op_version] 0-management: Peer 10.32.1.144 accepted
[2016-03-22 12:25:22.106554] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Accepted peer request
[2016-03-22 12:25:22.106635] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:25:22.106627] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Accepted peer request
[2016-03-22 12:25:22.106634] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_CONNECTED'
[2016-03-22 12:25:22.106704] D [MSGID: 0] [glusterd-handshake.c:1613:glusterd_event_connected_inject] 0-glusterd: returning 0
[2016-03-22 12:25:22.106739] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Accepted peer request
[2016-03-22 12:25:22.106768] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_CONNECTED'
[2016-03-22 12:25:22.106796] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Accepted peer request
[2016-03-22 12:25:22.106828] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_PROBE'
[2016-03-22 12:25:22.106854] D [MSGID: 0] [glusterd-sm.c:287:glusterd_ac_reverse_probe_begin] 0-glusterd: returning with 0
[2016-03-22 12:25:22.106882] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Accepted peer request
[2016-03-22 12:25:22.106923] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Accepted peer request' to 'Peer is connected and Accepted' due to event 'GD_FRIEND_EVENT_CONNECTED'
[2016-03-22 12:25:22.106966] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:22.106995] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer is connected and Accepted
[2016-03-22 12:25:22.107038] D [MSGID: 0] [glusterd-store.c:3802:glusterd_store_create_peer_dir] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.107304] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:22.107455] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 2 times between [2016-03-22 12:25:22.107304] and [2016-03-22 12:25:22.107450]
[2016-03-22 12:25:22.107454] D [MSGID: 0] [glusterd-store.c:3931:glusterd_store_peer_write] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.121627] D [MSGID: 0] [glusterd-store.c:3957:glusterd_store_perform_peer_store] 0-glusterd: Returning 0
[2016-03-22 12:25:22.121690] D [MSGID: 0] [glusterd-store.c:3978:glusterd_store_peerinfo] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.121729] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer is connected and Accepted
[2016-03-22 12:25:22.121758] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_PROBE'
[2016-03-22 12:25:22.121789] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.1.144 found.. state: 10
[2016-03-22 12:25:22.121972] D [MSGID: 0] [glusterd-rpc-ops.c:1477:glusterd_rpc_probe] 0-glusterd: Returning 0
[2016-03-22 12:25:22.122007] D [MSGID: 0] [glusterd-sm.c:413:glusterd_ac_friend_probe] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.122035] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer is connected and Accepted
[2016-03-22 12:25:22.122067] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Peer is connected and Accepted' to 'Peer is connected and Accepted' due to event 'GD_FRIEND_EVENT_PROBE'
[2016-03-22 12:25:22.122095] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:22.122123] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer is connected and Accepted
[2016-03-22 12:25:22.122165] D [MSGID: 0] [glusterd-store.c:3802:glusterd_store_create_peer_dir] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.122410] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:22.122567] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 2 times between [2016-03-22 12:25:22.122410] and [2016-03-22 12:25:22.122562]
[2016-03-22 12:25:22.122566] D [MSGID: 0] [glusterd-store.c:3931:glusterd_store_peer_write] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.138002] D [MSGID: 0] [glusterd-store.c:3957:glusterd_store_perform_peer_store] 0-glusterd: Returning 0
[2016-03-22 12:25:22.138067] D [MSGID: 0] [glusterd-store.c:3978:glusterd_store_peerinfo] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.138399] I [MSGID: 106511] [glusterd-rpc-ops.c:256:__glusterd_probe_cbk] 0-management: Received probe resp from uuid: 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2, host: 10.32.1.144
[2016-03-22 12:25:22.138499] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer is connected and Accepted
[2016-03-22 12:25:22.138536] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_INIT_FRIEND_REQ'
[2016-03-22 12:25:22.138646] I [MSGID: 106511] [glusterd-rpc-ops.c:416:__glusterd_probe_cbk] 0-glusterd: Received resp to probe req
[2016-03-22 12:25:22.138681] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer is connected and Accepted
[2016-03-22 12:25:22.138777] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_INIT_FRIEND_REQ'
[2016-03-22 12:25:22.138806] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer is connected and Accepted
[2016-03-22 12:25:22.139548] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:25:22.138904] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer is connected and Accepted
[2016-03-22 12:25:22.139547] D [MSGID: 0] [glusterd-utils.c:2504:glusterd_add_volume_to_dict] 0-management: Returning with 0
[2016-03-22 12:25:22.140509] D [MSGID: 0] [glusterd-rpc-ops.c:1579:glusterd_rpc_friend_add] 0-glusterd: Returning 0
[2016-03-22 12:25:22.140608] D [MSGID: 0] [glusterd-sm.c:334:glusterd_ac_friend_add] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.140638] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer is connected and Accepted
[2016-03-22 12:25:22.140743] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Peer is connected and Accepted' to 'Sent and Received peer request' due to event 'GD_FRIEND_EVENT_INIT_FRIEND_REQ'
[2016-03-22 12:25:22.140846] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:22.140875] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Sent and Received peer request
[2016-03-22 12:25:22.141005] D [MSGID: 0] [glusterd-store.c:3802:glusterd_store_create_peer_dir] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.141623] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:22.142016] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 2 times between [2016-03-22 12:25:22.141623] and [2016-03-22 12:25:22.142011]
[2016-03-22 12:25:22.142015] D [MSGID: 0] [glusterd-store.c:3931:glusterd_store_peer_write] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.156745] D [MSGID: 0] [glusterd-store.c:3957:glusterd_store_perform_peer_store] 0-glusterd: Returning 0
[2016-03-22 12:25:22.156808] D [MSGID: 0] [glusterd-store.c:3978:glusterd_store_peerinfo] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.157041] I [MSGID: 106493] [glusterd-rpc-ops.c:480:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2, host: 10.32.1.144, port: 0
[2016-03-22 12:25:22.157145] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Sent and Received peer request
[2016-03-22 12:25:22.157181] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_RCVD_ACC'
[2016-03-22 12:25:22.157279] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Sent and Received peer request
[2016-03-22 12:25:22.157308] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_RCVD_ACC'
[2016-03-22 12:25:22.157408] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Sent and Received peer request
[2016-03-22 12:25:22.157550] D [MSGID: 0] [glusterd-peer-utils.c:615:gd_add_friend_to_dict] 0-management: Returning 0
[2016-03-22 12:25:22.157826] D [MSGID: 0] [glusterd-rpc-ops.c:1674:glusterd_rpc_friend_update] 0-glusterd: Returning 0
[2016-03-22 12:25:22.157856] D [MSGID: 0] [glusterd-sm.c:590:glusterd_ac_send_friend_update] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.157976] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Sent and Received peer request
[2016-03-22 12:25:22.158010] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Sent and Received peer request' to 'Peer in Cluster' due to event 'GD_FRIEND_EVENT_RCVD_ACC'
[2016-03-22 12:25:22.158107] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:22.158137] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:22.158250] D [MSGID: 0] [glusterd-store.c:3802:glusterd_store_create_peer_dir] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.159259] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:22.159433] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 2 times between [2016-03-22 12:25:22.159259] and [2016-03-22 12:25:22.159427]
[2016-03-22 12:25:22.159432] D [MSGID: 0] [glusterd-store.c:3931:glusterd_store_peer_write] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.174452] D [MSGID: 0] [glusterd-store.c:3957:glusterd_store_perform_peer_store] 0-glusterd: Returning 0
[2016-03-22 12:25:22.174510] D [MSGID: 0] [glusterd-store.c:3978:glusterd_store_peerinfo] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.174583] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:22.174642] I [MSGID: 106492] [glusterd-handler.c:2715:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:22.175458] I [MSGID: 106502] [glusterd-handler.c:2760:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2016-03-22 12:25:22.175628] D [MSGID: 0] [glusterd-snapshot-utils.c:3595:glusterd_is_snapd_enabled] 0-management: Key features.uss not present in the dict for volume c_glusterfs
[2016-03-22 12:25:22.175728] D [MSGID: 0] [glusterd-utils.c:173:glusterd_lock] 0-management: Cluster lock held by 356dabed-4314-48be-9eef-d2711d01270d
[2016-03-22 12:25:22.192400] I [MSGID: 106493] [glusterd-rpc-ops.c:694:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:23.210623] D [MSGID: 0] [glusterd-mgmt-handler.c:151:glusterd_handle_mgmt_v3_lock_fn] 0-management: Received mgmt_v3 lock req from uuid: 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:23.210971] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:23.211464] D [MSGID: 0] [glusterd-locks.c:562:glusterd_mgmt_v3_lock] 0-management: Trying to acquire lock of vol c_glusterfs for 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2 as c_glusterfs_vol
[2016-03-22 12:25:23.214711] D [MSGID: 0] [glusterd-locks.c:618:glusterd_mgmt_v3_lock] 0-management: Lock for vol c_glusterfs successfully held by 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:23.214942] D [MSGID: 0] [glusterd-locks.c:512:glusterd_multiple_mgmt_v3_lock] 0-management: Returning 0
[2016-03-22 12:25:23.215659] D [MSGID: 0] [glusterd-mgmt-handler.c:53:glusterd_mgmt_v3_lock_send_resp] 0-management: Responded to mgmt_v3 lock, ret: 0
[2016-03-22 12:25:23.228046] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:23.228458] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:23.228616] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:23.228974] D [MSGID: 0] [glusterd-utils.c:884:glusterd_brickinfo_new] 0-management: Returning 0
[2016-03-22 12:25:23.229203] D [MSGID: 0] [glusterd-utils.c:1041:glusterd_brickinfo_new_from_brick] 0-management: Returning 0
[2016-03-22 12:25:23.229497] D [MSGID: 0] [glusterd-utils.c:512:glusterd_volinfo_new] 0-management: Returning 0
[2016-03-22 12:25:23.229619] D [MSGID: 0] [glusterd-utils.c:815:glusterd_volume_brickinfos_delete] 0-management: Returning 0
[2016-03-22 12:25:23.229809] D [MSGID: 0] [store.c:460:gf_store_handle_destroy] 0-: Returning 0
[2016-03-22 12:25:23.229975] D [MSGID: 0] [glusterd-utils.c:859:glusterd_volinfo_delete] 0-management: Returning 0
[2016-03-22 12:25:23.230231] D [MSGID: 0] [glusterd-utils.c:884:glusterd_brickinfo_new] 0-management: Returning 0
[2016-03-22 12:25:23.230388] D [MSGID: 0] [glusterd-utils.c:1041:glusterd_brickinfo_new_from_brick] 0-management: Returning 0
[2016-03-22 12:25:23.230663] D [MSGID: 0] [glusterd-utils.c:512:glusterd_volinfo_new] 0-management: Returning 0
[2016-03-22 12:25:23.230901] D [MSGID: 0] [glusterd-utils.c:815:glusterd_volume_brickinfos_delete] 0-management: Returning 0
[2016-03-22 12:25:23.231117] D [MSGID: 0] [store.c:460:gf_store_handle_destroy] 0-: Returning 0
[2016-03-22 12:25:23.231228] D [MSGID: 0] [glusterd-utils.c:859:glusterd_volinfo_delete] 0-management: Returning 0
[2016-03-22 12:25:23.231378] D [MSGID: 0] [glusterd-utils.c:884:glusterd_brickinfo_new] 0-management: Returning 0
[2016-03-22 12:25:23.231573] D [MSGID: 0] [glusterd-utils.c:1041:glusterd_brickinfo_new_from_brick] 0-management: Returning 0
[2016-03-22 12:25:23.233209] D [MSGID: 0] [glusterd-peer-utils.c:133:glusterd_peerinfo_find_by_hostname] 0-management: Unable to find friend: 10.32.0.48
[2016-03-22 12:25:23.233969] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.0.48 
[2016-03-22 12:25:23.237294] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.0.48 is local address at interface efd
[2016-03-22 12:25:23.237595] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:23.237667] D [MSGID: 0] [glusterd-utils.c:925:glusterd_resolve_brick] 0-management: Returning 0
[2016-03-22 12:25:23.237779] D [MSGID: 0] [glusterd-utils.c:5656:glusterd_new_brick_validate] 0-management: returning 0 
[2016-03-22 12:25:23.238044] E [MSGID: 106451] [glusterd-utils.c:5798:glusterd_is_path_in_use] 0-management: /opt/lvmdir/c2/brick is already part of a volume [File exists]
[2016-03-22 12:25:23.238565] D [MSGID: 0] [glusterd-peer-utils.c:133:glusterd_peerinfo_find_by_hostname] 0-management: Unable to find friend: 10.32.0.48
[2016-03-22 12:25:23.238891] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.0.48 
[2016-03-22 12:25:23.245314] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.0.48 is local address at interface efd
[2016-03-22 12:25:23.245427] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:23.245561] D [MSGID: 0] [glusterd-brick-ops.c:1741:glusterd_op_stage_add_brick] 0-management: Returning 0
[2016-03-22 12:25:23.245641] D [MSGID: 0] [glusterd-mgmt.c:198:gd_mgmt_v3_pre_validate_fn] 0-management: OP = 8. Returning 0
[2016-03-22 12:25:23.245877] D [MSGID: 0] [glusterd-mgmt-handler.c:271:glusterd_mgmt_v3_pre_validate_send_resp] 0-management: Responded to pre validation, ret: 0
[2016-03-22 12:25:24.271833] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:24.271973] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:24.272007] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:24.272077] I [MSGID: 106062] [glusterd-brick-ops.c:1268:glusterd_op_perform_add_bricks] 0-management: replica-count is set 2
[2016-03-22 12:25:24.272109] I [MSGID: 106062] [glusterd-brick-ops.c:1273:glusterd_op_perform_add_bricks] 0-management: type is set 2, need to change it
[2016-03-22 12:25:24.272189] D [MSGID: 0] [glusterd-utils.c:884:glusterd_brickinfo_new] 0-management: Returning 0
[2016-03-22 12:25:24.272227] D [MSGID: 0] [glusterd-utils.c:1041:glusterd_brickinfo_new_from_brick] 0-management: Returning 0
[2016-03-22 12:25:24.272635] D [MSGID: 0] [glusterd-peer-utils.c:133:glusterd_peerinfo_find_by_hostname] 0-management: Unable to find friend: 10.32.0.48
[2016-03-22 12:25:24.272746] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.0.48 
[2016-03-22 12:25:24.273523] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.0.48 is local address at interface efd
[2016-03-22 12:25:24.273570] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:24.273603] D [MSGID: 0] [glusterd-utils.c:925:glusterd_resolve_brick] 0-management: Returning 0
[2016-03-22 12:25:24.273641] D [MSGID: 0] [glusterd-brick-ops.c:113:add_brick_at_right_order] 0-management: brick:/opt/lvmdir/c2/brick index=1, count=0
[2016-03-22 12:25:24.273839] D [MSGID: 0] [glusterd-volgen.c:5109:generate_brick_volfiles] 0-management: Found a brick - 10.32.1.144:/opt/lvmdir/c2/brick
[2016-03-22 12:25:24.274567] D [MSGID: 0] [glusterd-volgen.c:1268:server_check_marker_off] 0-glusterd: Returning 0
[2016-03-22 12:25:24.275659] D [MSGID: 0] [glusterd-volgen.c:5109:generate_brick_volfiles] 0-management: Found a brick - 10.32.0.48:/opt/lvmdir/c2/brick
[2016-03-22 12:25:24.276437] D [MSGID: 0] [glusterd-volgen.c:1268:server_check_marker_off] 0-glusterd: Returning 0
[2016-03-22 12:25:24.277415] D [MSGID: 0] [glusterd-volgen.c:5120:generate_brick_volfiles] 0-management: Returning 0
[2016-03-22 12:25:24.277497] D [MSGID: 0] [glusterd-utils.c:884:glusterd_brickinfo_new] 0-management: Returning 0
[2016-03-22 12:25:24.277540] D [MSGID: 0] [glusterd-utils.c:1041:glusterd_brickinfo_new_from_brick] 0-management: Returning 0
[2016-03-22 12:25:24.277936] D [MSGID: 0] [glusterd-peer-utils.c:133:glusterd_peerinfo_find_by_hostname] 0-management: Unable to find friend: 10.32.0.48
[2016-03-22 12:25:24.278049] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.0.48 
[2016-03-22 12:25:24.278753] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.0.48 is local address at interface efd
[2016-03-22 12:25:24.278789] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:24.278823] D [MSGID: 0] [glusterd-utils.c:1326:glusterd_volume_brickinfo_get] 0-management: Found brick 10.32.0.48:/opt/lvmdir/c2/brick in volume c_glusterfs
[2016-03-22 12:25:24.278853] D [MSGID: 0] [glusterd-utils.c:1335:glusterd_volume_brickinfo_get] 0-management: Returning 0
[2016-03-22 12:25:24.278883] D [MSGID: 0] [glusterd-utils.c:1359:glusterd_volume_brickinfo_get_by_brick] 0-glusterd: Returning 0
[2016-03-22 12:25:24.313855] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:24.314079] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:24.314156] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=6 from gen=7 to slot->gen=8, fd=15, slot->fd=15
[2016-03-22 12:25:24.321551] D [MSGID: 0] [glusterd-handshake.c:414:glusterd_get_args_from_dict] 0-management: brick_name = /opt/lvmdir/c2/brick
[2016-03-22 12:25:24.321609] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning -1
[2016-03-22 12:25:24.321782] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.0.48 
[2016-03-22 12:25:24.322572] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.0.48 is local address at interface efd
[2016-03-22 12:25:24.322658] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning -1
[2016-03-22 12:25:24.322692] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:24.322720] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:24.322807] D [MSGID: 0] [glusterd-handshake.c:841:__server_getspec] 0-management: Look for missing snap creates for /opt/lvmdir/c2/brick
[2016-03-22 12:25:24.354282] I [MSGID: 106143] [glusterd-pmap.c:229:pmap_registry_bind] 0-pmap: adding brick /opt/lvmdir/c2/brick on port 49730
[2016-03-22 12:25:24.354470] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.1.144 
[2016-03-22 12:25:24.356542] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.1.144 " repeated 2 times between [2016-03-22 12:25:24.354470] and [2016-03-22 12:25:24.355931]
[2016-03-22 12:25:24.356540] D [MSGID: 0] [common-utils.c:3149:gf_is_local_addr] 0-management: 10.32.1.144 is not local
[2016-03-22 12:25:24.356847] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.0.48 
[2016-03-22 12:25:24.357413] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.0.48 is local address at interface efd
[2016-03-22 12:25:24.359592] I [rpc-clnt.c:984:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2016-03-22 12:25:24.359629] D [rpc-clnt.c:1001:rpc_clnt_connection_init] 0-management: disable ping-timeout
[2016-03-22 12:25:24.359670] D [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3.7.6/rpc-transport/socket.so
[2016-03-22 12:25:24.359824] D [socket.c:3794:socket_init] 0-management: disabling nodelay
[2016-03-22 12:25:24.359848] D [socket.c:3845:socket_init] 0-management: Configued transport.tcp-user-timeout=0
[2016-03-22 12:25:24.359873] D [socket.c:3928:socket_init] 0-management: SSL support on the I/O path is NOT enabled
[2016-03-22 12:25:24.359899] D [socket.c:3931:socket_init] 0-management: SSL support for glusterd is NOT enabled
[2016-03-22 12:25:24.359930] D [socket.c:3948:socket_init] 0-management: using system polling thread
[2016-03-22 12:25:24.360061] D [MSGID: 0] [glusterd-handler.c:3323:glusterd_rpc_create] 0-management: returning 0
[2016-03-22 12:25:24.360107] D [MSGID: 0] [glusterd-utils.c:1572:glusterd_brick_connect] 0-glusterd: Returning 0
[2016-03-22 12:25:24.360157] D [MSGID: 0] [glusterd-utils.c:4539:glusterd_brick_start] 0-management: returning 0 
[2016-03-22 12:25:24.360199] D [MSGID: 0] [glusterd-brick-ops.c:1408:glusterd_op_perform_add_bricks] 0-glusterd: Restart gsyncd session, if it's already running.
[2016-03-22 12:25:24.360242] D [MSGID: 0] [glusterd-brick-ops.c:1436:glusterd_op_perform_add_bricks] 0-glusterd: Returning 0
[2016-03-22 12:25:24.360295] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already stopped
[2016-03-22 12:25:24.360945] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff79d1e430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff7678eab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff7678f8b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:25:24.361036] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:24.361067] D [MSGID: 0] [glusterd-nfs-svc.c:135:glusterd_nfssvc_stop] 0-management: Returning 0
[2016-03-22 12:25:24.361293] D [MSGID: 0] [glusterd-volgen.c:4809:build_nfs_graph] 0-glusterd: Returning 0
[2016-03-22 12:25:24.361846] D [MSGID: 0] [glusterd-nfs-svc.c:106:glusterd_nfssvc_manager] 0-management: Returning 0
[2016-03-22 12:25:24.362209] D [MSGID: 0] [glusterd-volgen.c:881:_xl_link_children] 0-management: glustershd:c_glusterfs-replicate-0
[2016-03-22 12:25:24.363514] D [MSGID: 0] [glusterd-shd-svc.c:78:glusterd_shdsvc_create_volfile] 0-management: Returning 0
[2016-03-22 12:25:24.363570] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: glustershd already stopped
[2016-03-22 12:25:24.364429] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff79d1e430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff7678eab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff7678f8b8] ))))) 0-: /var/run/gluster/517bcdf9740ac943aa9615504f526f12.socket: ping timer event already removed
[2016-03-22 12:25:24.364549] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:24.364682] D [MSGID: 0] [glusterd-svc-mgmt.c:195:glusterd_svc_start] 0-management: Starting glustershd service
[2016-03-22 12:25:24.375740] D [MSGID: 0] [glusterd-svc-mgmt.c:208:glusterd_svc_start] 0-management: Returning 0
[2016-03-22 12:25:24.376054] D [MSGID: 0] [glusterd-shd-svc.c:177:glusterd_shdsvc_start] 0-management: Returning 0
[2016-03-22 12:25:24.376365] W [socket.c:3009:socket_connect] 0-glustershd: Ignore failed connection attempt on /var/run/gluster/517bcdf9740ac943aa9615504f526f12.socket, (No such file or directory) 
[2016-03-22 12:25:24.378180] D [MSGID: 0] [glusterd-shd-svc.c:139:glusterd_shdsvc_manager] 0-management: Returning 0
[2016-03-22 12:25:24.378506] D [MSGID: 0] [glusterd-quotad-svc.c:111:glusterd_quotadsvc_manager] 0-management: Returning 0
[2016-03-22 12:25:24.378697] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: bitd already stopped
[2016-03-22 12:25:24.380378] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff79d1e430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff7678eab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff7678f8b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:25:24.380609] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:24.380647] D [MSGID: 0] [glusterd-bitd-svc.c:109:glusterd_bitdsvc_manager] 0-management: Returning 0
[2016-03-22 12:25:24.381112] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: scrub already stopped
[2016-03-22 12:25:24.382170] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff79d1e430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff7678eab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff7678f8b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:25:24.382332] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:24.382365] D [MSGID: 0] [glusterd-scrub-svc.c:104:glusterd_scrubsvc_manager] 0-management: Returning 0
[2016-03-22 12:25:24.382463] D [MSGID: 0] [glusterd-mgmt.c:294:gd_mgmt_v3_commit_fn] 0-management: OP = 8. Returning 0
[2016-03-22 12:25:24.382883] D [MSGID: 0] [glusterd-mgmt-handler.c:537:glusterd_mgmt_v3_commit_send_resp] 0-management: Responded to commit, ret: 0
[2016-03-22 12:25:24.383411] D [MSGID: 0] [glusterd-utils.c:1400:glusterd_volinfo_find_by_volume_id] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:24.383612] D [MSGID: 0] [glusterd-utils.c:884:glusterd_brickinfo_new] 0-management: Returning 0
[2016-03-22 12:25:24.383653] D [MSGID: 0] [glusterd-utils.c:1041:glusterd_brickinfo_new_from_brick] 0-management: Returning 0
[2016-03-22 12:25:24.384095] D [MSGID: 0] [glusterd-peer-utils.c:133:glusterd_peerinfo_find_by_hostname] 0-management: Unable to find friend: 10.32.0.48
[2016-03-22 12:25:24.384222] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.0.48 
[2016-03-22 12:25:24.385061] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.0.48 is local address at interface efd
[2016-03-22 12:25:24.385100] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:24.385134] D [MSGID: 0] [glusterd-utils.c:1326:glusterd_volume_brickinfo_get] 0-management: Found brick 10.32.0.48:/opt/lvmdir/c2/brick in volume c_glusterfs
[2016-03-22 12:25:24.385165] D [MSGID: 0] [glusterd-utils.c:1335:glusterd_volume_brickinfo_get] 0-management: Returning 0
[2016-03-22 12:25:24.385198] D [MSGID: 0] [glusterd-utils.c:1359:glusterd_volume_brickinfo_get_by_brick] 0-glusterd: Returning 0
[2016-03-22 12:25:24.385229] D [MSGID: 0] [glusterd-handler.c:4897:__glusterd_brick_rpc_notify] 0-management: Connected to 10.32.0.48:/opt/lvmdir/c2/brick
[2016-03-22 12:25:24.385276] D [MSGID: 0] [glusterd-utils.c:4869:glusterd_set_brick_status] 0-glusterd: Setting brick 10.32.0.48:/opt/lvmdir/c2/brick status to started
[2016-03-22 12:25:24.385379] D [MSGID: 0] [glusterd-svc-mgmt.c:315:glusterd_svc_common_rpc_notify] 0-management: glustershd has connected with glusterd.
[2016-03-22 12:25:24.385465] W [socket.c:588:__socket_rwv] 0-glustershd: readv on /var/run/gluster/517bcdf9740ac943aa9615504f526f12.socket failed (Invalid argument)
[2016-03-22 12:25:24.385488] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:24.385932] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup-0x19b20)[0x3fff79d1eaf0] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_notify-0x18e68)[0x3fff79d1f808] (--> /usr/lib64/libgfrpc.so.0(rpc_transport_notify-0x1d8bc)[0x3fff79d1a85c] ))))) 0-: /var/run/gluster/517bcdf9740ac943aa9615504f526f12.socket: ping timer event already removed
[2016-03-22 12:25:24.386005] I [MSGID: 106006] [glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: glustershd has disconnected from glusterd.
[2016-03-22 12:25:24.386042] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=8 from gen=1 to slot->gen=2, fd=17, slot->fd=17
[2016-03-22 12:25:24.418286] D [MSGID: 0] [glusterd-handshake.c:409:glusterd_get_args_from_dict] 0-management: No brick name present
[2016-03-22 12:25:24.418340] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning -1
[2016-03-22 12:25:24.418546] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 127.0.0.1 
[2016-03-22 12:25:24.434590] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:24.434624] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:24.434672] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=9 from gen=1 to slot->gen=2, fd=18, slot->fd=18
[2016-03-22 12:25:24.484007] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:24.484126] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:24.484165] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:24.484361] D [MSGID: 0] [glusterd-volgen.c:5109:generate_brick_volfiles] 0-management: Found a brick - 10.32.1.144:/opt/lvmdir/c2/brick
[2016-03-22 12:25:24.485235] D [MSGID: 0] [glusterd-volgen.c:1268:server_check_marker_off] 0-glusterd: Returning 0
[2016-03-22 12:25:24.486238] D [MSGID: 0] [glusterd-volgen.c:5109:generate_brick_volfiles] 0-management: Found a brick - 10.32.0.48:/opt/lvmdir/c2/brick
[2016-03-22 12:25:24.486964] D [MSGID: 0] [glusterd-volgen.c:1268:server_check_marker_off] 0-glusterd: Returning 0
[2016-03-22 12:25:24.487873] D [MSGID: 0] [glusterd-volgen.c:5120:generate_brick_volfiles] 0-management: Returning 0
[2016-03-22 12:25:24.488049] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.1.144 found.. state: 3
[2016-03-22 12:25:24.488081] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:24.488110] D [MSGID: 0] [glusterd-utils.c:1326:glusterd_volume_brickinfo_get] 0-management: Found brick 10.32.1.144:/opt/lvmdir/c2/brick in volume c_glusterfs
[2016-03-22 12:25:24.488138] D [MSGID: 0] [glusterd-utils.c:1335:glusterd_volume_brickinfo_get] 0-management: Returning 0
[2016-03-22 12:25:24.488496] D [MSGID: 0] [glusterd-peer-utils.c:133:glusterd_peerinfo_find_by_hostname] 0-management: Unable to find friend: 10.32.0.48
[2016-03-22 12:25:24.488624] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.0.48 
[2016-03-22 12:25:24.489334] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.0.48 is local address at interface efd
[2016-03-22 12:25:24.489374] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:24.489404] D [MSGID: 0] [glusterd-utils.c:1326:glusterd_volume_brickinfo_get] 0-management: Found brick 10.32.0.48:/opt/lvmdir/c2/brick in volume c_glusterfs
[2016-03-22 12:25:24.489431] D [MSGID: 0] [glusterd-utils.c:1335:glusterd_volume_brickinfo_get] 0-management: Returning 0
[2016-03-22 12:25:24.491248] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.1.144 found.. state: 3
[2016-03-22 12:25:24.491293] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:24.491322] D [MSGID: 0] [glusterd-utils.c:1326:glusterd_volume_brickinfo_get] 0-management: Found brick 10.32.1.144:/opt/lvmdir/c2/brick in volume c_glusterfs
[2016-03-22 12:25:24.491349] D [MSGID: 0] [glusterd-utils.c:1335:glusterd_volume_brickinfo_get] 0-management: Returning 0
[2016-03-22 12:25:24.491670] D [MSGID: 0] [glusterd-peer-utils.c:133:glusterd_peerinfo_find_by_hostname] 0-management: Unable to find friend: 10.32.0.48
[2016-03-22 12:25:24.491768] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.0.48 
[2016-03-22 12:25:24.492417] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.0.48 is local address at interface efd
[2016-03-22 12:25:24.492455] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:24.492484] D [MSGID: 0] [glusterd-utils.c:1326:glusterd_volume_brickinfo_get] 0-management: Found brick 10.32.0.48:/opt/lvmdir/c2/brick in volume c_glusterfs
[2016-03-22 12:25:24.492512] D [MSGID: 0] [glusterd-utils.c:1335:glusterd_volume_brickinfo_get] 0-management: Returning 0
[2016-03-22 12:25:24.494269] D [MSGID: 0] [glusterd-store.c:1058:glusterd_store_create_volume_dir] 0-management: Returning with 0
[2016-03-22 12:25:24.494436] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:24.495983] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 19 times between [2016-03-22 12:25:24.494436] and [2016-03-22 12:25:24.495978]
[2016-03-22 12:25:24.495982] D [MSGID: 0] [glusterd-store.c:440:glusterd_store_snapd_write] 0-management: Returning 0
[2016-03-22 12:25:24.514297] D [MSGID: 0] [glusterd-store.c:503:glusterd_store_perform_snapd_store] 0-management: Returning 0
[2016-03-22 12:25:24.514358] D [MSGID: 0] [glusterd-store.c:568:glusterd_store_snapd_info] 0-management: Returning with 0
[2016-03-22 12:25:24.514427] D [MSGID: 0] [glusterd-store.c:753:_storeopts] 0-management: Storing in volinfo:key= performance.readdir-ahead, val=on
[2016-03-22 12:25:24.514574] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:24.514615] D [MSGID: 0] [glusterd-store.c:753:_storeopts] 0-management: Storing in volinfo:key= network.ping-timeout, val=4
[2016-03-22 12:25:24.514718] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:24.514773] D [MSGID: 0] [glusterd-store.c:753:_storeopts] 0-management: Storing in volinfo:key= nfs.disable, val=on
[2016-03-22 12:25:24.514864] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:24.514898] D [MSGID: 0] [glusterd-store.c:1104:glusterd_store_volinfo_write] 0-management: Returning 0
[2016-03-22 12:25:24.515023] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:24.515692] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 8 times between [2016-03-22 12:25:24.515023] and [2016-03-22 12:25:24.515688]
[2016-03-22 12:25:24.515691] D [MSGID: 0] [glusterd-store.c:414:glusterd_store_brickinfo_write] 0-management: Returning 0
[2016-03-22 12:25:24.515778] D [MSGID: 0] [glusterd-store.c:464:glusterd_store_perform_brick_store] 0-management: Returning 0
[2016-03-22 12:25:24.515808] D [MSGID: 0] [glusterd-store.c:533:glusterd_store_brickinfo] 0-management: Returning with 0
[2016-03-22 12:25:24.515925] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:24.524006] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:25:24.524239] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:24.524728] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 7 times between [2016-03-22 12:25:24.524239] and [2016-03-22 12:25:24.524724]
[2016-03-22 12:25:24.524727] D [MSGID: 0] [glusterd-store.c:414:glusterd_store_brickinfo_write] 0-management: Returning 0
[2016-03-22 12:25:24.524811] D [MSGID: 0] [glusterd-store.c:464:glusterd_store_perform_brick_store] 0-management: Returning 0
[2016-03-22 12:25:24.524842] D [MSGID: 0] [glusterd-store.c:533:glusterd_store_brickinfo] 0-management: Returning with 0
[2016-03-22 12:25:24.524873] D [MSGID: 0] [glusterd-store.c:1325:glusterd_store_brickinfos] 0-management: Returning 0
[2016-03-22 12:25:24.524904] D [MSGID: 0] [glusterd-store.c:1439:glusterd_store_perform_volume_store] 0-management: Returning 0
[2016-03-22 12:25:24.577820] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:24.578106] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 3 times between [2016-03-22 12:25:24.577820] and [2016-03-22 12:25:24.578099]
[2016-03-22 12:25:24.578104] D [MSGID: 0] [glusterd-store.c:1383:glusterd_store_node_state_write] 0-management: Returning 0
[2016-03-22 12:25:24.596071] D [MSGID: 0] [glusterd-store.c:1411:glusterd_store_perform_node_state_store] 0-management: Returning 0
[2016-03-22 12:25:24.596703] D [MSGID: 0] [glusterd-utils.c:2052:glusterd_volume_compute_cksum] 0-management: Returning with 0
[2016-03-22 12:25:24.596740] D [MSGID: 0] [glusterd-store.c:1645:glusterd_store_volinfo] 0-management: Returning 0
[2016-03-22 12:25:24.596892] D [MSGID: 0] [glusterd-mgmt-handler.c:670:glusterd_mgmt_v3_post_validate_send_resp] 0-management: Responded to post validation, ret: 0
[2016-03-22 12:25:24.597688] D [MSGID: 0] [glusterd-handshake.c:414:glusterd_get_args_from_dict] 0-management: brick_name = /opt/lvmdir/c2/brick
[2016-03-22 12:25:24.597745] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning -1
[2016-03-22 12:25:24.597911] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.0.48 
[2016-03-22 12:25:24.598602] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.0.48 is local address at interface efd
[2016-03-22 12:25:24.598648] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning -1
[2016-03-22 12:25:24.598681] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:24.598709] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:24.598804] D [MSGID: 0] [glusterd-handshake.c:841:__server_getspec] 0-management: Look for missing snap creates for /opt/lvmdir/c2/brick
[2016-03-22 12:25:24.599158] D [MSGID: 0] [glusterd-handshake.c:409:glusterd_get_args_from_dict] 0-management: No brick name present
[2016-03-22 12:25:24.599200] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning -1
[2016-03-22 12:25:24.599337] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 127.0.0.1 
[2016-03-22 12:25:24.600223] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:24.600249] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:24.600296] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=9 from gen=4 to slot->gen=5, fd=20, slot->fd=20
[2016-03-22 12:25:24.600656] D [MSGID: 0] [glusterd-mgmt-handler.c:874:glusterd_handle_mgmt_v3_unlock_fn] 0-management: Received volume unlock req from uuid: 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:24.600698] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:24.600785] D [MSGID: 0] [glusterd-locks.c:669:glusterd_mgmt_v3_unlock] 0-management: Trying to release lock of vol c_glusterfs for 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2 as c_glusterfs_vol
[2016-03-22 12:25:24.600829] D [MSGID: 0] [glusterd-locks.c:714:glusterd_mgmt_v3_unlock] 0-management: Lock for vol c_glusterfs successfully released
[2016-03-22 12:25:24.600865] D [MSGID: 0] [glusterd-locks.c:451:glusterd_multiple_mgmt_v3_unlock] 0-management: Returning 0
[2016-03-22 12:25:24.600959] D [MSGID: 0] [glusterd-mgmt-handler.c:789:glusterd_mgmt_v3_unlock_send_resp] 0-management: Responded to mgmt_v3 unlock, ret: 0
[2016-03-22 12:25:24.770567] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:24.770622] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:24.770685] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=9 from gen=7 to slot->gen=8, fd=18, slot->fd=18
[2016-03-22 12:25:24.828063] D [MSGID: 0] [glusterd-handshake.c:409:glusterd_get_args_from_dict] 0-management: No brick name present
[2016-03-22 12:25:24.828120] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:24.828146] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:24.828298] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.0.48 
[2016-03-22 12:25:24.829036] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.0.48 is local address at interface efd
[2016-03-22 12:25:24.829113] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:24.829144] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:24.847471] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:24.847499] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:24.847540] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=10 from gen=1 to slot->gen=2, fd=19, slot->fd=19
[2016-03-22 12:25:27.437822] D [MSGID: 0] [glusterd-svc-mgmt.c:315:glusterd_svc_common_rpc_notify] 0-management: glustershd has connected with glusterd.
[2016-03-22 12:25:28.282381] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:28.282431] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:28.282478] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=11 from gen=1 to slot->gen=2, fd=20, slot->fd=20
[2016-03-22 12:25:28.330090] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:28.330145] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:28.330192] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=11 from gen=4 to slot->gen=5, fd=20, slot->fd=20
[2016-03-22 12:25:32.436689] D [MSGID: 0] [glusterd-handshake.c:409:glusterd_get_args_from_dict] 0-management: No brick name present
[2016-03-22 12:25:32.436744] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:32.436770] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:32.436890] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 127.0.0.1 
[2016-03-22 12:25:32.436937] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:32.436968] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:32.454878] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:32.454935] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:32.454996] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=13 from gen=1 to slot->gen=2, fd=22, slot->fd=22
[2016-03-22 12:25:34.516624] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:34.516685] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:34.516777] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=12 from gen=1 to slot->gen=2, fd=21, slot->fd=21
[2016-03-22 12:25:34.519149] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:34.519179] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:34.519216] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=11 from gen=7 to slot->gen=8, fd=20, slot->fd=20
[2016-03-22 12:25:38.517484] D [MSGID: 0] [glusterd-handshake.c:409:glusterd_get_args_from_dict] 0-management: No brick name present
[2016-03-22 12:25:38.517538] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:38.517566] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:38.517716] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 127.0.0.1 
[2016-03-22 12:25:38.517754] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:38.517784] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:38.536944] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:38.536972] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:38.537012] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=13 from gen=4 to slot->gen=5, fd=22, slot->fd=22
[2016-03-22 12:25:42.239858] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:42.239911] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:42.240003] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=12 from gen=4 to slot->gen=5, fd=21, slot->fd=21
[2016-03-22 12:25:42.241904] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:42.241945] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:42.241979] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=11 from gen=10 to slot->gen=11, fd=20, slot->fd=20
[2016-03-22 12:25:42.516327] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:42.516369] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:42.516410] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=11 from gen=13 to slot->gen=14, fd=20, slot->fd=20
[2016-03-22 12:25:50.441616] D [rpc-clnt-ping.c:300:rpc_clnt_start_ping] 0-management: returning as transport is already disconnected OR there are no frames (0 || 0)
[2016-03-22 12:25:50.442056] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff79d1e430] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_unref-0x1a098)[0x3fff79d1e548] (--> /usr/lib64/libgfrpc.so.0(+0x1a224)[0x3fff79d23224] ))))) 0-: 10.32.1.144:24007: ping timer event already removed
[2016-03-22 12:25:50.442148] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:50.442370] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff79d79f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff79d22990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup-0x19b20)[0x3fff79d1eaf0] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_notify-0x18e68)[0x3fff79d1f808] (--> /usr/lib64/libgfrpc.so.0(rpc_transport_notify-0x1d8bc)[0x3fff79d1a85c] ))))) 0-: 10.32.1.144:24007: ping timer event already removed
[2016-03-22 12:25:50.442433] D [MSGID: 0] [glusterd-peer-utils.c:1005:glusterd_peerinfo_find_by_generation] 0-management: Friend with generation: 2, not found
[2016-03-22 12:25:50.442473] D [MSGID: 106129] [glusterd-handler.c:5032:__glusterd_peer_rpc_notify] 0-management: Could not find peer 10.32.1.144(74c9e3bb-8657-4e74-9888-d1c1d76bb1f2) [No such file or directory]
[2016-03-22 12:25:50.442565] I [MSGID: 101053] [mem-pool.c:616:mem_pool_destroy] 0-management: size=588 max=3 total=6
[2016-03-22 12:25:50.442608] I [MSGID: 101053] [mem-pool.c:616:mem_pool_destroy] 0-management: size=124 max=3 total=6
[2016-03-22 12:25:50.442693] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=2 from gen=1 to slot->gen=2, fd=13, slot->fd=13
[2016-03-22 12:25:50.695048] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:50.695105] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:50.695180] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=3 from gen=1 to slot->gen=2, fd=12, slot->fd=12
[2016-03-22 12:25:52.442308] D [rpc-clnt-ping.c:300:rpc_clnt_start_ping] 0-management: returning as transport is already disconnected OR there are no frames (0 || 0)
[2016-03-22 12:25:58.095270] D [MSGID: 0] [glusterd-handshake.c:409:glusterd_get_args_from_dict] 0-management: No brick name present
[2016-03-22 12:25:58.095327] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:58.095354] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:58.095471] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 127.0.0.1 
[2016-03-22 12:25:58.095509] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:58.095538] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:58.113067] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:58.113099] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:58.113145] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=11 from gen=16 to slot->gen=17, fd=20, slot->fd=20
[2016-03-22 12:26:01.240973] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:26:01.241032] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:26:01.241117] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=3 from gen=4 to slot->gen=5, fd=13, slot->fd=13
[2016-03-22 12:26:01.243066] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:26:01.243093] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:26:01.243127] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=2 from gen=4 to slot->gen=5, fd=12, slot->fd=12
[2016-03-22 12:26:01.506738] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:26:01.506783] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:26:01.506823] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=2 from gen=7 to slot->gen=8, fd=12, slot->fd=12
[2016-03-22 12:26:04.600999] D [MSGID: 0] [glusterd-handshake.c:409:glusterd_get_args_from_dict] 0-management: No brick name present
[2016-03-22 12:26:04.601056] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:26:04.601084] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:26:04.601208] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 127.0.0.1 
[2016-03-22 12:26:04.601247] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:26:04.601276] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:26:04.619413] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:26:04.619441] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:26:04.619484] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=11 from gen=19 to slot->gen=20, fd=20, slot->fd=20
[2016-03-22 12:26:06.676670] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:26:06.676732] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:26:06.676806] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=3 from gen=7 to slot->gen=8, fd=13, slot->fd=13
[2016-03-22 12:26:06.678620] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:26:06.678654] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:26:06.678696] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=2 from gen=10 to slot->gen=11, fd=12, slot->fd=12
[2016-03-22 12:26:06.940933] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:26:06.940987] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:26:06.941029] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=2 from gen=13 to slot->gen=14, fd=12, slot->fd=12
[2016-03-22 12:27:18.449259] D [logging.c:1828:gf_log_flush_timeout_cbk] 0-logging-infra: Log timer timed out. About to flush outstanding messages if present
[2016-03-22 12:27:18.449419] D [logging.c:1790:__gf_log_inject_timer_event] 0-logging-infra: Starting timer now. Timeout = 120, current buf size = 5
[2016-03-22 12:29:18.459309] D [logging.c:1828:gf_log_flush_timeout_cbk] 0-logging-infra: Log timer timed out. About to flush outstanding messages if present
[2016-03-22 12:29:18.459368] D [logging.c:1790:__gf_log_inject_timer_event] 0-logging-infra: Starting timer now. Timeout = 120, current buf size = 5
$ 

EVOA_8300-10_6B> lhsh 000300 cat /var/log/glusterfs/glfsheal-c_glusterfs.log

160322-13:29:24 10.67.29.90 16.0c CPP_MOM-CPP-LSV184-gen1_gen1_COMPLETE stopfile=/tmp/13910
$ lhsh 000300 cat /var/log/glusterfs/glfsheal-c_glusterfs.log
[2016-03-22 12:25:32.423278] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:32.451390] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2016-03-22 12:25:32.452544] I [MSGID: 104045] [glfs-master.c:95:notify] 0-gfapi: New graph 30303033-3030-2d33-3434-352d32303136 (0) coming up
[2016-03-22 12:25:32.452617] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-417: parent translators are ready, attempting connect on transport
[2016-03-22 12:25:32.453635] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-418: parent translators are ready, attempting connect on transport
[2016-03-22 12:25:32.454799] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-418: changing port to 49730 (from 0)
[2016-03-22 12:25:32.455025] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-417: changing port to 49719 (from 0)
[2016-03-22 12:25:32.455796] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-418: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:25:32.456386] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-418: Connected to c_glusterfs-client-418, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:25:32.456430] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-418: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:25:32.456596] I [MSGID: 108005] [afr-common.c:3841:afr_notify] 0-c_glusterfs-replicate-0: Subvolume 'c_glusterfs-client-418' came back up; going online.
[2016-03-22 12:25:32.456741] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-418: Server lk version = 1
[2016-03-22 12:25:32.457017] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-417: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:25:32.457760] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-417: Connected to c_glusterfs-client-417, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:25:32.457800] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-417: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:25:32.490207] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-417: Server lk version = 1
[2016-03-22 12:25:32.492856] I [MSGID: 108031] [afr-common.c:1782:afr_local_discovery_cbk] 0-c_glusterfs-replicate-0: selecting local read_child c_glusterfs-client-418
[2016-03-22 12:25:32.493506] I [MSGID: 104041] [glfs-resolve.c:862:__glfs_active_subvol] 0-c_glusterfs: switched to graph 30303033-3030-2d33-3434-352d32303136 (0)
[2016-03-22 12:25:38.503384] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:38.533130] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2016-03-22 12:25:38.534384] I [MSGID: 104045] [glfs-master.c:95:notify] 0-gfapi: New graph 30303033-3030-2d35-3233-342d32303136 (0) coming up
[2016-03-22 12:25:38.534461] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-417: parent translators are ready, attempting connect on transport
[2016-03-22 12:25:38.535510] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-418: parent translators are ready, attempting connect on transport
[2016-03-22 12:25:38.536842] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-417: changing port to 49719 (from 0)
[2016-03-22 12:25:38.536864] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-418: changing port to 49730 (from 0)
[2016-03-22 12:25:38.538356] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-417: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:25:38.538671] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-418: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:25:38.539114] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-417: Connected to c_glusterfs-client-417, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:25:38.539146] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-417: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:25:38.539202] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-418: Connected to c_glusterfs-client-418, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:25:38.539245] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-418: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:25:38.539313] I [MSGID: 108005] [afr-common.c:3841:afr_notify] 0-c_glusterfs-replicate-0: Subvolume 'c_glusterfs-client-417' came back up; going online.
[2016-03-22 12:25:38.539629] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-417: Server lk version = 1
[2016-03-22 12:25:38.570792] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-418: Server lk version = 1
[2016-03-22 12:25:38.572970] I [MSGID: 108031] [afr-common.c:1782:afr_local_discovery_cbk] 0-c_glusterfs-replicate-0: selecting local read_child c_glusterfs-client-418
[2016-03-22 12:25:38.573889] I [MSGID: 104041] [glfs-resolve.c:862:__glfs_active_subvol] 0-c_glusterfs: switched to graph 30303033-3030-2d35-3233-342d32303136 (0)
[2016-03-22 12:25:58.082090] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:58.109406] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2016-03-22 12:25:58.110585] I [MSGID: 104045] [glfs-master.c:95:notify] 0-gfapi: New graph 30303033-3030-2d35-3533-362d32303136 (0) coming up
[2016-03-22 12:25:58.110647] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-417: parent translators are ready, attempting connect on transport
[2016-03-22 12:25:58.111711] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-418: parent translators are ready, attempting connect on transport
[2016-03-22 12:25:58.112983] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-417: changing port to 49719 (from 0)
[2016-03-22 12:25:58.113001] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-418: changing port to 49730 (from 0)
[2016-03-22 12:25:58.114471] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-417: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:25:58.114639] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-418: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:25:58.115131] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-418: Connected to c_glusterfs-client-418, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:25:58.115166] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-418: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:25:58.115207] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-417: Connected to c_glusterfs-client-417, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:25:58.115243] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-417: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:25:58.115330] I [MSGID: 108005] [afr-common.c:3841:afr_notify] 0-c_glusterfs-replicate-0: Subvolume 'c_glusterfs-client-418' came back up; going online.
[2016-03-22 12:25:58.115414] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-418: Server lk version = 1
[2016-03-22 12:25:58.146237] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-417: Server lk version = 1
[2016-03-22 12:25:58.148223] I [MSGID: 108031] [afr-common.c:1782:afr_local_discovery_cbk] 0-c_glusterfs-replicate-0: selecting local read_child c_glusterfs-client-418
[2016-03-22 12:25:58.149242] I [MSGID: 104041] [glfs-resolve.c:862:__glfs_active_subvol] 0-c_glusterfs: switched to graph 30303033-3030-2d35-3533-362d32303136 (0)
[2016-03-22 12:26:04.586125] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:26:04.615411] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2016-03-22 12:26:04.616809] I [MSGID: 104045] [glfs-master.c:95:notify] 0-gfapi: New graph 30303033-3030-2d35-3831-302d32303136 (0) coming up
[2016-03-22 12:26:04.616894] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-417: parent translators are ready, attempting connect on transport
[2016-03-22 12:26:04.618066] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-418: parent translators are ready, attempting connect on transport
[2016-03-22 12:26:04.619107] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-417: changing port to 49719 (from 0)
[2016-03-22 12:26:04.619346] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-418: changing port to 49730 (from 0)
[2016-03-22 12:26:04.620666] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-418: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:26:04.620779] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-417: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:26:04.621204] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-418: Connected to c_glusterfs-client-418, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:26:04.621237] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-418: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:26:04.621408] I [MSGID: 108005] [afr-common.c:3841:afr_notify] 0-c_glusterfs-replicate-0: Subvolume 'c_glusterfs-client-418' came back up; going online.
[2016-03-22 12:26:04.621488] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-418: Server lk version = 1
[2016-03-22 12:26:04.621560] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-417: Connected to c_glusterfs-client-417, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:26:04.621594] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-417: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:26:04.652065] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-417: Server lk version = 1
[2016-03-22 12:26:04.654171] I [MSGID: 108031] [afr-common.c:1782:afr_local_discovery_cbk] 0-c_glusterfs-replicate-0: selecting local read_child c_glusterfs-client-418
[2016-03-22 12:26:04.655131] I [MSGID: 104041] [glfs-resolve.c:862:__glfs_active_subvol] 0-c_glusterfs: switched to graph 30303033-3030-2d35-3831-302d32303136 (0)
$ 

EVOA_8300-10_6B> lhsh 000300 cat /var/log/glusterfs/glustershd.log

160322-13:29:25 10.67.29.90 16.0c CPP_MOM-CPP-LSV184-gen1_gen1_COMPLETE stopfile=/tmp/13910
$ lhsh 000300 cat /var/log/glusterfs/glustershd.log
[2016-03-22 12:25:16.493146] I [MSGID: 100030] [glusterfsd.c:2318:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.6 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/glustershd -p /system/glusterd/glustershd/run/glustershd.pid -l /var/log/glusterfs/glustershd.log -S /var/run/gluster/517bcdf9740ac943aa9615504f526f12.socket --xlator-option *replicate*.node-uuid=356dabed-4314-48be-9eef-d2711d01270d)
[2016-03-22 12:25:16.517475] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:16.522275] I [graph.c:269:gf_add_cmdline_options] 0-c_glusterfs-replicate-0: adding option 'node-uuid' for volume 'c_glusterfs-replicate-0' with value '356dabed-4314-48be-9eef-d2711d01270d'
[2016-03-22 12:25:16.527471] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2016-03-22 12:25:16.529006] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-416: parent translators are ready, attempting connect on transport
[2016-03-22 12:25:16.530271] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-417: parent translators are ready, attempting connect on transport
[2016-03-22 12:25:16.530802] E [MSGID: 114058] [client-handshake.c:1524:client_query_portmap_cbk] 0-c_glusterfs-client-416: failed to get the port number for remote subvolume. Please run 'gluster volume status' on server to see if brick process is running.
[2016-03-22 12:25:16.530919] I [MSGID: 114018] [client.c:2042:client_rpc_notify] 0-c_glusterfs-client-416: disconnected from c_glusterfs-client-416. Client process will keep trying to connect to glusterd until brick's port is available
Final graph:
+------------------------------------------------------------------------------+
  1: volume c_glusterfs-client-416
  2:     type protocol/client
  3:     option ping-timeout 4
  4:     option remote-host 10.32.0.48
  5:     option remote-subvolume /opt/lvmdir/c2/brick
  6:     option transport-type socket
  7:     option username 06afe471-c710-4493-9ab0-b74b9e66f01f
  8:     option password 6440d9dd-6ccb-44a1-a0eb-c6898ca84178
  9: end-volume
 10:  
 11: volume c_glusterfs-client-417
 12:     type protocol/client
 13:     option ping-timeout 4
 14:     option remote-host 10.32.1.144
 15:     option remote-subvolume /opt/lvmdir/c2/brick
 16:     option transport-type socket
 17:     option username 06afe471-c710-4493-9ab0-b74b9e66f01f
 18:     option password 6440d9dd-6ccb-44a1-a0eb-c6898ca84178
 19: end-volume
 20:  
 21: volume c_glusterfs-replicate-0
 22:     type cluster/replicate
 23:     option node-uuid 356dabed-4314-48be-9eef-d2711d01270d
 24:     option background-self-heal-count 0
 25:     option metadata-self-heal on
 26:     option data-self-heal on
 27:     option entry-self-heal on
 28:     option self-heal-daemon enable
 29:     option iam-self-heal-daemon yes
 30:     subvolumes c_glusterfs-client-416 c_glusterfs-client-417
 31: end-volume
 32:  
 33: volume glustershd
 34:     type debug/io-stats
 35:     subvolumes c_glusterfs-replicate-0
 36: end-volume
 37:  
+------------------------------------------------------------------------------+
[2016-03-22 12:25:16.532391] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-417: changing port to 49719 (from 0)
[2016-03-22 12:25:16.534264] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-417: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:25:16.535153] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-417: Connected to c_glusterfs-client-417, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:25:16.535194] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-417: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:25:16.535350] I [MSGID: 108005] [afr-common.c:3841:afr_notify] 0-c_glusterfs-replicate-0: Subvolume 'c_glusterfs-client-417' came back up; going online.
[2016-03-22 12:25:16.535697] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-417: Server lk version = 1
[2016-03-22 12:25:20.549035] I [glusterfsd-mgmt.c:57:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2016-03-22 12:25:20.642349] I [glusterfsd-mgmt.c:1513:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2016-03-22 12:25:20.753108] W [glusterfsd.c:1236:cleanup_and_exit] (-->/lib64/libpthread.so.0() [0x80f02fb730] -->/usr/sbin/glusterfs(glusterfs_sigwaiter-0x1bdfc) [0x10009b04] -->/usr/sbin/glusterfs(cleanup_and_exit-0x1c02c) [0x100098bc] ) 0-: received signum (15), shutting down
[2016-03-22 12:25:24.391076] I [MSGID: 100030] [glusterfsd.c:2318:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.6 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/glustershd -p /system/glusterd/glustershd/run/glustershd.pid -l /var/log/glusterfs/glustershd.log -S /var/run/gluster/517bcdf9740ac943aa9615504f526f12.socket --xlator-option *replicate*.node-uuid=356dabed-4314-48be-9eef-d2711d01270d)
[2016-03-22 12:25:24.417116] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:24.424621] I [graph.c:269:gf_add_cmdline_options] 0-c_glusterfs-replicate-0: adding option 'node-uuid' for volume 'c_glusterfs-replicate-0' with value '356dabed-4314-48be-9eef-d2711d01270d'
[2016-03-22 12:25:24.429220] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2016-03-22 12:25:24.430508] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-417: parent translators are ready, attempting connect on transport
[2016-03-22 12:25:24.431622] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-418: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
  1: volume c_glusterfs-client-417
  2:     type protocol/client
  3:     option ping-timeout 4
  4:     option remote-host 10.32.1.144
  5:     option remote-subvolume /opt/lvmdir/c2/brick
  6:     option transport-type socket
  7:     option username 06afe471-c710-4493-9ab0-b74b9e66f01f
  8:     option password 6440d9dd-6ccb-44a1-a0eb-c6898ca84178
  9: end-volume
 10:  
 11: volume c_glusterfs-client-418
 12:     type protocol/client
 13:     option ping-timeout 4
 14:     option remote-host 10.32.0.48
 15:     option remote-subvolume /opt/lvmdir/c2/brick
 16:     option transport-type socket
 17:     option username 06afe471-c710-4493-9ab0-b74b9e66f01f
 18:     option password 6440d9dd-6ccb-44a1-a0eb-c6898ca84178
 19: end-volume
 20:  
 21: volume c_glusterfs-replicate-0
 22:     type cluster/replicate
 23:     option node-uuid 356dabed-4314-48be-9eef-d2711d01270d
 24:     option background-self-heal-count 0
 25:     option metadata-self-heal on
 26:     option data-self-heal on
 27:     option entry-self-heal on
 28:     option self-heal-daemon enable
 29:     option iam-self-heal-daemon yes
 30:     subvolumes c_glusterfs-client-417 c_glusterfs-client-418
 31: end-volume
 32:  
 33: volume glustershd
 34:     type debug/io-stats
 35:     subvolumes c_glusterfs-replicate-0
 36: end-volume
 37:  
+------------------------------------------------------------------------------+
[2016-03-22 12:25:24.434498] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-418: changing port to 49730 (from 0)
[2016-03-22 12:25:24.437543] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-418: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:25:24.445446] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-418: Connected to c_glusterfs-client-418, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:25:24.445508] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-418: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:25:24.445720] I [MSGID: 108005] [afr-common.c:3841:afr_notify] 0-c_glusterfs-replicate-0: Subvolume 'c_glusterfs-client-418' came back up; going online.
[2016-03-22 12:25:24.445821] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-418: Server lk version = 1
[2016-03-22 12:25:24.485008] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-417: changing port to 49719 (from 0)
[2016-03-22 12:25:24.487170] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-417: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:25:24.487878] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-417: Connected to c_glusterfs-client-417, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:25:24.488013] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-417: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:25:24.488635] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-417: Server lk version = 1
[2016-03-22 12:25:24.494996] I [glusterfsd-mgmt.c:57:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2016-03-22 12:25:24.599728] I [glusterfsd-mgmt.c:1513:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2016-03-22 12:25:24.664759] I [MSGID: 108026] [afr-self-heal-common.c:651:afr_log_selfheal] 0-c_glusterfs-replicate-0: Completed data selfheal on 0d40ab37-e9c2-423d-a81d-8090a6b7596c. source=0 sinks=1 
$ 

EVOA_8300-10_6B> lhsh 000300 cat /var/log/glusterfs/mnt-c.log

160322-13:29:26 10.67.29.90 16.0c CPP_MOM-CPP-LSV184-gen1_gen1_COMPLETE stopfile=/tmp/13910
$ lhsh 000300 cat /var/log/glusterfs/mnt-c.log
[2016-03-22 12:25:24.803655] I [MSGID: 100030] [glusterfsd.c:2318:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.6 (args: /usr/sbin/glusterfs --volfile-server=10.32.0.48 --volfile-id=c_glusterfs /mnt/c)
[2016-03-22 12:25:24.827307] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:24.843764] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2016-03-22 12:25:24.844950] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-417: parent translators are ready, attempting connect on transport
[2016-03-22 12:25:24.845997] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-418: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
  1: volume c_glusterfs-client-417
  2:     type protocol/client
  3:     option ping-timeout 4
  4:     option remote-host 10.32.1.144
  5:     option remote-subvolume /opt/lvmdir/c2/brick
  6:     option transport-type socket
  7:     option username 06afe471-c710-4493-9ab0-b74b9e66f01f
  8:     option password 6440d9dd-6ccb-44a1-a0eb-c6898ca84178
  9:     option send-gids true
 10: end-volume
 11:  
 12: volume c_glusterfs-client-418
 13:     type protocol/client
 14:     option ping-timeout 4
 15:     option remote-host 10.32.0.48
 16:     option remote-subvolume /opt/lvmdir/c2/brick
 17:     option transport-type socket
 18:     option username 06afe471-c710-4493-9ab0-b74b9e66f01f
 19:     option password 6440d9dd-6ccb-44a1-a0eb-c6898ca84178
 20:     option send-gids true
 21: end-volume
 22:  
 23: volume c_glusterfs-replicate-0
[2016-03-22 12:25:24.847335] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-418: changing port to 49730 (from 0)
 24:     type cluster/replicate
 25:     subvolumes c_glusterfs-client-417 c_glusterfs-client-418
 26: end-volume
 27:  
 28: volume c_glusterfs-dht
 29:     type cluster/distribute
 30:     subvolumes c_glusterfs-replicate-0
 31: end-volume
 32:  
 33: volume c_glusterfs-write-behind
 34:     type performance/write-behind
 35:     subvolumes c_glusterfs-dht
 36: end-volume
 37:  
[2016-03-22 12:25:24.847495] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-417: changing port to 49719 (from 0)
 38: volume c_glusterfs-read-ahead
 39:     type performance/read-ahead
 40:     subvolumes c_glusterfs-write-behind
 41: end-volume
 42:  
 43: volume c_glusterfs-readdir-ahead
 44:     type performance/readdir-ahead
 45:     subvolumes c_glusterfs-read-ahead
 46: end-volume
 47:  
 48: volume c_glusterfs-io-cache
 49:     type performance/io-cache
 50:     subvolumes c_glusterfs-readdir-ahead
 51: end-volume
 52:  
 53: volume c_glusterfs-quick-read
 54:     type performance/quick-read
 55:     subvolumes c_glusterfs-io-cache
 56: end-volume
 57:  
 58: volume c_glusterfs-open-behind
 59:     type performance/open-behind
 60:     subvolumes c_glusterfs-quick-read
 61: end-volume
 62:  
 63: volume c_glusterfs-md-cache
 64:     type performance/md-cache
 65:     subvolumes c_glusterfs-open-behind
 66: end-volume
 67:  
 68: volume c_glusterfs
 69:     type debug/io-stats
 70:     option latency-measurement off
 71:     option count-fop-hits off
 72:     subvolumes c_glusterfs-md-cache
 73: end-volume
 74:  
 75: volume meta-autoload
 76:     type meta
 77:     subvolumes c_glusterfs
 78: end-volume
 79:  
+------------------------------------------------------------------------------+
[2016-03-22 12:25:24.849487] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-418: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:25:24.849908] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-417: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:25:24.850129] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-418: Connected to c_glusterfs-client-418, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:25:24.850169] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-418: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:25:24.850339] I [MSGID: 108005] [afr-common.c:3841:afr_notify] 0-c_glusterfs-replicate-0: Subvolume 'c_glusterfs-client-418' came back up; going online.
[2016-03-22 12:25:24.850445] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-418: Server lk version = 1
[2016-03-22 12:25:24.850657] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-417: Connected to c_glusterfs-client-417, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:25:24.850687] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-417: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:25:24.859970] I [fuse-bridge.c:5137:fuse_graph_setup] 0-fuse: switched to graph 0
[2016-03-22 12:25:24.860228] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-417: Server lk version = 1
[2016-03-22 12:25:24.860574] I [fuse-bridge.c:4030:fuse_init] 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.22 kernel 7.22
[2016-03-22 12:25:24.862802] I [MSGID: 108031] [afr-common.c:1782:afr_local_discovery_cbk] 0-c_glusterfs-replicate-0: selecting local read_child c_glusterfs-client-418
[2016-03-22 12:25:25.925962] W [MSGID: 108008] [afr-self-heal-name.c:359:afr_selfheal_name_gfid_mismatch_check] 0-c_glusterfs-replicate-0: GFID mismatch for <gfid:90643798-3dc6-4ae3-ac34-0993e14d14e2>/CELLO_SECURITYEVENT_LOG.xml 1609541d-60c0-4199-ac52-78c8d21cf9a4 on c_glusterfs-client-418 and 9e927ed6-f456-4ec1-9724-c86e17bd028a on c_glusterfs-client-417
[2016-03-22 12:25:25.927538] W [fuse-bridge.c:462:fuse_entry_cbk] 0-glusterfs-fuse: 72: LOOKUP() /logfiles/security/CELLO_SECURITYEVENT_LOG.xml => -1 (Input/output error)
[2016-03-22 12:25:25.941221] W [fuse-bridge.c:462:fuse_entry_cbk] 0-glusterfs-fuse: 78: LOOKUP() /logfiles/security/CELLO_SECURITYEVENT_LOG.xml => -1 (Input/output error)
[2016-03-22 12:25:26.789070] W [fuse-bridge.c:462:fuse_entry_cbk] 0-glusterfs-fuse: 409: LOOKUP() /logfiles/security/CELLO_SECURITYEVENT_LOG.xml => -1 (Input/output error)
The message "W [MSGID: 108008] [afr-self-heal-name.c:359:afr_selfheal_name_gfid_mismatch_check] 0-c_glusterfs-replicate-0: GFID mismatch for <gfid:90643798-3dc6-4ae3-ac34-0993e14d14e2>/CELLO_SECURITYEVENT_LOG.xml 1609541d-60c0-4199-ac52-78c8d21cf9a4 on c_glusterfs-client-418 and 9e927ed6-f456-4ec1-9724-c86e17bd028a on c_glusterfs-client-417" repeated 2 times between [2016-03-22 12:25:25.925962] and [2016-03-22 12:25:26.784162]
[2016-03-22 12:25:31.794567] W [MSGID: 108008] [afr-self-heal-name.c:359:afr_selfheal_name_gfid_mismatch_check] 0-c_glusterfs-replicate-0: GFID mismatch for <gfid:69eee88c-ddb6-4c27-b990-48123191b112>/corbasecurity 0e689d54-2822-4620-afe2-0109d4c45efc on c_glusterfs-client-418 and 288fe1cf-e5b0-4c40-813d-52d652afd4f2 on c_glusterfs-client-417
[2016-03-22 12:25:31.795367] W [fuse-bridge.c:462:fuse_entry_cbk] 0-glusterfs-fuse: 1871: LOOKUP() /security/corbasecurity => -1 (Input/output error)
[2016-03-22 12:25:31.805137] W [fuse-bridge.c:462:fuse_entry_cbk] 0-glusterfs-fuse: 1872: LOOKUP() /security/corbasecurity => -1 (Input/output error)
[2016-03-22 12:25:31.804386] W [MSGID: 108008] [afr-self-heal-name.c:359:afr_selfheal_name_gfid_mismatch_check] 0-c_glusterfs-replicate-0: GFID mismatch for <gfid:69eee88c-ddb6-4c27-b990-48123191b112>/corbasecurity 0e689d54-2822-4620-afe2-0109d4c45efc on c_glusterfs-client-418 and 288fe1cf-e5b0-4c40-813d-52d652afd4f2 on c_glusterfs-client-417
[2016-03-22 12:25:31.988862] W [MSGID: 108008] [afr-self-heal-name.c:359:afr_selfheal_name_gfid_mismatch_check] 0-c_glusterfs-replicate-0: GFID mismatch for <gfid:d9cd3be0-3fa4-4d1e-8a8d-a8523535ef0a>/nameroot.ior 50f131be-29bf-49bf-a843-7d7417ea3fcc on c_glusterfs-client-418 and 087044af-daae-414a-bd60-bf528890e0d4 on c_glusterfs-client-417
[2016-03-22 12:25:31.989885] W [fuse-bridge.c:462:fuse_entry_cbk] 0-glusterfs-fuse: 1886: LOOKUP() /public_html/cello/ior_files/nameroot.ior => -1 (Input/output error)
[2016-03-22 12:25:32.004011] W [fuse-bridge.c:462:fuse_entry_cbk] 0-glusterfs-fuse: 1888: LOOKUP() /public_html/cello/ior_files/nameroot.ior => -1 (Input/output error)
[2016-03-22 12:25:32.031697] W [fuse-bridge.c:462:fuse_entry_cbk] 0-glusterfs-fuse: 1899: LOOKUP() /public_html/cello/ior_files/nameroot.ior => -1 (Input/output error)
The message "W [MSGID: 108008] [afr-self-heal-name.c:359:afr_selfheal_name_gfid_mismatch_check] 0-c_glusterfs-replicate-0: GFID mismatch for <gfid:d9cd3be0-3fa4-4d1e-8a8d-a8523535ef0a>/nameroot.ior 50f131be-29bf-49bf-a843-7d7417ea3fcc on c_glusterfs-client-418 and 087044af-daae-414a-bd60-bf528890e0d4 on c_glusterfs-client-417" repeated 2 times between [2016-03-22 12:25:31.988862] and [2016-03-22 12:25:32.030684]
[2016-03-22 12:25:46.602558] I [MSGID: 109066] [dht-rename.c:1411:dht_rename] 0-c_glusterfs-dht: renaming /loadmodules_norepl/CXC1724447%24_R40T/cello/emas/CPPClientConfig.xml.tmp (hash=c_glusterfs-replicate-0/cache=c_glusterfs-replicate-0) => /loadmodules_norepl/CXC1724447%24_R40T/cello/emas/CPPClientConfig.xml (hash=c_glusterfs-replicate-0/cache=c_glusterfs-replicate-0)
$ 

EVOA_8300-10_6B> lhsh 000300 cat /var/log/glusterfs/snaps

160322-13:29:28 10.67.29.90 16.0c CPP_MOM-CPP-LSV184-gen1_gen1_COMPLETE stopfile=/tmp/13910
key_verify failed for server_host_key
Unable to connect to 10.67.29.90:22

EVOA_8300-10_6B> 

EVOA_8300-10_6B> 

EVOA_8300-10_6B> 

EVOA_8300-10_6B> lhsh 000300 cat /var/log/glusterfs/snaps

160322-13:29:36 10.67.29.90 16.0c CPP_MOM-CPP-LSV184-gen1_gen1_COMPLETE stopfile=/tmp/13910
$ lhsh 000300 cat /var/log/glusterfs/snaps
cat: /var/log/glusterfs/snaps: No such file or directory
$ 

EVOA_8300-10_6B> lhsh 000300 ls /var/log/glusterfs/snaps

160322-13:29:44 10.67.29.90 16.0c CPP_MOM-CPP-LSV184-gen1_gen1_COMPLETE stopfile=/tmp/13910
$ lhsh 000300 ls /var/log/glusterfs/snaps
ls: cannot access /var/log/glusterfs/snaps: No such file or directory
$ 

EVOA_8300-10_6B> 

EVOA_8300-10_6B> 

EVOA_8300-10_6B> 

EVOA_8300-10_6B> 

EVOA_8300-10_6B> 

EVOA_8300-10_6B> lhsh 000300 ls /var/log/glusterfs/snaps

160322-13:29:59 10.67.29.90 16.0c CPP_MOM-CPP-LSV184-gen1_gen1_COMPLETE stopfile=/tmp/13910
$ lhsh 000300 ls /var/log/glusterfs/snaps
ls: cannot access /var/log/glusterfs/snaps: No such file or directory
$ 

EVOA_8300-10_6B> lhsh 000300 ls /var/log/glusterfs/

160322-13:30:05 10.67.29.90 16.0c CPP_MOM-CPP-LSV184-gen1_gen1_COMPLETE stopfile=/tmp/13910
$ lhsh 000300 ls /var/log/glusterfs/
bricks
cli.log
cmd_history.log
etc-glusterfs-glusterd.vol.log
glfsheal-c_glusterfs.log
glustershd.log
mnt-c.log
$ 

EVOA_8300-10_6B> lhsh 002500 ls /var/log/glusterfs/

160322-13:30:15 10.67.29.90 16.0c CPP_MOM-CPP-LSV184-gen1_gen1_COMPLETE stopfile=/tmp/13910
$ lhsh 002500 ls /var/log/glusterfs/
bricks
cli.log
cmd_history.log
etc-glusterfs-glusterd.vol.log
glfsheal-c_glusterfs.log
glustershd.log
mnt-c.log
$ 

EVOA_8300-10_6B> lhsh 002500 ls /var/log/glusterfs/bricks

160322-13:30:23 10.67.29.90 16.0c CPP_MOM-CPP-LSV184-gen1_gen1_COMPLETE stopfile=/tmp/13910
$ lhsh 002500 ls /var/log/glusterfs/bricks
opt-lvmdir-c2-brick.log
$ 

EVOA_8300-10_6B> lhsh 002500 cat /var/log/glusterfs/bricks/opt-lvmdir-c2-brick.log

160322-13:32:01 10.67.29.90 16.0c CPP_MOM-CPP-LSV184-gen1_gen1_COMPLETE stopfile=/tmp/13910
$ lhsh 002500 cat /var/log/glusterfs/bricks/opt-lvmdir-c2-brick.log
[2016-03-22 12:20:08.815596] I [MSGID: 100030] [glusterfsd.c:2318:main] 0-/usr/sbin/glusterfsd: Started running /usr/sbin/glusterfsd version 3.7.6 (args: /usr/sbin/glusterfsd -s 10.32.1.144 --volfile-id c_glusterfs.10.32.1.144.opt-lvmdir-c2-brick -p /system/glusterd/vols/c_glusterfs/run/10.32.1.144-opt-lvmdir-c2-brick.pid -S /var/run/gluster/697c0e4a16ebc734cd06fd9150723005.socket --brick-name /opt/lvmdir/c2/brick -l /var/log/glusterfs/bricks/opt-lvmdir-c2-brick.log --xlator-option *-posix.glusterd-uuid=74c9e3bb-8657-4e74-9888-d1c1d76bb1f2 --brick-port 49719 --xlator-option c_glusterfs-server.listen-port=49719)
[2016-03-22 12:20:08.835394] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:20:12.423424] I [graph.c:269:gf_add_cmdline_options] 0-c_glusterfs-server: adding option 'listen-port' for volume 'c_glusterfs-server' with value '49719'
[2016-03-22 12:20:12.423487] I [graph.c:269:gf_add_cmdline_options] 0-c_glusterfs-posix: adding option 'glusterd-uuid' for volume 'c_glusterfs-posix' with value '74c9e3bb-8657-4e74-9888-d1c1d76bb1f2'
[2016-03-22 12:20:12.423974] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2016-03-22 12:20:12.424048] I [MSGID: 115034] [server.c:403:_check_for_auth_option] 0-/opt/lvmdir/c2/brick: skip format check for non-addr auth option auth.login./opt/lvmdir/c2/brick.allow
[2016-03-22 12:20:12.424092] I [MSGID: 115034] [server.c:403:_check_for_auth_option] 0-/opt/lvmdir/c2/brick: skip format check for non-addr auth option auth.login.06afe471-c710-4493-9ab0-b74b9e66f01f.password
[2016-03-22 12:20:12.425707] I [rpcsvc.c:2215:rpcsvc_set_outstanding_rpc_limit] 0-rpc-service: Configured rpc.outstanding-rpc-limit with value 64
[2016-03-22 12:20:12.425862] W [MSGID: 101002] [options.c:957:xl_opt_validate] 0-c_glusterfs-server: option 'listen-port' is deprecated, preferred is 'transport.socket.listen-port', continuing with correction
[2016-03-22 12:20:12.429159] I [trash.c:2366:init] 0-c_glusterfs-trash: no option specified for 'eliminate', using NULL
[2016-03-22 12:20:12.431127] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-server: option 'rpc-auth.auth-glusterfs' is not recognized
[2016-03-22 12:20:12.431173] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-server: option 'rpc-auth.auth-unix' is not recognized
[2016-03-22 12:20:12.431202] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-server: option 'rpc-auth.auth-null' is not recognized
[2016-03-22 12:20:12.431248] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-quota: option 'timeout' is not recognized
[2016-03-22 12:20:12.431276] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-marker: option 'quota-version' is not recognized
[2016-03-22 12:20:12.431309] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-trash: option 'brick-path' is not recognized
[2016-03-22 12:20:12.441835] W [MSGID: 113036] [posix.c:2018:posix_rename] 0-c_glusterfs-posix: found directory at /opt/lvmdir/c2/brick/.trashcan/ while expecting ENOENT [File exists]
Final graph:
+------------------------------------------------------------------------------+
  1: volume c_glusterfs-posix
  2:     type storage/posix
  3:     option glusterd-uuid 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
  4:     option directory /opt/lvmdir/c2/brick
  5:     option volume-id 0992c376-e66e-4bc4-a0c7-2ca28f894aa2
  6: end-volume
  7:  
  8: volume c_glusterfs-trash
  9:     type features/trash
 10:     option trash-dir .trashcan
 11:     option brick-path /opt/lvmdir/c2/brick
 12:     option trash-internal-op off
 13:     subvolumes c_glusterfs-posix
 14: end-volume
 15:  
 16: volume c_glusterfs-changelog
 17:     type features/changelog
 18:     option changelog-brick /opt/lvmdir/c2/brick
 19:     option changelog-dir /opt/lvmdir/c2/brick/.glusterfs/changelogs
 20:     option changelog-barrier-timeout 120
 21:     subvolumes c_glusterfs-trash
 22: end-volume
 23:  
 24: volume c_glusterfs-bitrot-stub
 25:     type features/bitrot-stub
 26:     option export /opt/lvmdir/c2/brick
 27:     subvolumes c_glusterfs-changelog
 28: end-volume
 29:  
 30: volume c_glusterfs-access-control
 31:     type features/access-control
 32:     subvolumes c_glusterfs-bitrot-stub
 33: end-volume
 34:  
 35: volume c_glusterfs-locks
 36:     type features/locks
 37:     subvolumes c_glusterfs-access-control
 38: end-volume
 39:  
 40: volume c_glusterfs-upcall
 41:     type features/upcall
 42:     option cache-invalidation off
 43:     subvolumes c_glusterfs-locks
 44: end-volume
 45:  
 46: volume c_glusterfs-io-threads
 47:     type performance/io-threads
 48:     subvolumes c_glusterfs-upcall
 49: end-volume
 50:  
 51: volume c_glusterfs-marker
 52:     type features/marker
 53:     option volume-uuid 0992c376-e66e-4bc4-a0c7-2ca28f894aa2
 54:     option timestamp-file /system/glusterd/vols/c_glusterfs/marker.tstamp
 55:     option quota-version 0
 56:     option xtime off
 57:     option gsync-force-xtime off
 58:     option quota off
 59:     option inode-quota off
 60:     subvolumes c_glusterfs-io-threads
 61: end-volume
 62:  
 63: volume c_glusterfs-barrier
 64:     type features/barrier
 65:     option barrier disable
 66:     option barrier-timeout 120
 67:     subvolumes c_glusterfs-marker
 68: end-volume
 69:  
 70: volume c_glusterfs-index
 71:     type features/index
 72:     option index-base /opt/lvmdir/c2/brick/.glusterfs/indices
 73:     subvolumes c_glusterfs-barrier
 74: end-volume
 75:  
 76: volume c_glusterfs-quota
 77:     type features/quota
 78:     option volume-uuid c_glusterfs
 79:     option server-quota off
 80:     option timeout 0
 81:     option deem-statfs off
 82:     subvolumes c_glusterfs-index
 83: end-volume
 84:  
 85: volume c_glusterfs-worm
 86:     type features/worm
 87:     option worm off
 88:     subvolumes c_glusterfs-quota
 89: end-volume
 90:  
 91: volume c_glusterfs-read-only
 92:     type features/read-only
 93:     option read-only off
 94:     subvolumes c_glusterfs-worm
 95: end-volume
 96:  
 97: volume /opt/lvmdir/c2/brick
 98:     type debug/io-stats
 99:     option latency-measurement off
100:     option count-fop-hits off
101:     subvolumes c_glusterfs-read-only
102: end-volume
103:  
104: volume c_glusterfs-server
105:     type protocol/server
106:     option transport.socket.listen-port 49719
107:     option rpc-auth.auth-glusterfs on
108:     option rpc-auth.auth-unix on
109:     option rpc-auth.auth-null on
110:     option rpc-auth-allow-insecure on
111:     option transport-type tcp
112:     option auth.login./opt/lvmdir/c2/brick.allow 06afe471-c710-4493-9ab0-b74b9e66f01f
113:     option auth.login.06afe471-c710-4493-9ab0-b74b9e66f01f.password 6440d9dd-6ccb-44a1-a0eb-c6898ca84178
114:     option auth.addr./opt/lvmdir/c2/brick.allow *
115:     option transport.tcp-user-timeout 4
116:     subvolumes /opt/lvmdir/c2/brick
117: end-volume
118:  
+------------------------------------------------------------------------------+
[2016-03-22 12:20:16.376572] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:20:16.376638] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-2688-2016/03/22-12:20:08:789697-c_glusterfs-client-417-0-0 (version: 3.7.6)
[2016-03-22 12:20:24.182325] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:20:24.182387] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-13758-2016/03/22-12:20:24:131840-c_glusterfs-client-417-0-0 (version: 3.7.6)
[2016-03-22 12:20:24.225935] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:20:24.225985] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-3863-2016/03/22-12:20:24:176727-c_glusterfs-client-417-0-0 (version: 3.7.6)
[2016-03-22 12:20:26.302319] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-13758-2016/03/22-12:20:24:131840-c_glusterfs-client-417-0-0
[2016-03-22 12:20:26.302397] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-13758-2016/03/22-12:20:24:131840-c_glusterfs-client-417-0-0
[2016-03-22 12:20:26.376398] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:20:26.376440] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-13811-2016/03/22-12:20:26:332512-c_glusterfs-client-417-0-0 (version: 3.7.6)
[2016-03-22 12:20:26.548809] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:20:26.548843] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-4047-2016/03/22-12:08:36:393193-c_glusterfs-client-417-0-1 (version: 3.7.6)
[2016-03-22 12:20:26.549072] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:20:26.549107] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-2617-2016/03/22-12:08:12:335429-c_glusterfs-client-417-0-1 (version: 3.7.6)
[2016-03-22 12:20:28.436085] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-13811-2016/03/22-12:20:26:332512-c_glusterfs-client-417-0-0
[2016-03-22 12:20:28.436153] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-13811-2016/03/22-12:20:26:332512-c_glusterfs-client-417-0-0
[2016-03-22 12:20:28.640959] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:20:28.640996] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-3927-2016/03/22-12:20:28:599572-c_glusterfs-client-417-0-0 (version: 3.7.6)
[2016-03-22 12:20:30.696237] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-3927-2016/03/22-12:20:28:599572-c_glusterfs-client-417-0-0
[2016-03-22 12:20:30.696313] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-3927-2016/03/22-12:20:28:599572-c_glusterfs-client-417-0-0
[2016-03-22 12:20:41.128228] W [MSGID: 113096] [posix-handle.c:754:posix_handle_hard] 0-c_glusterfs-posix: mismatching ino/dev between file /opt/lvmdir/c2/brick/java/CXC1736660_P91A001.jar (1212/64769) and handle /opt/lvmdir/c2/brick/.glusterfs/06/cb/06cbfab1-c59a-48f2-b8f7-3e21d09fb478 (343/64769)
[2016-03-22 12:20:42.002869] W [MSGID: 113096] [posix-handle.c:754:posix_handle_hard] 0-c_glusterfs-posix: mismatching ino/dev between file /opt/lvmdir/c2/brick/java/install/CXC1721196_P91A001.jar (1213/64769) and handle /opt/lvmdir/c2/brick/.glusterfs/64/ed/64edead4-e3e6-4f52-af32-0d6729cc7f18 (522/64769)
[2016-03-22 12:20:42.530237] W [MSGID: 113096] [posix-handle.c:754:posix_handle_hard] 0-c_glusterfs-posix: mismatching ino/dev between file /opt/lvmdir/c2/brick/java/install/CXC1721553_P91A001.jar (1200/64769) and handle /opt/lvmdir/c2/brick/.glusterfs/85/1b/851b2126-79b7-44dc-8dbd-691c11db4612 (6083/64769)
[2016-03-22 12:20:42.576136] W [MSGID: 113096] [posix-handle.c:754:posix_handle_hard] 0-c_glusterfs-posix: mismatching ino/dev between file /opt/lvmdir/c2/brick/java/install/CXC1736660_P91A351.jar (3620/64769) and handle /opt/lvmdir/c2/brick/.glusterfs/ed/70/ed70040f-63fd-40ea-be83-889318ce52a2 (7056/64769)
[2016-03-22 12:20:42.628757] W [MSGID: 113096] [posix-handle.c:754:posix_handle_hard] 0-c_glusterfs-posix: mismatching ino/dev between file /opt/lvmdir/c2/brick/java/install/CXC1730438_P91A024.jar (3631/64769) and handle /opt/lvmdir/c2/brick/.glusterfs/7e/33/7e33cd4b-d8d5-40ea-b208-19e6f2160edc (4944/64769)
[2016-03-22 12:24:25.936214] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-4047-2016/03/22-12:08:36:393193-c_glusterfs-client-417-0-1
[2016-03-22 12:24:25.936330] W [inodelk.c:404:pl_inodelk_log_cleanup] 0-c_glusterfs-server: releasing lock on 0d40ab37-e9c2-423d-a81d-8090a6b7596c held by {client=0x3fff8400b800, pid=0 lk-owner=4044007cff3f0000}
[2016-03-22 12:24:25.936380] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /logfiles/systemlog/syslog
[2016-03-22 12:24:25.936495] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-4047-2016/03/22-12:08:36:393193-c_glusterfs-client-417-0-1
[2016-03-22 12:24:28.459645] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-2617-2016/03/22-12:08:12:335429-c_glusterfs-client-417-0-1
[2016-03-22 12:24:28.459756] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-2617-2016/03/22-12:08:12:335429-c_glusterfs-client-417-0-1
[2016-03-22 12:25:00.310889] E [MSGID: 113001] [posix.c:5103:_posix_handle_xattr_keyvalue_pair] 0-c_glusterfs-posix: setxattr failed on /opt/lvmdir/c2/brick/.glusterfs/df/00/df00d265-4b67-415d-95a7-f9e4cac83029 while doing xattrop: key=trusted.afr.c_glusterfs-client-416 [No space left on device]
[2016-03-22 12:25:00.311011] I [MSGID: 115073] [server-rpc-fops.c:1851:server_xattrop_cbk] 0-c_glusterfs-server: 150: XATTROP /configuration/oamrdncy.cfg (df00d265-4b67-415d-95a7-f9e4cac83029) ==> (No space left on device) [No space left on device]
[2016-03-22 12:25:00.332256] E [MSGID: 113001] [posix.c:5110:_posix_handle_xattr_keyvalue_pair] 0-c_glusterfs-posix: fsetxattr failed on gfid=df00d265-4b67-415d-95a7-f9e4cac83029 while doing xattrop: key=trusted.afr.c_glusterfs-client-416 (No space left on device) [No space left on device]
[2016-03-22 12:25:00.332348] I [MSGID: 115073] [server-rpc-fops.c:1893:server_fxattrop_cbk] 0-c_glusterfs-server: 157: FXATTROP 1 (df00d265-4b67-415d-95a7-f9e4cac83029) ==> (No space left on device) [No space left on device]
[2016-03-22 12:25:00.361447] E [MSGID: 113001] [posix.c:5103:_posix_handle_xattr_keyvalue_pair] 0-c_glusterfs-posix: setxattr failed on /opt/lvmdir/c2/brick/.glusterfs/df/00/df00d265-4b67-415d-95a7-f9e4cac83029 while doing xattrop: key=trusted.afr.c_glusterfs-client-416 [No space left on device]
[2016-03-22 12:25:00.361570] I [MSGID: 115073] [server-rpc-fops.c:1851:server_xattrop_cbk] 0-c_glusterfs-server: 167: XATTROP /configuration/oamrdncy.cfg (df00d265-4b67-415d-95a7-f9e4cac83029) ==> (No space left on device) [No space left on device]
[2016-03-22 12:25:00.387214] E [MSGID: 113001] [posix.c:5110:_posix_handle_xattr_keyvalue_pair] 0-c_glusterfs-posix: fsetxattr failed on gfid=df00d265-4b67-415d-95a7-f9e4cac83029 while doing xattrop: key=trusted.afr.c_glusterfs-client-416 (No space left on device) [No space left on device]
[2016-03-22 12:25:00.387326] I [MSGID: 115073] [server-rpc-fops.c:1893:server_fxattrop_cbk] 0-c_glusterfs-server: 174: FXATTROP 1 (df00d265-4b67-415d-95a7-f9e4cac83029) ==> (No space left on device) [No space left on device]
[2016-03-22 12:25:20.059778] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:25:20.059830] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-2541-2016/03/22-12:25:16:484538-c_glusterfs-client-417-0-0 (version: 3.7.6)
[2016-03-22 12:25:20.453068] I [glusterfsd-mgmt.c:57:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2016-03-22 12:25:20.534272] I [glusterfsd-mgmt.c:1513:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2016-03-22 12:25:20.541953] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:25:20.541995] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-3863-2016/03/22-12:20:24:176727-c_glusterfs-client-417-2-0 (version: 3.7.6)
[2016-03-22 12:25:20.611103] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-3863-2016/03/22-12:20:24:176727-c_glusterfs-client-417-0-0
[2016-03-22 12:25:20.611183] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /logfiles/systemlog/syslog
[2016-03-22 12:25:20.611203] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1730627_P91A006.jar
[2016-03-22 12:25:20.611272] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1736858_P91A024.jar
[2016-03-22 12:25:20.611322] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1721555_P91A015.jar
[2016-03-22 12:25:20.611370] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1734470_P91A006.jar
[2016-03-22 12:25:20.611425] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1723048_P91A009.jar
[2016-03-22 12:25:20.611483] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1721371_P91A006.jar
[2016-03-22 12:25:20.611529] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1726067_P91A008.jar
[2016-03-22 12:25:20.611573] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1724100_P91A037.jar
[2016-03-22 12:25:20.611618] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1721553_P91A027.jar
[2016-03-22 12:25:20.611675] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1734156_P91A006.jar
[2016-03-22 12:25:20.611718] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/CXC1721218_P91A015.jar
[2016-03-22 12:25:20.611760] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1328917_P91A025.jar
[2016-03-22 12:25:20.611813] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1720482_P91A009.jar
[2016-03-22 12:25:20.611856] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1730437_P91A034.jar
[2016-03-22 12:25:20.611899] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1735764_P91A006.jar
[2016-03-22 12:25:20.611947] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1734543_P91A011.jar
[2016-03-22 12:25:20.611990] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/CXC1721282_P91A136.jar
[2016-03-22 12:25:20.612033] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1721203_P91A009.jar
[2016-03-22 12:25:20.612081] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/CXC1725907_P91A020.jar
[2016-03-22 12:25:20.612127] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1733632_P91A011.jar
[2016-03-22 12:25:20.612237] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1721199_P91A012.jar
[2016-03-22 12:25:20.612281] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1721208_P91A006.jar
[2016-03-22 12:25:20.612323] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1720327_P91A013.jar
[2016-03-22 12:25:20.612366] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1721216_P91A009.jar
[2016-03-22 12:25:20.612421] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1320960_P91A038.jar
[2016-03-22 12:25:20.612488] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-3863-2016/03/22-12:20:24:176727-c_glusterfs-client-417-0-0
[2016-03-22 12:25:20.760143] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-2541-2016/03/22-12:25:16:484538-c_glusterfs-client-417-0-0
[2016-03-22 12:25:20.760249] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-2541-2016/03/22-12:25:16:484538-c_glusterfs-client-417-0-0
[2016-03-22 12:25:23.261871] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-2688-2016/03/22-12:20:08:789697-c_glusterfs-client-417-0-0
[2016-03-22 12:25:23.261963] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-2688-2016/03/22-12:20:08:789697-c_glusterfs-client-417-0-0
[2016-03-22 12:25:24.312762] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:25:24.312805] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-9822-2016/03/22-12:25:24:267102-c_glusterfs-client-417-0-0 (version: 3.7.6)
[2016-03-22 12:25:24.393965] I [glusterfsd-mgmt.c:57:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2016-03-22 12:25:24.484889] I [glusterfsd-mgmt.c:1513:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2016-03-22 12:25:24.487663] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:25:24.487695] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-3016-2016/03/22-12:25:24:376927-c_glusterfs-client-417-0-0 (version: 3.7.6)
[2016-03-22 12:25:24.498684] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:25:24.498721] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-3863-2016/03/22-12:20:24:176727-c_glusterfs-client-417-4-0 (version: 3.7.6)
[2016-03-22 12:25:24.770833] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:25:24.770872] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-9876-2016/03/22-12:25:24:721512-c_glusterfs-client-417-0-0 (version: 3.7.6)
[2016-03-22 12:25:24.850416] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:25:24.850469] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-3084-2016/03/22-12:25:24:794681-c_glusterfs-client-417-0-0 (version: 3.7.6)
[2016-03-22 12:25:24.971053] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-3863-2016/03/22-12:20:24:176727-c_glusterfs-client-417-2-0
[2016-03-22 12:25:24.971140] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/CXC1733266%2_P91A006.jar
[2016-03-22 12:25:24.971190] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/CXC1721210_P91A018.jar
[2016-03-22 12:25:24.971258] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1737515_P91A006.jar
[2016-03-22 12:25:24.971384] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/CXC1736660_P91A351.jar
[2016-03-22 12:25:24.971448] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1721196_P91A010.jar
[2016-03-22 12:25:24.971501] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1729990_P91A013.jar
[2016-03-22 12:25:24.971563] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1723378_P91A006.jar
[2016-03-22 12:25:24.971658] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1721558_P91A015.jar
[2016-03-22 12:25:24.971757] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/CXC1322157%3_P91A006.jar
[2016-03-22 12:25:24.971800] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/CXC1729858_P91A006.jar
[2016-03-22 12:25:24.971860] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/CXC1730438_P91A024.jar
[2016-03-22 12:25:24.971903] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/CXC1721206_P91A016.jar
[2016-03-22 12:25:24.971998] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /java/install/CXC1721212_P91A013.jar
[2016-03-22 12:25:24.972090] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-3863-2016/03/22-12:20:24:176727-c_glusterfs-client-417-2-0
[2016-03-22 12:25:28.249872] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-9876-2016/03/22-12:25:24:721512-c_glusterfs-client-417-0-0
[2016-03-22 12:25:28.249937] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-9876-2016/03/22-12:25:24:721512-c_glusterfs-client-417-0-0
[2016-03-22 12:25:28.327665] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:25:28.327704] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-10534-2016/03/22-12:25:28:282378-c_glusterfs-client-417-0-0 (version: 3.7.6)
[2016-03-22 12:25:32.241255] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-10534-2016/03/22-12:25:28:282378-c_glusterfs-client-417-0-0
[2016-03-22 12:25:32.241346] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-10534-2016/03/22-12:25:28:282378-c_glusterfs-client-417-0-0
[2016-03-22 12:25:32.457526] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:25:32.457568] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-3445-2016/03/22-12:25:32:413154-c_glusterfs-client-417-0-0 (version: 3.7.6)
[2016-03-22 12:25:34.517242] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-3445-2016/03/22-12:25:32:413154-c_glusterfs-client-417-0-0
[2016-03-22 12:25:34.517325] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-3445-2016/03/22-12:25:32:413154-c_glusterfs-client-417-0-0
[2016-03-22 12:25:38.538856] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:25:38.538906] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-5234-2016/03/22-12:25:38:493848-c_glusterfs-client-417-0-0 (version: 3.7.6)
[2016-03-22 12:25:42.240153] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-5234-2016/03/22-12:25:38:493848-c_glusterfs-client-417-0-0
[2016-03-22 12:25:42.240227] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-5234-2016/03/22-12:25:38:493848-c_glusterfs-client-417-0-0
[2016-03-22 12:25:42.516299] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:25:42.516341] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-12305-2016/03/22-12:25:42:465431-c_glusterfs-client-417-0-0 (version: 3.7.6)
[2016-03-22 12:25:44.576225] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-12305-2016/03/22-12:25:42:465431-c_glusterfs-client-417-0-0
[2016-03-22 12:25:44.576313] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-12305-2016/03/22-12:25:42:465431-c_glusterfs-client-417-0-0
[2016-03-22 12:25:58.114999] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:25:58.115037] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-5536-2016/03/22-12:25:58:72993-c_glusterfs-client-417-0-0 (version: 3.7.6)
[2016-03-22 12:26:01.241231] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-5536-2016/03/22-12:25:58:72993-c_glusterfs-client-417-0-0
[2016-03-22 12:26:01.241308] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-5536-2016/03/22-12:25:58:72993-c_glusterfs-client-417-0-0
[2016-03-22 12:26:01.506886] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:26:01.506924] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-17300-2016/03/22-12:26:01:464261-c_glusterfs-client-417-0-0 (version: 3.7.6)
[2016-03-22 12:26:03.566230] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-17300-2016/03/22-12:26:01:464261-c_glusterfs-client-417-0-0
[2016-03-22 12:26:03.566346] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-17300-2016/03/22-12:26:01:464261-c_glusterfs-client-417-0-0
[2016-03-22 12:26:04.621314] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:26:04.621358] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-5810-2016/03/22-12:26:04:577106-c_glusterfs-client-417-0-0 (version: 3.7.6)
[2016-03-22 12:26:06.676961] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-5810-2016/03/22-12:26:04:577106-c_glusterfs-client-417-0-0
[2016-03-22 12:26:06.677068] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-5810-2016/03/22-12:26:04:577106-c_glusterfs-client-417-0-0
[2016-03-22 12:26:06.941048] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:26:06.941085] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-17561-2016/03/22-12:26:06:899162-c_glusterfs-client-417-0-0 (version: 3.7.6)
[2016-03-22 12:26:09.242757] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-17561-2016/03/22-12:26:06:899162-c_glusterfs-client-417-0-0
[2016-03-22 12:26:09.242826] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-17561-2016/03/22-12:26:06:899162-c_glusterfs-client-417-0-0
$ 

EVOA_8300-10_6B> lhsh 002500 cat /var/log/glusterfs/cli.log

160322-13:32:10 10.67.29.90 16.0c CPP_MOM-CPP-LSV184-gen1_gen1_COMPLETE stopfile=/tmp/13910
$ lhsh 002500 cat /var/log/glusterfs/cli.log
[2016-03-22 12:20:08.600498] I [cli.c:721:main] 0-cli: Started running gluster with version 3.7.6
[2016-03-22 12:20:08.614437] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:20:08.615193] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:20:08.615653] I [cli-rpc-ops.c:769:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2016-03-22 12:20:08.615926] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:20:08.653380] I [cli.c:721:main] 0-cli: Started running gluster with version 3.7.6
[2016-03-22 12:20:08.666063] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:20:08.666658] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:20:08.667143] I [cli-rpc-ops.c:769:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2016-03-22 12:20:08.667409] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:20:28.581626] I [cli.c:721:main] 0-cli: Started running /usr/sbin/gluster with version 3.7.6
[2016-03-22 12:20:28.593913] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:20:28.594612] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:20:30.696444] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:25:20.157727] I [cli.c:721:main] 0-cli: Started running gluster with version 3.7.6
[2016-03-22 12:25:20.173262] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:20.173424] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:20.175848] I [cli-rpc-ops.c:769:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2016-03-22 12:25:20.176156] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:25:20.218663] I [cli.c:721:main] 0-cli: Started running gluster with version 3.7.6
[2016-03-22 12:25:20.235067] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:20.235182] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:20.237552] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:25:20.275045] I [cli.c:721:main] 0-cli: Started running gluster with version 3.7.6
[2016-03-22 12:25:20.287918] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:20.288332] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:20.291475] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:25:20.328698] I [cli.c:721:main] 0-cli: Started running gluster with version 3.7.6
[2016-03-22 12:25:20.344160] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:20.344623] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:20.379359] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:25:20.417108] I [cli.c:721:main] 0-cli: Started running gluster with version 3.7.6
[2016-03-22 12:25:20.429507] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:20.430426] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:20.644046] I [cli-rpc-ops.c:2695:gf_cli_remove_brick_cbk] 0-cli: Received resp to remove brick
[2016-03-22 12:25:20.644221] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:25:20.678474] I [cli.c:721:main] 0-cli: Started running gluster with version 3.7.6
[2016-03-22 12:25:20.690882] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:20.691015] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:20.692067] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:25:20.730608] I [cli.c:721:main] 0-cli: Started running gluster with version 3.7.6
[2016-03-22 12:25:20.744845] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:20.745304] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:20.749580] I [cli-rpc-ops.c:189:gf_cli_deprobe_cbk] 0-cli: Received resp to deprobe
[2016-03-22 12:25:20.749690] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:25:20.789240] I [cli.c:721:main] 0-cli: Started running gluster with version 3.7.6
[2016-03-22 12:25:20.802885] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:20.803239] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:20.805630] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:25:20.843718] I [cli.c:721:main] 0-cli: Started running gluster with version 3.7.6
[2016-03-22 12:25:20.857070] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:20.857210] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:20.859969] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:25:20.899873] I [cli.c:721:main] 0-cli: Started running gluster with version 3.7.6
[2016-03-22 12:25:20.912893] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:20.913327] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:22.047678] I [cli-rpc-ops.c:137:gf_cli_probe_cbk] 0-cli: Received resp to probe
[2016-03-22 12:25:22.047863] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:25:23.084311] I [cli.c:721:main] 0-cli: Started running gluster with version 3.7.6
[2016-03-22 12:25:23.097375] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:23.097808] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:23.100364] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:25:23.136124] I [cli.c:721:main] 0-cli: Started running gluster with version 3.7.6
[2016-03-22 12:25:23.149665] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:23.149814] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:23.153047] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:25:23.189145] I [cli.c:721:main] 0-cli: Started running gluster with version 3.7.6
[2016-03-22 12:25:23.202420] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:23.202556] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:24.601684] I [cli-rpc-ops.c:2465:gf_cli_add_brick_cbk] 0-cli: Received resp to add brick
[2016-03-22 12:25:24.601828] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:25:24.646962] I [cli.c:721:main] 0-cli: Started running gluster with version 3.7.6
[2016-03-22 12:25:24.659541] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:24.659888] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:24.662752] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:25:24.673935] I [cli.c:721:main] 0-cli: Started running gluster with version 3.7.6
[2016-03-22 12:25:24.686246] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:24.686354] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:24.688676] I [cli-rpc-ops.c:769:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2016-03-22 12:25:24.689012] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:25:24.702038] I [cli.c:721:main] 0-cli: Started running gluster with version 3.7.6
[2016-03-22 12:25:24.714663] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:24.715555] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:27.714097] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:28.250424] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:25:28.261786] I [cli.c:721:main] 0-cli: Started running /usr/sbin/gluster with version 3.7.6
[2016-03-22 12:25:28.274194] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:28.274959] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:31.273627] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:32.241788] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:25:37.746634] I [cli.c:721:main] 0-cli: Started running gluster with version 3.7.6
[2016-03-22 12:25:37.759516] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:37.759898] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:37.763454] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:25:37.775164] I [cli.c:721:main] 0-cli: Started running gluster with version 3.7.6
[2016-03-22 12:25:37.787816] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:37.788266] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:37.791530] I [cli-rpc-ops.c:769:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2016-03-22 12:25:37.791831] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:25:42.443856] I [cli.c:721:main] 0-cli: Started running /usr/sbin/gluster with version 3.7.6
[2016-03-22 12:25:42.457199] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:42.458008] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:44.576729] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:25:57.375497] I [cli.c:721:main] 0-cli: Started running gluster with version 3.7.6
[2016-03-22 12:25:57.387669] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:57.388030] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:57.390453] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:25:57.402456] I [cli.c:721:main] 0-cli: Started running gluster with version 3.7.6
[2016-03-22 12:25:57.414674] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:57.415056] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:57.417029] I [cli-rpc-ops.c:769:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2016-03-22 12:25:57.417329] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:26:01.445279] I [cli.c:721:main] 0-cli: Started running /usr/sbin/gluster with version 3.7.6
[2016-03-22 12:26:01.457789] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:26:01.458535] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:26:03.566770] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:26:03.882164] I [cli.c:721:main] 0-cli: Started running gluster with version 3.7.6
[2016-03-22 12:26:03.894828] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:26:03.895178] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:26:03.897702] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:26:03.909724] I [cli.c:721:main] 0-cli: Started running gluster with version 3.7.6
[2016-03-22 12:26:03.922158] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:26:03.922557] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:26:03.925431] I [cli-rpc-ops.c:769:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2016-03-22 12:26:03.925732] I [input.c:36:cli_batch] 0-: Exiting with: 0
[2016-03-22 12:26:06.880429] I [cli.c:721:main] 0-cli: Started running /usr/sbin/gluster with version 3.7.6
[2016-03-22 12:26:06.892832] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:26:06.893649] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:26:09.243260] I [input.c:36:cli_batch] 0-: Exiting with: 0
$ 

EVOA_8300-10_6B> lhsh 002500 cat /var/log/glusterfs/cmd_history.log

160322-13:32:11 10.67.29.90 16.0c CPP_MOM-CPP-LSV184-gen1_gen1_COMPLETE stopfile=/tmp/13910
$ lhsh 002500 cat /var/log/glusterfs/cmd_history.log
[2016-03-22 12:25:20.356493]  : volume status : SUCCESS
[2016-03-22 12:25:20.378752]  : volume status : SUCCESS
[2016-03-22 12:25:20.643359]  : volume remove-brick c_glusterfs replica 1 10.32.0.48:/opt/lvmdir/c2/brick force : SUCCESS
[2016-03-22 12:25:20.749424]  : peer detach 10.32.0.48 : SUCCESS    
[2016-03-22 12:25:22.047372]  : peer probe 10.32.0.48 : SUCCESS    
[2016-03-22 12:25:24.601439]  : volume add-brick c_glusterfs replica 2 10.32.0.48:/opt/lvmdir/c2/brick force : SUCCESS
$ 

EVOA_8300-10_6B> lhsh 002500 cat /var/log/glusterfs/etc-glusterfs-glusterd.vol.log

160322-13:32:11 10.67.29.90 16.0c CPP_MOM-CPP-LSV184-gen1_gen1_COMPLETE stopfile=/tmp/13910
$ lhsh 002500 cat /var/log/glusterfs/etc-glusterfs-glusterd.vol.log
[2016-03-22 12:20:07.087112] I [MSGID: 100030] [glusterfsd.c:2318:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.7.6 (args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level DEBUG)
[2016-03-22 12:20:07.089550] D [MSGID: 0] [glusterfsd.c:1984:glusterfs_pidfile_update] 0-glusterfsd: pidfile /var/run/glusterd.pid updated with pid 2329
[2016-03-22 12:20:07.089726] D [logging.c:1790:__gf_log_inject_timer_event] 0-logging-infra: Starting timer now. Timeout = 120, current buf size = 5
[2016-03-22 12:20:07.090409] D [MSGID: 0] [glusterfsd.c:638:get_volfp] 0-glusterfsd: loading volume file /etc/glusterfs/glusterd.vol
[2016-03-22 12:20:07.096564] I [MSGID: 106478] [glusterd.c:1350:init] 0-management: Maximum allowed open file descriptors set to 65536
[2016-03-22 12:20:07.099198] I [MSGID: 106479] [glusterd.c:1399:init] 0-management: Using /system/glusterd as working directory
[2016-03-22 12:20:07.100015] D [MSGID: 0] [glusterd.c:410:glusterd_rpcsvc_options_build] 0-glusterd: listen-backlog value: 128
[2016-03-22 12:20:07.100410] D [rpcsvc.c:2317:rpcsvc_init] 0-rpc-service: RPC service inited.
[2016-03-22 12:20:07.100464] D [rpcsvc.c:1874:rpcsvc_program_register] 0-rpc-service: New program registered: GF-DUMP, Num: 123451501, Ver: 1, Port: 0
[2016-03-22 12:20:07.100570] D [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3.7.6/rpc-transport/socket.so
[2016-03-22 12:20:07.113393] D [socket.c:3845:socket_init] 0-socket.management: Configued transport.tcp-user-timeout=0
[2016-03-22 12:20:07.113459] D [socket.c:3928:socket_init] 0-socket.management: SSL support on the I/O path is NOT enabled
[2016-03-22 12:20:07.113482] D [socket.c:3931:socket_init] 0-socket.management: SSL support for glusterd is NOT enabled
[2016-03-22 12:20:07.113504] D [socket.c:3948:socket_init] 0-socket.management: using system polling thread
[2016-03-22 12:20:07.113551] D [name.c:564:server_fill_address_family] 0-socket.management: option address-family not specified, defaulting to inet
[2016-03-22 12:20:07.113888] D [rpcsvc.c:1874:rpcsvc_program_register] 0-rpc-service: New program registered: GlusterD svc peer, Num: 1238437, Ver: 2, Port: 0
[2016-03-22 12:20:07.113945] D [rpcsvc.c:1874:rpcsvc_program_register] 0-rpc-service: New program registered: GlusterD svc cli read-only, Num: 1238463, Ver: 2, Port: 0
[2016-03-22 12:20:07.113972] D [rpcsvc.c:1874:rpcsvc_program_register] 0-rpc-service: New program registered: GlusterD svc mgmt, Num: 1238433, Ver: 2, Port: 0
[2016-03-22 12:20:07.113996] D [rpcsvc.c:1874:rpcsvc_program_register] 0-rpc-service: New program registered: GlusterD svc mgmt v3, Num: 1238433, Ver: 3, Port: 0
[2016-03-22 12:20:07.114026] D [rpcsvc.c:1874:rpcsvc_program_register] 0-rpc-service: New program registered: Gluster Portmap, Num: 34123456, Ver: 1, Port: 0
[2016-03-22 12:20:07.114053] D [rpcsvc.c:1874:rpcsvc_program_register] 0-rpc-service: New program registered: Gluster Handshake, Num: 14398633, Ver: 2, Port: 0
[2016-03-22 12:20:07.114089] D [rpcsvc.c:1874:rpcsvc_program_register] 0-rpc-service: New program registered: Gluster MGMT Handshake, Num: 1239873, Ver: 1, Port: 0
[2016-03-22 12:20:07.114202] D [rpcsvc.c:2317:rpcsvc_init] 0-rpc-service: RPC service inited.
[2016-03-22 12:20:07.114227] D [rpcsvc.c:1874:rpcsvc_program_register] 0-rpc-service: New program registered: GF-DUMP, Num: 123451501, Ver: 1, Port: 0
[2016-03-22 12:20:07.114278] D [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3.7.6/rpc-transport/socket.so
[2016-03-22 12:20:07.114420] D [socket.c:3794:socket_init] 0-socket.management: disabling nodelay
[2016-03-22 12:20:07.114448] D [socket.c:3845:socket_init] 0-socket.management: Configued transport.tcp-user-timeout=0
[2016-03-22 12:20:07.114480] D [socket.c:3928:socket_init] 0-socket.management: SSL support on the I/O path is NOT enabled
[2016-03-22 12:20:07.114504] D [socket.c:3931:socket_init] 0-socket.management: SSL support for glusterd is NOT enabled
[2016-03-22 12:20:07.114525] D [socket.c:3948:socket_init] 0-socket.management: using system polling thread
[2016-03-22 12:20:07.114691] D [rpcsvc.c:1874:rpcsvc_program_register] 0-rpc-service: New program registered: GlusterD svc cli, Num: 1238463, Ver: 2, Port: 0
[2016-03-22 12:20:07.114720] D [rpcsvc.c:1874:rpcsvc_program_register] 0-rpc-service: New program registered: Gluster Handshake (CLI Getspec), Num: 14398633, Ver: 2, Port: 0
[2016-03-22 12:20:07.114825] D [MSGID: 0] [glusterd-utils.c:5974:glusterd_sm_tr_log_init] 0-glusterd: returning 0
[2016-03-22 12:20:07.114866] D [MSGID: 0] [glusterd.c:1691:init] 0-management: cannot get run-with-valgrind value
[2016-03-22 12:20:07.119496] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:20:07.119560] D [MSGID: 0] [store.c:439:gf_store_handle_retrieve] 0-: Returning 0
[2016-03-22 12:20:07.119976] D [MSGID: 0] [store.c:307:gf_store_retrieve_value] 0-: key operating-version found
[2016-03-22 12:20:07.120065] I [MSGID: 106513] [glusterd-store.c:2047:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30706
[2016-03-22 12:20:07.120548] D [MSGID: 0] [glusterd-utils.c:512:glusterd_volinfo_new] 0-management: Returning 0
[2016-03-22 12:20:07.128309] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:20:07.128384] D [MSGID: 0] [store.c:439:gf_store_handle_retrieve] 0-: Returning 0
[2016-03-22 12:20:07.128462] D [MSGID: 0] [store.c:501:gf_store_iter_new] 0-: Returning with 0
[2016-03-22 12:20:07.128796] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.128829] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = type value = 2
[2016-03-22 12:20:07.128876] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.128931] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = count value = 2
[2016-03-22 12:20:07.128984] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.129015] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = status value = 1
[2016-03-22 12:20:07.129063] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.129094] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = sub_count value = 2
[2016-03-22 12:20:07.129139] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.129170] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = stripe_count value = 1
[2016-03-22 12:20:07.129212] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.129243] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = replica_count value = 2
[2016-03-22 12:20:07.129287] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.129318] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = disperse_count value = 0
[2016-03-22 12:20:07.129361] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.129391] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = redundancy_count value = 0
[2016-03-22 12:20:07.129435] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.129465] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = version value = 2273
[2016-03-22 12:20:07.129508] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.129539] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = transport-type value = 0
[2016-03-22 12:20:07.129582] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.129614] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = volume-id value = 0992c376-e66e-4bc4-a0c7-2ca28f894aa2
[2016-03-22 12:20:07.129728] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.129761] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = username value = 06afe471-c710-4493-9ab0-b74b9e66f01f
[2016-03-22 12:20:07.129814] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.129845] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = password value = 6440d9dd-6ccb-44a1-a0eb-c6898ca84178
[2016-03-22 12:20:07.129890] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.129938] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = op-version value = 3
[2016-03-22 12:20:07.129993] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.130025] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = client-op-version value = 3
[2016-03-22 12:20:07.130069] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.130100] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = quota-version value = 0
[2016-03-22 12:20:07.130149] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.130182] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = parent_volname value = N/A
[2016-03-22 12:20:07.130231] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.130261] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = restored_from_snap value = 00000000-0000-0000-0000-000000000000
[2016-03-22 12:20:07.130310] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.130345] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = snap-max-hard-limit value = 256
[2016-03-22 12:20:07.130397] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.130429] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = performance.readdir-ahead value = on
[2016-03-22 12:20:07.130527] D [MSGID: 0] [glusterd-store.c:2725:glusterd_store_update_volinfo] 0-management: Parsed as Volume-set:key=performance.readdir-ahead,value:on
[2016-03-22 12:20:07.130573] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.130606] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = network.ping-timeout value = 4
[2016-03-22 12:20:07.130649] D [MSGID: 0] [glusterd-store.c:2725:glusterd_store_update_volinfo] 0-management: Parsed as Volume-set:key=network.ping-timeout,value:4
[2016-03-22 12:20:07.130699] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.130731] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = nfs.disable value = on
[2016-03-22 12:20:07.130804] D [MSGID: 0] [glusterd-store.c:2725:glusterd_store_update_volinfo] 0-management: Parsed as Volume-set:key=nfs.disable,value:on
[2016-03-22 12:20:07.130854] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.130888] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = brick-0 value = 10.32.0.48:-opt-lvmdir-c2-brick
[2016-03-22 12:20:07.130957] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.130991] D [MSGID: 0] [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-management: key = brick-1 value = 10.32.1.144:-opt-lvmdir-c2-brick
[2016-03-22 12:20:07.131050] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with -1
[2016-03-22 12:20:07.131181] D [MSGID: 0] [store.c:501:gf_store_iter_new] 0-: Returning with 0
[2016-03-22 12:20:07.131258] D [MSGID: 0] [glusterd-utils.c:884:glusterd_brickinfo_new] 0-management: Returning 0
[2016-03-22 12:20:07.131353] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.132123] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0" repeated 22 times between [2016-03-22 12:20:07.131353] and [2016-03-22 12:20:07.131735]
[2016-03-22 12:20:07.132122] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:20:07.132228] D [MSGID: 0] [store.c:439:gf_store_handle_retrieve] 0-: Returning 0
[2016-03-22 12:20:07.132295] D [MSGID: 0] [store.c:501:gf_store_iter_new] 0-: Returning with 0
[2016-03-22 12:20:07.132610] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.132831] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0" repeated 7 times between [2016-03-22 12:20:07.132610] and [2016-03-22 12:20:07.132808]
[2016-03-22 12:20:07.132830] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with -1
[2016-03-22 12:20:07.133019] D [MSGID: 0] [glusterd-utils.c:884:glusterd_brickinfo_new] 0-management: Returning 0
[2016-03-22 12:20:07.133074] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:07.139699] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:20:07.139758] D [MSGID: 0] [store.c:439:gf_store_handle_retrieve] 0-: Returning 0
[2016-03-22 12:20:07.139823] D [MSGID: 0] [store.c:501:gf_store_iter_new] 0-: Returning with 0
[2016-03-22 12:20:07.140259] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:08.467827] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0" repeated 7 times between [2016-03-22 12:20:07.140259] and [2016-03-22 12:20:08.467804]
[2016-03-22 12:20:08.467825] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with -1
[2016-03-22 12:20:08.468022] D [MSGID: 0] [glusterd-store.c:2402:glusterd_store_retrieve_bricks] 0-management: Returning with 0
[2016-03-22 12:20:08.469179] D [MSGID: 0] [glusterd-utils.c:2052:glusterd_volume_compute_cksum] 0-management: Returning with 0
[2016-03-22 12:20:08.478260] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:20:08.478459] D [MSGID: 0] [glusterd-store.c:4376:glusterd_store_retrieve_quota_version] 0-management: Version absent
[2016-03-22 12:20:08.478497] D [MSGID: 0] [store.c:460:gf_store_handle_destroy] 0-: Returning 0
[2016-03-22 12:20:08.485685] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:20:08.485854] D [MSGID: 0] [glusterd-utils.c:2052:glusterd_volume_compute_cksum] 0-management: Returning with 0
[2016-03-22 12:20:08.492732] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:20:08.492967] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:20:08.508006] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:20:08.493059] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:20:08.508004] D [MSGID: 0] [store.c:460:gf_store_handle_destroy] 0-: Returning 0
[2016-03-22 12:20:08.508153] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:20:08.508180] D [MSGID: 0] [store.c:439:gf_store_handle_retrieve] 0-: Returning 0
[2016-03-22 12:20:08.508236] D [MSGID: 0] [store.c:501:gf_store_iter_new] 0-: Returning with 0
[2016-03-22 12:20:08.508579] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:08.508678] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0" repeated 3 times between [2016-03-22 12:20:08.508579] and [2016-03-22 12:20:08.508660]
[2016-03-22 12:20:08.508677] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with -1
[2016-03-22 12:20:08.508796] D [MSGID: 0] [glusterd-store.c:3067:glusterd_store_retrieve_volumes] 0-management: Returning with 0
[2016-03-22 12:20:08.516159] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:20:08.516207] D [MSGID: 0] [store.c:439:gf_store_handle_retrieve] 0-: Returning 0
[2016-03-22 12:20:08.516256] D [MSGID: 0] [store.c:501:gf_store_iter_new] 0-: Returning with 0
[2016-03-22 12:20:08.516612] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:08.516652] D [MSGID: 0] [glusterd-utils.c:5974:glusterd_sm_tr_log_init] 0-glusterd: returning 0
[2016-03-22 12:20:08.516696] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:08.516745] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:20:08.516733] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:08.516744] D [MSGID: 0] [glusterd-peer-utils.c:458:glusterd_peer_hostname_new] 0-glusterd: Returning 0
[2016-03-22 12:20:08.516821] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with -1
[2016-03-22 12:20:08.516892] I [MSGID: 106498] [glusterd-handler.c:3579:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0
[2016-03-22 12:20:08.517039] D [MSGID: 0] [glusterd-handler.c:3391:glusterd_transport_inet_options_build] 0-glusterd: Returning 0
[2016-03-22 12:20:08.517107] I [rpc-clnt.c:984:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2016-03-22 12:20:08.517130] D [rpc-clnt.c:996:rpc_clnt_connection_init] 0-management: setting ping-timeout to 30
[2016-03-22 12:20:08.517160] D [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3.7.6/rpc-transport/socket.so
[2016-03-22 12:20:08.517280] D [socket.c:3845:socket_init] 0-management: Configued transport.tcp-user-timeout=-1
[2016-03-22 12:20:08.517308] D [socket.c:3928:socket_init] 0-management: SSL support on the I/O path is NOT enabled
[2016-03-22 12:20:08.517327] D [socket.c:3931:socket_init] 0-management: SSL support for glusterd is NOT enabled
[2016-03-22 12:20:08.517346] D [socket.c:3948:socket_init] 0-management: using system polling thread
[2016-03-22 12:20:08.518766] D [MSGID: 0] [common-utils.c:333:gf_resolve_ip6] 0-resolver: returning ip-10.32.0.48 (port-24007) for hostname: 10.32.0.48 and port: 24007
[2016-03-22 12:20:08.518797] D [socket.c:2833:socket_fix_ssl_opts] 0-management: disabling SSL for portmapper connection
[2016-03-22 12:20:08.518846] W [socket.c:869:__socket_keepalive] 0-socket: failed to set TCP_USER_TIMEOUT -1000 on socket 13, Invalid argument
[2016-03-22 12:20:08.518869] E [socket.c:2965:socket_connect] 0-management: Failed to set keep-alive: Invalid argument
[2016-03-22 12:20:08.519247] D [MSGID: 0] [glusterd-handler.c:3323:glusterd_rpc_create] 0-management: returning 0
[2016-03-22 12:20:08.519286] D [MSGID: 0] [glusterd-store.c:4118:glusterd_store_retrieve_peers] 0-management: Returning with 0
[2016-03-22 12:20:08.519723] D [MSGID: 0] [glusterd-store.c:3501:glusterd_store_retrieve_missed_snaps_list] 0-management: EOF for missed_snap_list
[2016-03-22 12:20:08.519758] D [MSGID: 0] [glusterd-store.c:3607:glusterd_store_retrieve_snaps] 0-management: Returning with 0
[2016-03-22 12:20:08.519796] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.0.48 found.. state: 3
[2016-03-22 12:20:08.519825] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:20:08.519859] D [MSGID: 0] [glusterd-utils.c:925:glusterd_resolve_brick] 0-management: Returning 0
[2016-03-22 12:20:08.520165] D [MSGID: 0] [glusterd-peer-utils.c:133:glusterd_peerinfo_find_by_hostname] 0-management: Unable to find friend: 10.32.1.144
[2016-03-22 12:20:08.520266] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.1.144 
[2016-03-22 12:20:08.520928] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.1.144 is local address at interface efd
[2016-03-22 12:20:08.521000] D [MSGID: 0] [store.c:307:gf_store_retrieve_value] 0-: key UUID found
[2016-03-22 12:20:08.521070] D [MSGID: 0] [glusterd-store.c:2122:glusterd_retrieve_uuid] 0-management: Returning 0
[2016-03-22 12:20:08.521129] I [MSGID: 106544] [glusterd.c:159:glusterd_uuid_init] 0-management: retrieved UUID: 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:20:08.521158] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:20:08.521185] D [MSGID: 0] [glusterd-utils.c:925:glusterd_resolve_brick] 0-management: Returning 0
[2016-03-22 12:20:08.521218] D [MSGID: 0] [glusterd-store.c:4339:glusterd_restore] 0-management: Returning 0
[2016-03-22 12:20:08.527783] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:20:08.527814] D [MSGID: 0] [store.c:439:gf_store_handle_retrieve] 0-: Returning 0
[2016-03-22 12:20:08.527860] D [MSGID: 0] [store.c:501:gf_store_iter_new] 0-: Returning with 0
[2016-03-22 12:20:08.528170] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with 0
[2016-03-22 12:20:08.528212] D [MSGID: 0] [store.c:613:gf_store_iter_get_next] 0-: Returning with -1
[2016-03-22 12:20:08.528272] D [MSGID: 0] [store.c:460:gf_store_handle_destroy] 0-: Returning 0
Final graph:
[2016-03-22 12:20:08.528491] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
+------------------------------------------------------------------------------+
  1: volume management
  2:     type mgmt/glusterd
  3:     option rpc-auth.auth-glusterfs on
  4:     option rpc-auth.auth-unix on
  5:     option rpc-auth.auth-null on
  6:     option rpc-auth-allow-insecure on
  7:     option transport.socket.listen-backlog 128
  8:     option ping-timeout 30
  9:     option transport.socket.read-fail-log off
 10:     option transport.socket.keepalive-interval 2
 11:     option transport.socket.keepalive-time 10
 12:     option transport-type socket
 13:     option working-directory /system/glusterd
 14: end-volume
 15:  
+------------------------------------------------------------------------------+
[2016-03-22 12:20:08.528747] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:08.528792] D [MSGID: 0] [glusterd-handler.c:5042:__glusterd_peer_rpc_notify] 0-management: got RPC_CLNT_CONNECT
[2016-03-22 12:20:08.528892] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:08.529824] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8b729f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff8b6d3990] (--> /usr/lib64/libgfrpc.so.0(+0x1a188)[0x3fff8b6d4188] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_submit-0x1a4d8)[0x3fff8b6cf0a8] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_submit_request-0x129078)[0x3fff88074de8] ))))) 0-: 10.32.0.48:24007: ping timer event already removed
[2016-03-22 12:20:08.530164] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:20:08.530215] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:08.530239] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2016-03-22 12:20:08.530266] D [MSGID: 0] [glusterd-handshake.c:2098:__glusterd_peer_dump_version_cbk] 0-management: Proceeding to op-version handshake with peer 10.32.0.48
[2016-03-22 12:20:08.530365] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:08.530943] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:20:08.530919] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:08.530942] D [MSGID: 0] [glusterd-handshake.c:1660:gd_validate_peer_op_version] 0-management: Peer 10.32.0.48 accepted
[2016-03-22 12:20:08.546174] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:08.546254] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:20:08.546247] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:08.546254] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_CONNECTED'
[2016-03-22 12:20:08.546330] D [MSGID: 0] [glusterd-handshake.c:1613:glusterd_event_connected_inject] 0-glusterd: returning 0
[2016-03-22 12:20:08.546383] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:08.546417] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_CONNECTED'
[2016-03-22 12:20:08.546449] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:08.546876] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:20:08.546486] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:08.546876] D [MSGID: 0] [glusterd-utils.c:2504:glusterd_add_volume_to_dict] 0-management: Returning with 0
[2016-03-22 12:20:08.547437] D [MSGID: 0] [glusterd-rpc-ops.c:1579:glusterd_rpc_friend_add] 0-glusterd: Returning 0
[2016-03-22 12:20:08.547467] D [MSGID: 0] [glusterd-sm.c:334:glusterd_ac_friend_add] 0-glusterd: Returning with 0
[2016-03-22 12:20:08.547499] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:08.547535] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Peer in Cluster' to 'Peer in Cluster' due to event 'GD_FRIEND_EVENT_CONNECTED'
[2016-03-22 12:20:08.547568] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:20:08.547596] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:08.547643] D [MSGID: 0] [glusterd-store.c:3802:glusterd_store_create_peer_dir] 0-glusterd: Returning with 0
[2016-03-22 12:20:08.547932] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:20:08.548100] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 2 times between [2016-03-22 12:20:08.547932] and [2016-03-22 12:20:08.548096]
[2016-03-22 12:20:08.548099] D [MSGID: 0] [glusterd-store.c:3931:glusterd_store_peer_write] 0-glusterd: Returning with 0
[2016-03-22 12:20:08.562936] D [MSGID: 0] [glusterd-store.c:3957:glusterd_store_perform_peer_store] 0-glusterd: Returning 0
[2016-03-22 12:20:08.562993] D [MSGID: 0] [glusterd-store.c:3978:glusterd_store_peerinfo] 0-glusterd: Returning with 0
[2016-03-22 12:20:08.615142] I [MSGID: 106488] [glusterd-handler.c:1472:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req
[2016-03-22 12:20:08.615254] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:20:08.615289] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:20:08.619065] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:20:08.619101] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:20:08.619151] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=3 from gen=1 to slot->gen=2, fd=7, slot->fd=7
[2016-03-22 12:20:08.666683] I [MSGID: 106488] [glusterd-handler.c:1472:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req
[2016-03-22 12:20:08.666769] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:20:08.666805] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:20:08.669435] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:20:08.669469] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:20:08.669507] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=3 from gen=4 to slot->gen=5, fd=7, slot->fd=7
[2016-03-22 12:20:08.750720] I [MSGID: 106493] [glusterd-rpc-ops.c:480:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 356dabed-4314-48be-9eef-d2711d01270d, host: 10.32.0.48, port: 0
[2016-03-22 12:20:08.750786] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:08.750821] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_RCVD_ACC'
[2016-03-22 12:20:08.750853] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:08.750882] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_RCVD_ACC'
[2016-03-22 12:20:08.750922] D [MSGID: 0] [glusterd-sm.c:203:glusterd_ac_none] 0-glusterd: Returning with 0
[2016-03-22 12:20:08.750952] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:08.750981] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Peer in Cluster' to 'Peer in Cluster' due to event 'GD_FRIEND_EVENT_RCVD_ACC'
[2016-03-22 12:20:08.751010] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:20:08.751039] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:08.751084] D [MSGID: 0] [glusterd-store.c:3802:glusterd_store_create_peer_dir] 0-glusterd: Returning with 0
[2016-03-22 12:20:08.751347] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:20:08.751501] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 2 times between [2016-03-22 12:20:08.751347] and [2016-03-22 12:20:08.751497]
[2016-03-22 12:20:08.751501] D [MSGID: 0] [glusterd-store.c:3931:glusterd_store_peer_write] 0-glusterd: Returning with 0
[2016-03-22 12:20:08.765967] D [MSGID: 0] [glusterd-store.c:3957:glusterd_store_perform_peer_store] 0-glusterd: Returning 0
[2016-03-22 12:20:08.766029] D [MSGID: 0] [glusterd-store.c:3978:glusterd_store_peerinfo] 0-glusterd: Returning with 0
[2016-03-22 12:20:08.767555] D [MSGID: 0] [glusterd-utils.c:173:glusterd_lock] 0-management: Cluster lock held by 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:20:08.768977] I [rpc-clnt.c:984:rpc_clnt_connection_init] 0-nfs: setting frame-timeout to 600
[2016-03-22 12:20:08.769022] D [rpc-clnt.c:1001:rpc_clnt_connection_init] 0-nfs: disable ping-timeout
[2016-03-22 12:20:08.769068] D [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3.7.6/rpc-transport/socket.so
[2016-03-22 12:20:08.769194] D [socket.c:3794:socket_init] 0-nfs: disabling nodelay
[2016-03-22 12:20:08.769227] D [socket.c:3845:socket_init] 0-nfs: Configued transport.tcp-user-timeout=0
[2016-03-22 12:20:08.769262] D [socket.c:3928:socket_init] 0-nfs: SSL support on the I/O path is NOT enabled
[2016-03-22 12:20:08.769290] D [socket.c:3931:socket_init] 0-nfs: SSL support for glusterd is NOT enabled
[2016-03-22 12:20:08.769314] D [socket.c:3948:socket_init] 0-nfs: using system polling thread
[2016-03-22 12:20:08.769411] D [MSGID: 0] [glusterd-svc-mgmt.c:109:glusterd_svc_init_common] 0-management: Returning 0
[2016-03-22 12:20:08.769454] D [MSGID: 0] [glusterd-nfs-svc.c:84:glusterd_nfssvc_manager] 0-management: nfs service initialized
[2016-03-22 12:20:08.769795] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already stopped
[2016-03-22 12:20:08.770454] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8b729f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff8b6d3990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff8b6cf430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff88141ab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff881428b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:20:08.770557] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:20:08.770594] D [MSGID: 0] [glusterd-nfs-svc.c:135:glusterd_nfssvc_stop] 0-management: Returning 0
[2016-03-22 12:20:08.770862] D [MSGID: 0] [glusterd-volgen.c:4809:build_nfs_graph] 0-glusterd: Returning 0
[2016-03-22 12:20:08.771559] D [MSGID: 0] [glusterd-nfs-svc.c:106:glusterd_nfssvc_manager] 0-management: Returning 0
[2016-03-22 12:20:08.772433] I [rpc-clnt.c:984:rpc_clnt_connection_init] 0-glustershd: setting frame-timeout to 600
[2016-03-22 12:20:08.772463] D [rpc-clnt.c:1001:rpc_clnt_connection_init] 0-glustershd: disable ping-timeout
[2016-03-22 12:20:08.772500] D [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3.7.6/rpc-transport/socket.so
[2016-03-22 12:20:08.772630] D [socket.c:3794:socket_init] 0-glustershd: disabling nodelay
[2016-03-22 12:20:08.772657] D [socket.c:3845:socket_init] 0-glustershd: Configued transport.tcp-user-timeout=0
[2016-03-22 12:20:08.772692] D [socket.c:3928:socket_init] 0-glustershd: SSL support on the I/O path is NOT enabled
[2016-03-22 12:20:08.772716] D [socket.c:3931:socket_init] 0-glustershd: SSL support for glusterd is NOT enabled
[2016-03-22 12:20:08.772739] D [socket.c:3948:socket_init] 0-glustershd: using system polling thread
[2016-03-22 12:20:08.772822] D [MSGID: 0] [glusterd-svc-mgmt.c:109:glusterd_svc_init_common] 0-management: Returning 0
[2016-03-22 12:20:08.772858] D [MSGID: 0] [glusterd-shd-svc.c:98:glusterd_shdsvc_manager] 0-management: shd service initialized
[2016-03-22 12:20:08.773309] D [MSGID: 0] [glusterd-volgen.c:881:_xl_link_children] 0-management: glustershd:c_glusterfs-replicate-0
[2016-03-22 12:20:08.774723] D [MSGID: 0] [glusterd-shd-svc.c:78:glusterd_shdsvc_create_volfile] 0-management: Returning 0
[2016-03-22 12:20:08.775745] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: glustershd already stopped
[2016-03-22 12:20:08.776396] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8b729f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff8b6d3990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff8b6cf430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff88141ab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff881428b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:20:08.776477] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:20:08.776586] D [MSGID: 0] [glusterd-svc-mgmt.c:195:glusterd_svc_start] 0-management: Starting glustershd service
[2016-03-22 12:20:08.785631] D [MSGID: 0] [glusterd-svc-mgmt.c:208:glusterd_svc_start] 0-management: Returning 0
[2016-03-22 12:20:08.785783] D [MSGID: 0] [glusterd-shd-svc.c:177:glusterd_shdsvc_start] 0-management: Returning 0
[2016-03-22 12:20:08.785931] W [socket.c:3009:socket_connect] 0-glustershd: Ignore failed connection attempt on , (No such file or directory) 
[2016-03-22 12:20:08.786053] D [MSGID: 0] [glusterd-shd-svc.c:139:glusterd_shdsvc_manager] 0-management: Returning 0
[2016-03-22 12:20:08.786616] I [rpc-clnt.c:984:rpc_clnt_connection_init] 0-quotad: setting frame-timeout to 600
[2016-03-22 12:20:08.786644] D [rpc-clnt.c:1001:rpc_clnt_connection_init] 0-quotad: disable ping-timeout
[2016-03-22 12:20:08.786683] D [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3.7.6/rpc-transport/socket.so
[2016-03-22 12:20:08.786827] D [socket.c:3794:socket_init] 0-quotad: disabling nodelay
[2016-03-22 12:20:08.786855] D [socket.c:3845:socket_init] 0-quotad: Configued transport.tcp-user-timeout=0
[2016-03-22 12:20:08.786884] D [socket.c:3928:socket_init] 0-quotad: SSL support on the I/O path is NOT enabled
[2016-03-22 12:20:08.786923] D [socket.c:3931:socket_init] 0-quotad: SSL support for glusterd is NOT enabled
[2016-03-22 12:20:08.786947] D [socket.c:3948:socket_init] 0-quotad: using system polling thread
[2016-03-22 12:20:08.787040] D [MSGID: 0] [glusterd-svc-mgmt.c:109:glusterd_svc_init_common] 0-management: Returning 0
[2016-03-22 12:20:08.787077] D [MSGID: 0] [glusterd-quotad-svc.c:72:glusterd_quotadsvc_manager] 0-management: quotad service initialized
[2016-03-22 12:20:08.787688] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: quotad already stopped
[2016-03-22 12:20:08.788382] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8b729f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff8b6d3990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff8b6cf430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff88141ab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff881428b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:20:08.788470] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:20:08.788506] D [MSGID: 0] [glusterd-quotad-svc.c:111:glusterd_quotadsvc_manager] 0-management: Returning 0
[2016-03-22 12:20:08.789031] I [rpc-clnt.c:984:rpc_clnt_connection_init] 0-bitd: setting frame-timeout to 600
[2016-03-22 12:20:08.789064] D [rpc-clnt.c:1001:rpc_clnt_connection_init] 0-bitd: disable ping-timeout
[2016-03-22 12:20:08.789103] D [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3.7.6/rpc-transport/socket.so
[2016-03-22 12:20:08.789228] D [socket.c:3794:socket_init] 0-bitd: disabling nodelay
[2016-03-22 12:20:08.789256] D [socket.c:3845:socket_init] 0-bitd: Configued transport.tcp-user-timeout=0
[2016-03-22 12:20:08.789294] D [socket.c:3928:socket_init] 0-bitd: SSL support on the I/O path is NOT enabled
[2016-03-22 12:20:08.789317] D [socket.c:3931:socket_init] 0-bitd: SSL support for glusterd is NOT enabled
[2016-03-22 12:20:08.789340] D [socket.c:3948:socket_init] 0-bitd: using system polling thread
[2016-03-22 12:20:08.789438] D [MSGID: 0] [glusterd-svc-mgmt.c:109:glusterd_svc_init_common] 0-management: Returning 0
[2016-03-22 12:20:08.789482] D [MSGID: 0] [glusterd-bitd-svc.c:84:glusterd_bitdsvc_manager] 0-management: BitD service initialized
[2016-03-22 12:20:08.790357] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: bitd already stopped
[2016-03-22 12:20:08.791012] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8b729f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff8b6d3990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff8b6cf430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff88141ab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff881428b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:20:08.791096] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:20:08.791133] D [MSGID: 0] [glusterd-bitd-svc.c:109:glusterd_bitdsvc_manager] 0-management: Returning 0
[2016-03-22 12:20:08.791716] I [rpc-clnt.c:984:rpc_clnt_connection_init] 0-scrub: setting frame-timeout to 600
[2016-03-22 12:20:08.791750] D [rpc-clnt.c:1001:rpc_clnt_connection_init] 0-scrub: disable ping-timeout
[2016-03-22 12:20:08.791785] D [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3.7.6/rpc-transport/socket.so
[2016-03-22 12:20:08.791966] D [socket.c:3794:socket_init] 0-scrub: disabling nodelay
[2016-03-22 12:20:08.791996] D [socket.c:3845:socket_init] 0-scrub: Configued transport.tcp-user-timeout=0
[2016-03-22 12:20:08.792025] D [socket.c:3928:socket_init] 0-scrub: SSL support on the I/O path is NOT enabled
[2016-03-22 12:20:08.792054] D [socket.c:3931:socket_init] 0-scrub: SSL support for glusterd is NOT enabled
[2016-03-22 12:20:08.792076] D [socket.c:3948:socket_init] 0-scrub: using system polling thread
[2016-03-22 12:20:08.792162] D [MSGID: 0] [glusterd-svc-mgmt.c:109:glusterd_svc_init_common] 0-management: Returning 0
[2016-03-22 12:20:08.792200] D [MSGID: 0] [glusterd-scrub-svc.c:79:glusterd_scrubsvc_manager] 0-management: scrub service initialized
[2016-03-22 12:20:08.792682] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: scrub already stopped
[2016-03-22 12:20:08.793353] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8b729f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff8b6d3990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff8b6cf430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff88141ab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff881428b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:20:08.793439] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:20:08.793476] D [MSGID: 0] [glusterd-scrub-svc.c:104:glusterd_scrubsvc_manager] 0-management: Returning 0
[2016-03-22 12:20:08.793508] D [MSGID: 0] [glusterd-utils.c:4564:glusterd_restart_bricks] 0-management: starting the volume c_glusterfs
[2016-03-22 12:20:08.793542] D [MSGID: 0] [glusterd-utils.c:4539:glusterd_brick_start] 0-management: returning 0 
[2016-03-22 12:20:08.804857] I [rpc-clnt.c:984:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2016-03-22 12:20:08.804981] D [rpc-clnt.c:1001:rpc_clnt_connection_init] 0-management: disable ping-timeout
[2016-03-22 12:20:08.805034] D [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3.7.6/rpc-transport/socket.so
[2016-03-22 12:20:08.805206] D [socket.c:3794:socket_init] 0-management: disabling nodelay
[2016-03-22 12:20:08.805234] D [socket.c:3845:socket_init] 0-management: Configued transport.tcp-user-timeout=0
[2016-03-22 12:20:08.805263] D [socket.c:3928:socket_init] 0-management: SSL support on the I/O path is NOT enabled
[2016-03-22 12:20:08.805285] D [socket.c:3931:socket_init] 0-management: SSL support for glusterd is NOT enabled
[2016-03-22 12:20:08.805307] D [socket.c:3948:socket_init] 0-management: using system polling thread
[2016-03-22 12:20:08.805487] D [MSGID: 0] [glusterd-handler.c:3323:glusterd_rpc_create] 0-management: returning 0
[2016-03-22 12:20:08.805536] D [MSGID: 0] [glusterd-utils.c:1572:glusterd_brick_connect] 0-glusterd: Returning 0
[2016-03-22 12:20:08.805593] D [MSGID: 0] [glusterd-utils.c:4539:glusterd_brick_start] 0-management: returning 0 
[2016-03-22 12:20:08.805645] D [MSGID: 0] [glusterd-snapshot-utils.c:3595:glusterd_is_snapd_enabled] 0-management: Key features.uss not present in the dict for volume c_glusterfs
[2016-03-22 12:20:08.806383] D [MSGID: 0] [glusterd-svc-mgmt.c:315:glusterd_svc_common_rpc_notify] 0-management: glustershd has connected with glusterd.
[2016-03-22 12:20:08.806545] W [socket.c:588:__socket_rwv] 0-glustershd: readv on /var/run/gluster/8426fb5fe867c36346144bff93f74572.socket failed (Invalid argument)
[2016-03-22 12:20:08.806572] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:20:08.806560] D [MSGID: 0] [glusterd-utils.c:1400:glusterd_volinfo_find_by_volume_id] 0-management: Volume c_glusterfs found
[2016-03-22 12:20:08.806673] D [MSGID: 0] [glusterd-utils.c:884:glusterd_brickinfo_new] 0-management: Returning 0
[2016-03-22 12:20:08.806717] D [MSGID: 0] [glusterd-utils.c:1041:glusterd_brickinfo_new_from_brick] 0-management: Returning 0
[2016-03-22 12:20:08.807005] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8b729f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff8b6d3990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup-0x19b20)[0x3fff8b6cfaf0] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_notify-0x18e68)[0x3fff8b6d0808] (--> /usr/lib64/libgfrpc.so.0(rpc_transport_notify-0x1d8bc)[0x3fff8b6cb85c] ))))) 0-: /var/run/gluster/8426fb5fe867c36346144bff93f74572.socket: ping timer event already removed
[2016-03-22 12:20:08.807229] D [MSGID: 0] [glusterd-peer-utils.c:133:glusterd_peerinfo_find_by_hostname] 0-management: Unable to find friend: 10.32.1.144
[2016-03-22 12:20:08.807365] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.1.144 
[2016-03-22 12:20:08.808190] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.1.144 is local address at interface efd
[2016-03-22 12:20:08.808250] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:20:08.808287] D [MSGID: 0] [glusterd-utils.c:1326:glusterd_volume_brickinfo_get] 0-management: Found brick 10.32.1.144:/opt/lvmdir/c2/brick in volume c_glusterfs
[2016-03-22 12:20:08.808322] D [MSGID: 0] [glusterd-utils.c:1335:glusterd_volume_brickinfo_get] 0-management: Returning 0
[2016-03-22 12:20:08.808355] D [MSGID: 0] [glusterd-utils.c:1359:glusterd_volume_brickinfo_get_by_brick] 0-glusterd: Returning 0
[2016-03-22 12:20:08.808393] D [MSGID: 0] [glusterd-handler.c:4897:__glusterd_brick_rpc_notify] 0-management: Connected to 10.32.1.144:/opt/lvmdir/c2/brick
[2016-03-22 12:20:08.808429] D [MSGID: 0] [glusterd-utils.c:4869:glusterd_set_brick_status] 0-glusterd: Setting brick 10.32.1.144:/opt/lvmdir/c2/brick status to started
[2016-03-22 12:20:08.808493] I [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:20:08.808529] I [MSGID: 106006] [glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: glustershd has disconnected from glusterd.
[2016-03-22 12:20:08.808603] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=3 from gen=7 to slot->gen=8, fd=7, slot->fd=7
[2016-03-22 12:20:08.808971] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8b729f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff8b6d3990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup-0x19b20)[0x3fff8b6cfaf0] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_notify-0x18e68)[0x3fff8b6d0808] (--> /usr/lib64/libgfrpc.so.0(rpc_transport_notify-0x1d8bc)[0x3fff8b6cb85c] ))))) 0-: /var/run/gluster/697c0e4a16ebc734cd06fd9150723005.socket: ping timer event already removed
[2016-03-22 12:20:08.809042] D [MSGID: 0] [glusterd-utils.c:1400:glusterd_volinfo_find_by_volume_id] 0-management: Volume c_glusterfs found
[2016-03-22 12:20:08.809103] D [MSGID: 0] [glusterd-utils.c:884:glusterd_brickinfo_new] 0-management: Returning 0
[2016-03-22 12:20:08.809149] D [MSGID: 0] [glusterd-utils.c:1041:glusterd_brickinfo_new_from_brick] 0-management: Returning 0
[2016-03-22 12:20:08.809569] D [MSGID: 0] [glusterd-peer-utils.c:133:glusterd_peerinfo_find_by_hostname] 0-management: Unable to find friend: 10.32.1.144
[2016-03-22 12:20:08.809713] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.1.144 
[2016-03-22 12:20:08.810537] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.1.144 is local address at interface efd
[2016-03-22 12:20:08.810592] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:20:08.810632] D [MSGID: 0] [glusterd-utils.c:1326:glusterd_volume_brickinfo_get] 0-management: Found brick 10.32.1.144:/opt/lvmdir/c2/brick in volume c_glusterfs
[2016-03-22 12:20:08.810665] D [MSGID: 0] [glusterd-utils.c:1335:glusterd_volume_brickinfo_get] 0-management: Returning 0
[2016-03-22 12:20:08.810704] D [MSGID: 0] [glusterd-utils.c:1359:glusterd_volume_brickinfo_get_by_brick] 0-glusterd: Returning 0
[2016-03-22 12:20:08.810754] I [MSGID: 106005] [glusterd-handler.c:4908:__glusterd_brick_rpc_notify] 0-management: Brick 10.32.1.144:/opt/lvmdir/c2/brick has disconnected from glusterd.
[2016-03-22 12:20:08.810788] D [MSGID: 0] [glusterd-utils.c:4872:glusterd_set_brick_status] 0-glusterd: Setting brick 10.32.1.144:/opt/lvmdir/c2/brick status to stopped
[2016-03-22 12:20:08.810830] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=4 from gen=1 to slot->gen=2, fd=8, slot->fd=8
[2016-03-22 12:20:08.826922] D [MSGID: 0] [glusterd-handshake.c:409:glusterd_get_args_from_dict] 0-management: No brick name present
[2016-03-22 12:20:08.826984] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning -1
[2016-03-22 12:20:08.827160] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 127.0.0.1 
[2016-03-22 12:20:08.836193] D [MSGID: 0] [glusterd-handshake.c:414:glusterd_get_args_from_dict] 0-management: brick_name = /opt/lvmdir/c2/brick
[2016-03-22 12:20:08.836258] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning -1
[2016-03-22 12:20:08.836447] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.1.144 
[2016-03-22 12:20:08.837241] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.1.144 is local address at interface efd
[2016-03-22 12:20:08.837309] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning -1
[2016-03-22 12:20:08.837347] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:20:08.837380] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:20:08.837794] D [MSGID: 0] [glusterd-handshake.c:841:__server_getspec] 0-management: Look for missing snap creates for /opt/lvmdir/c2/brick
[2016-03-22 12:20:08.839150] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:20:08.839182] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:20:08.839235] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=5 from gen=1 to slot->gen=2, fd=12, slot->fd=12
[2016-03-22 12:20:12.443546] I [MSGID: 106143] [glusterd-pmap.c:229:pmap_registry_bind] 0-pmap: adding brick /opt/lvmdir/c2/brick on port 49719
[2016-03-22 12:20:12.443715] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.0.48 
[2016-03-22 12:20:12.445716] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.0.48 " repeated 2 times between [2016-03-22 12:20:12.443715] and [2016-03-22 12:20:12.445164]
[2016-03-22 12:20:12.445714] D [MSGID: 0] [common-utils.c:3149:gf_is_local_addr] 0-management: 10.32.0.48 is not local
[2016-03-22 12:20:12.445860] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.1.144 
[2016-03-22 12:20:12.446334] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.1.144 is local address at interface efd
[2016-03-22 12:20:15.662574] D [MSGID: 0] [glusterd-svc-mgmt.c:315:glusterd_svc_common_rpc_notify] 0-management: glustershd has connected with glusterd.
[2016-03-22 12:20:15.662769] D [MSGID: 0] [glusterd-utils.c:1400:glusterd_volinfo_find_by_volume_id] 0-management: Volume c_glusterfs found
[2016-03-22 12:20:15.662856] D [MSGID: 0] [glusterd-utils.c:884:glusterd_brickinfo_new] 0-management: Returning 0
[2016-03-22 12:20:15.662939] D [MSGID: 0] [glusterd-utils.c:1041:glusterd_brickinfo_new_from_brick] 0-management: Returning 0
[2016-03-22 12:20:15.663559] D [MSGID: 0] [glusterd-peer-utils.c:133:glusterd_peerinfo_find_by_hostname] 0-management: Unable to find friend: 10.32.1.144
[2016-03-22 12:20:15.663738] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.1.144 
[2016-03-22 12:20:15.664775] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.1.144 is local address at interface efd
[2016-03-22 12:20:15.664856] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:20:15.664925] D [MSGID: 0] [glusterd-utils.c:1326:glusterd_volume_brickinfo_get] 0-management: Found brick 10.32.1.144:/opt/lvmdir/c2/brick in volume c_glusterfs
[2016-03-22 12:20:15.664987] D [MSGID: 0] [glusterd-utils.c:1335:glusterd_volume_brickinfo_get] 0-management: Returning 0
[2016-03-22 12:20:15.665056] D [MSGID: 0] [glusterd-utils.c:1359:glusterd_volume_brickinfo_get_by_brick] 0-glusterd: Returning 0
[2016-03-22 12:20:15.665126] D [MSGID: 0] [glusterd-handler.c:4897:__glusterd_brick_rpc_notify] 0-management: Connected to 10.32.1.144:/opt/lvmdir/c2/brick
[2016-03-22 12:20:15.665194] D [MSGID: 0] [glusterd-utils.c:4869:glusterd_set_brick_status] 0-glusterd: Setting brick 10.32.1.144:/opt/lvmdir/c2/brick status to started
[2016-03-22 12:20:16.374751] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:20:16.374788] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:20:16.374837] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=7 from gen=1 to slot->gen=2, fd=16, slot->fd=16
[2016-03-22 12:20:23.795317] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:23.795857] I [MSGID: 106163] [glusterd-handshake.c:1193:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30706
[2016-03-22 12:20:23.796148] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:20:23.811243] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:20:23.796233] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:20:23.811241] I [MSGID: 106490] [glusterd-handler.c:2539:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 356dabed-4314-48be-9eef-d2711d01270d
[2016-03-22 12:20:23.812397] E [MSGID: 101076] [common-utils.c:2954:gf_get_hostname_from_ip] 0-common-utils: Could not lookup hostname of 10.32.0.48 : Temporary failure in name resolution
[2016-03-22 12:20:23.812447] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:23.812683] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_RCVD_FRIEND_REQ'
[2016-03-22 12:20:23.812726] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:23.812756] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_RCVD_FRIEND_REQ'
[2016-03-22 12:20:23.812785] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:23.812831] D [MSGID: 0] [glusterd-utils.c:2830:import_prdict_dict] 0-glusterd: Returning with 0
[2016-03-22 12:20:23.812879] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:20:23.812908] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:20:23.812947] D [MSGID: 0] [glusterd-utils.c:2757:glusterd_compare_friend_volume] 0-management: quota checksum absent for volume c_glusterfs in peer 10.32.0.48's response
[2016-03-22 12:20:23.812975] D [MSGID: 0] [glusterd-utils.c:2776:glusterd_compare_friend_volume] 0-management: Returning with ret: 0, status: 1
[2016-03-22 12:20:23.813003] D [MSGID: 0] [glusterd-utils.c:4078:glusterd_compare_friend_data] 0-management: Returning with ret: 0, status: 1
[2016-03-22 12:20:23.813086] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:20:23.826372] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_LOCAL_ACC'
[2016-03-22 12:20:23.826470] I [MSGID: 106493] [glusterd-handler.c:3780:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 10.32.0.48 (0), ret: 0
[2016-03-22 12:20:23.826501] D [MSGID: 0] [glusterd-sm.c:911:glusterd_ac_handle_friend_add_req] 0-glusterd: Returning with 0
[2016-03-22 12:20:23.826529] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:23.826561] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Peer in Cluster' to 'Peer in Cluster' due to event 'GD_FRIEND_EVENT_RCVD_FRIEND_REQ'
[2016-03-22 12:20:23.826589] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:20:23.826617] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:23.826655] D [MSGID: 0] [glusterd-store.c:3802:glusterd_store_create_peer_dir] 0-glusterd: Returning with 0
[2016-03-22 12:20:23.826840] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:20:23.826987] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 2 times between [2016-03-22 12:20:23.826840] and [2016-03-22 12:20:23.826983]
[2016-03-22 12:20:23.826986] D [MSGID: 0] [glusterd-store.c:3931:glusterd_store_peer_write] 0-glusterd: Returning with 0
[2016-03-22 12:20:23.841010] D [MSGID: 0] [glusterd-store.c:3957:glusterd_store_perform_peer_store] 0-glusterd: Returning 0
[2016-03-22 12:20:23.841041] D [MSGID: 0] [glusterd-store.c:3978:glusterd_store_peerinfo] 0-glusterd: Returning with 0
[2016-03-22 12:20:23.841186] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:23.841215] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_LOCAL_ACC'
[2016-03-22 12:20:23.841243] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:23.841300] D [MSGID: 0] [glusterd-peer-utils.c:615:gd_add_friend_to_dict] 0-management: Returning 0
[2016-03-22 12:20:23.841454] D [MSGID: 0] [glusterd-rpc-ops.c:1674:glusterd_rpc_friend_update] 0-glusterd: Returning 0
[2016-03-22 12:20:23.841484] D [MSGID: 0] [glusterd-sm.c:590:glusterd_ac_send_friend_update] 0-glusterd: Returning with 0
[2016-03-22 12:20:23.841521] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:23.841549] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Peer in Cluster' to 'Peer in Cluster' due to event 'GD_FRIEND_EVENT_LOCAL_ACC'
[2016-03-22 12:20:23.841576] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:20:23.841603] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:23.841636] D [MSGID: 0] [glusterd-store.c:3802:glusterd_store_create_peer_dir] 0-glusterd: Returning with 0
[2016-03-22 12:20:23.841794] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:20:23.841935] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 2 times between [2016-03-22 12:20:23.841794] and [2016-03-22 12:20:23.841932]
[2016-03-22 12:20:23.841935] D [MSGID: 0] [glusterd-store.c:3931:glusterd_store_peer_write] 0-glusterd: Returning with 0
[2016-03-22 12:20:23.856172] D [MSGID: 0] [glusterd-store.c:3957:glusterd_store_perform_peer_store] 0-glusterd: Returning 0
[2016-03-22 12:20:23.856202] D [MSGID: 0] [glusterd-store.c:3978:glusterd_store_peerinfo] 0-glusterd: Returning with 0
[2016-03-22 12:20:23.856260] I [MSGID: 106493] [glusterd-rpc-ops.c:694:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 356dabed-4314-48be-9eef-d2711d01270d
[2016-03-22 12:20:23.989084] D [MSGID: 0] [glusterd-handler.c:910:glusterd_req_ctx_create] 0-management: Received op from uuid 356dabed-4314-48be-9eef-d2711d01270d
[2016-03-22 12:20:23.989174] D [MSGID: 0] [glusterd-handler.c:985:__glusterd_handle_stage_op] 0-management: transaction ID = 77bff90d-3727-4d61-bc1d-dabf668c4e4e
[2016-03-22 12:20:23.989220] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:23.989254] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning -2
[2016-03-22 12:20:23.989280] D [MSGID: 0] [glusterd-handler.c:1005:__glusterd_handle_stage_op] 0-management: No transaction's opinfo set
[2016-03-22 12:20:23.989335] D [MSGID: 0] [glusterd-op-sm.c:311:glusterd_set_txn_opinfo] 0-management: Successfully set opinfo for transaction ID : 77bff90d-3727-4d61-bc1d-dabf668c4e4e
[2016-03-22 12:20:23.989368] D [MSGID: 0] [glusterd-op-sm.c:318:glusterd_set_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:20:23.989414] D [MSGID: 0] [glusterd-op-sm.c:7346:glusterd_op_sm_inject_event] 0-management: Enqueue event: 'GD_OP_EVENT_STAGE_OP'
[2016-03-22 12:20:23.989446] D [MSGID: 0] [glusterd-op-sm.c:7428:glusterd_op_sm] 0-management: Dequeued event of type: 'GD_OP_EVENT_STAGE_OP'
[2016-03-22 12:20:23.989476] D [MSGID: 0] [glusterd-op-sm.c:7431:glusterd_op_sm] 0-management: transaction ID = 77bff90d-3727-4d61-bc1d-dabf668c4e4e
[2016-03-22 12:20:23.989509] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : 77bff90d-3727-4d61-bc1d-dabf668c4e4e
[2016-03-22 12:20:23.989550] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:20:23.989592] D [MSGID: 0] [glusterd-op-sm.c:1695:glusterd_op_stage_status_volume] 0-management: Returning: 0
[2016-03-22 12:20:23.989620] D [MSGID: 0] [glusterd-op-sm.c:5515:glusterd_op_stage_validate] 0-management: OP = 18. Returning 0
[2016-03-22 12:20:23.989760] D [MSGID: 0] [glusterd-handler.c:2469:glusterd_op_stage_send_resp] 0-management: Responded to stage, ret: 0
[2016-03-22 12:20:23.989790] D [MSGID: 0] [glusterd-op-sm.c:5224:glusterd_op_ac_stage_op] 0-management: Returning with 0
[2016-03-22 12:20:23.989822] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Locked' to 'Staged' due to event 'GD_OP_EVENT_STAGE_OP'
[2016-03-22 12:20:23.989849] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:20:23.989882] D [MSGID: 0] [glusterd-op-sm.c:311:glusterd_set_txn_opinfo] 0-management: Successfully set opinfo for transaction ID : 77bff90d-3727-4d61-bc1d-dabf668c4e4e
[2016-03-22 12:20:23.989908] D [MSGID: 0] [glusterd-op-sm.c:318:glusterd_set_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:20:23.992069] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:23.992142] D [MSGID: 0] [glusterd-handler.c:910:glusterd_req_ctx_create] 0-management: Received op from uuid 356dabed-4314-48be-9eef-d2711d01270d
[2016-03-22 12:20:23.992197] D [MSGID: 0] [glusterd-handler.c:1092:__glusterd_handle_commit_op] 0-management: transaction ID = 77bff90d-3727-4d61-bc1d-dabf668c4e4e
[2016-03-22 12:20:23.992235] D [MSGID: 0] [glusterd-op-sm.c:7346:glusterd_op_sm_inject_event] 0-management: Enqueue event: 'GD_OP_EVENT_COMMIT_OP'
[2016-03-22 12:20:23.992265] D [MSGID: 0] [glusterd-op-sm.c:7428:glusterd_op_sm] 0-management: Dequeued event of type: 'GD_OP_EVENT_COMMIT_OP'
[2016-03-22 12:20:23.992295] D [MSGID: 0] [glusterd-op-sm.c:7431:glusterd_op_sm] 0-management: transaction ID = 77bff90d-3727-4d61-bc1d-dabf668c4e4e
[2016-03-22 12:20:23.992328] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : 77bff90d-3727-4d61-bc1d-dabf668c4e4e
[2016-03-22 12:20:23.992355] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:20:23.992412] D [MSGID: 0] [glusterd-rpc-ops.c:2220:glusterd_brick_op] 0-management: transaction ID = 77bff90d-3727-4d61-bc1d-dabf668c4e4e
[2016-03-22 12:20:23.992451] D [MSGID: 0] [glusterd-op-sm.c:7010:glusterd_op_bricks_select] 0-management: Returning 0
[2016-03-22 12:20:23.992479] D [MSGID: 0] [glusterd-rpc-ops.c:2323:glusterd_brick_op] 0-management: Returning 0
[2016-03-22 12:20:23.992507] D [MSGID: 0] [glusterd-op-sm.c:7346:glusterd_op_sm_inject_event] 0-management: Enqueue event: 'GD_OP_EVENT_ALL_ACK'
[2016-03-22 12:20:23.992533] D [MSGID: 0] [glusterd-op-sm.c:6896:glusterd_op_ac_send_brick_op] 0-management: Returning with 0
[2016-03-22 12:20:23.992562] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Staged' to 'Brick op Committed' due to event 'GD_OP_EVENT_COMMIT_OP'
[2016-03-22 12:20:23.992589] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:20:23.992622] D [MSGID: 0] [glusterd-op-sm.c:311:glusterd_set_txn_opinfo] 0-management: Successfully set opinfo for transaction ID : 77bff90d-3727-4d61-bc1d-dabf668c4e4e
[2016-03-22 12:20:23.992648] D [MSGID: 0] [glusterd-op-sm.c:318:glusterd_set_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:20:23.992675] D [MSGID: 0] [glusterd-op-sm.c:7428:glusterd_op_sm] 0-management: Dequeued event of type: 'GD_OP_EVENT_ALL_ACK'
[2016-03-22 12:20:23.992704] D [MSGID: 0] [glusterd-op-sm.c:7431:glusterd_op_sm] 0-management: transaction ID = 77bff90d-3727-4d61-bc1d-dabf668c4e4e
[2016-03-22 12:20:23.992746] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : 77bff90d-3727-4d61-bc1d-dabf668c4e4e
[2016-03-22 12:20:23.992773] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:20:23.992846] D [MSGID: 0] [glusterd-op-sm.c:3360:glusterd_op_status_volume] 0-management: Returning 0
[2016-03-22 12:20:23.992874] D [MSGID: 0] [glusterd-op-sm.c:5642:glusterd_op_commit_perform] 0-management: Returning 0
[2016-03-22 12:20:23.993011] D [MSGID: 0] [glusterd-handler.c:2511:glusterd_op_commit_send_resp] 0-management: Responded to commit, ret: 0
[2016-03-22 12:20:23.993041] D [MSGID: 0] [glusterd-op-sm.c:5341:glusterd_op_ac_commit_op] 0-management: Returning with 0
[2016-03-22 12:20:23.993068] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Brick op Committed' to 'Committed' due to event 'GD_OP_EVENT_ALL_ACK'
[2016-03-22 12:20:23.993095] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:20:23.993128] D [MSGID: 0] [glusterd-op-sm.c:311:glusterd_set_txn_opinfo] 0-management: Successfully set opinfo for transaction ID : 77bff90d-3727-4d61-bc1d-dabf668c4e4e
[2016-03-22 12:20:23.993154] D [MSGID: 0] [glusterd-op-sm.c:318:glusterd_set_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:20:23.998436] D [MSGID: 0] [glusterd-mgmt-handler.c:151:glusterd_handle_mgmt_v3_lock_fn] 0-management: Received mgmt_v3 lock req from uuid: 356dabed-4314-48be-9eef-d2711d01270d
[2016-03-22 12:20:23.998501] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:23.998572] D [MSGID: 0] [glusterd-op-sm.c:311:glusterd_set_txn_opinfo] 0-management: Successfully set opinfo for transaction ID : f0d643c2-ef27-420a-bdae-5df63cb0ea93
[2016-03-22 12:20:23.998600] D [MSGID: 0] [glusterd-op-sm.c:318:glusterd_set_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:20:23.998629] D [MSGID: 0] [glusterd-op-sm.c:7346:glusterd_op_sm_inject_event] 0-management: Enqueue event: 'GD_OP_EVENT_LOCK'
[2016-03-22 12:20:23.998657] D [MSGID: 0] [glusterd-op-sm.c:7428:glusterd_op_sm] 0-management: Dequeued event of type: 'GD_OP_EVENT_LOCK'
[2016-03-22 12:20:23.998687] D [MSGID: 0] [glusterd-op-sm.c:7431:glusterd_op_sm] 0-management: transaction ID = f0d643c2-ef27-420a-bdae-5df63cb0ea93
[2016-03-22 12:20:23.998722] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : f0d643c2-ef27-420a-bdae-5df63cb0ea93
[2016-03-22 12:20:23.998749] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:20:23.998787] D [MSGID: 0] [glusterd-locks.c:562:glusterd_mgmt_v3_lock] 0-management: Trying to acquire lock of vol c_glusterfs for 356dabed-4314-48be-9eef-d2711d01270d as c_glusterfs_vol
[2016-03-22 12:20:23.999415] D [MSGID: 0] [glusterd-locks.c:618:glusterd_mgmt_v3_lock] 0-management: Lock for vol c_glusterfs successfully held by 356dabed-4314-48be-9eef-d2711d01270d
[2016-03-22 12:20:23.999550] D [MSGID: 0] [glusterd-handler.c:2334:glusterd_op_mgmt_v3_lock_send_resp] 0-management: Responded to mgmt_v3 lock, ret: 0
[2016-03-22 12:20:23.999580] D [MSGID: 0] [glusterd-op-sm.c:3676:glusterd_op_ac_lock] 0-management: Lock Returned 0
[2016-03-22 12:20:23.999609] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Default' to 'Locked' due to event 'GD_OP_EVENT_LOCK'
[2016-03-22 12:20:23.999636] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:20:23.999669] D [MSGID: 0] [glusterd-op-sm.c:311:glusterd_set_txn_opinfo] 0-management: Successfully set opinfo for transaction ID : f0d643c2-ef27-420a-bdae-5df63cb0ea93
[2016-03-22 12:20:23.999695] D [MSGID: 0] [glusterd-op-sm.c:318:glusterd_set_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:20:24.001851] D [MSGID: 0] [glusterd-handler.c:910:glusterd_req_ctx_create] 0-management: Received op from uuid 356dabed-4314-48be-9eef-d2711d01270d
[2016-03-22 12:20:24.001947] D [MSGID: 0] [glusterd-handler.c:985:__glusterd_handle_stage_op] 0-management: transaction ID = f0d643c2-ef27-420a-bdae-5df63cb0ea93
[2016-03-22 12:20:24.001982] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:24.002018] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : f0d643c2-ef27-420a-bdae-5df63cb0ea93
[2016-03-22 12:20:24.002046] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:20:24.002076] D [MSGID: 0] [glusterd-op-sm.c:7346:glusterd_op_sm_inject_event] 0-management: Enqueue event: 'GD_OP_EVENT_STAGE_OP'
[2016-03-22 12:20:24.002106] D [MSGID: 0] [glusterd-op-sm.c:7428:glusterd_op_sm] 0-management: Dequeued event of type: 'GD_OP_EVENT_STAGE_OP'
[2016-03-22 12:20:24.002137] D [MSGID: 0] [glusterd-op-sm.c:7431:glusterd_op_sm] 0-management: transaction ID = f0d643c2-ef27-420a-bdae-5df63cb0ea93
[2016-03-22 12:20:24.002170] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : f0d643c2-ef27-420a-bdae-5df63cb0ea93
[2016-03-22 12:20:24.002198] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:20:24.002234] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:20:24.002261] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:20:24.002306] D [MSGID: 0] [glusterd-op-sm.c:1695:glusterd_op_stage_status_volume] 0-management: Returning: 0
[2016-03-22 12:20:24.002336] D [MSGID: 0] [glusterd-op-sm.c:5515:glusterd_op_stage_validate] 0-management: OP = 18. Returning 0
[2016-03-22 12:20:24.002486] D [MSGID: 0] [glusterd-handler.c:2469:glusterd_op_stage_send_resp] 0-management: Responded to stage, ret: 0
[2016-03-22 12:20:24.002518] D [MSGID: 0] [glusterd-op-sm.c:5224:glusterd_op_ac_stage_op] 0-management: Returning with 0
[2016-03-22 12:20:24.002550] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Locked' to 'Staged' due to event 'GD_OP_EVENT_STAGE_OP'
[2016-03-22 12:20:24.002579] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:20:24.002613] D [MSGID: 0] [glusterd-op-sm.c:311:glusterd_set_txn_opinfo] 0-management: Successfully set opinfo for transaction ID : f0d643c2-ef27-420a-bdae-5df63cb0ea93
[2016-03-22 12:20:24.002641] D [MSGID: 0] [glusterd-op-sm.c:318:glusterd_set_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:20:24.005340] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:24.005430] D [MSGID: 0] [glusterd-handler.c:910:glusterd_req_ctx_create] 0-management: Received op from uuid 356dabed-4314-48be-9eef-d2711d01270d
[2016-03-22 12:20:24.005491] D [MSGID: 0] [glusterd-handler.c:1092:__glusterd_handle_commit_op] 0-management: transaction ID = f0d643c2-ef27-420a-bdae-5df63cb0ea93
[2016-03-22 12:20:24.005522] D [MSGID: 0] [glusterd-op-sm.c:7346:glusterd_op_sm_inject_event] 0-management: Enqueue event: 'GD_OP_EVENT_COMMIT_OP'
[2016-03-22 12:20:24.005553] D [MSGID: 0] [glusterd-op-sm.c:7428:glusterd_op_sm] 0-management: Dequeued event of type: 'GD_OP_EVENT_COMMIT_OP'
[2016-03-22 12:20:24.005582] D [MSGID: 0] [glusterd-op-sm.c:7431:glusterd_op_sm] 0-management: transaction ID = f0d643c2-ef27-420a-bdae-5df63cb0ea93
[2016-03-22 12:20:24.005617] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : f0d643c2-ef27-420a-bdae-5df63cb0ea93
[2016-03-22 12:20:24.005645] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:20:24.005689] D [MSGID: 0] [glusterd-rpc-ops.c:2220:glusterd_brick_op] 0-management: transaction ID = f0d643c2-ef27-420a-bdae-5df63cb0ea93
[2016-03-22 12:20:24.005722] D [MSGID: 0] [glusterd-op-sm.c:7010:glusterd_op_bricks_select] 0-management: Returning 0
[2016-03-22 12:20:24.005751] D [MSGID: 0] [glusterd-rpc-ops.c:2323:glusterd_brick_op] 0-management: Returning 0
[2016-03-22 12:20:24.005780] D [MSGID: 0] [glusterd-op-sm.c:7346:glusterd_op_sm_inject_event] 0-management: Enqueue event: 'GD_OP_EVENT_ALL_ACK'
[2016-03-22 12:20:24.005807] D [MSGID: 0] [glusterd-op-sm.c:6896:glusterd_op_ac_send_brick_op] 0-management: Returning with 0
[2016-03-22 12:20:24.005836] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Staged' to 'Brick op Committed' due to event 'GD_OP_EVENT_COMMIT_OP'
[2016-03-22 12:20:24.005864] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:20:24.005899] D [MSGID: 0] [glusterd-op-sm.c:311:glusterd_set_txn_opinfo] 0-management: Successfully set opinfo for transaction ID : f0d643c2-ef27-420a-bdae-5df63cb0ea93
[2016-03-22 12:20:24.005926] D [MSGID: 0] [glusterd-op-sm.c:318:glusterd_set_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:20:24.005954] D [MSGID: 0] [glusterd-op-sm.c:7428:glusterd_op_sm] 0-management: Dequeued event of type: 'GD_OP_EVENT_ALL_ACK'
[2016-03-22 12:20:24.005983] D [MSGID: 0] [glusterd-op-sm.c:7431:glusterd_op_sm] 0-management: transaction ID = f0d643c2-ef27-420a-bdae-5df63cb0ea93
[2016-03-22 12:20:24.006017] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : f0d643c2-ef27-420a-bdae-5df63cb0ea93
[2016-03-22 12:20:24.006044] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:20:24.006090] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:20:24.006117] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:20:24.006563] D [MSGID: 0] [glusterd-snapshot-utils.c:3595:glusterd_is_snapd_enabled] 0-management: Key features.uss not present in the dict for volume c_glusterfs
[2016-03-22 12:20:24.006752] D [MSGID: 0] [glusterd-utils.c:4342:glusterd_add_node_to_dict] 0-management: Returning 0
[2016-03-22 12:20:24.007073] D [MSGID: 0] [glusterd-op-sm.c:3360:glusterd_op_status_volume] 0-management: Returning 0
[2016-03-22 12:20:24.007109] D [MSGID: 0] [glusterd-op-sm.c:5642:glusterd_op_commit_perform] 0-management: Returning 0
[2016-03-22 12:20:24.007234] D [MSGID: 0] [glusterd-handler.c:2511:glusterd_op_commit_send_resp] 0-management: Responded to commit, ret: 0
[2016-03-22 12:20:24.007265] D [MSGID: 0] [glusterd-op-sm.c:5341:glusterd_op_ac_commit_op] 0-management: Returning with 0
[2016-03-22 12:20:24.007293] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Brick op Committed' to 'Committed' due to event 'GD_OP_EVENT_ALL_ACK'
[2016-03-22 12:20:24.007321] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:20:24.007355] D [MSGID: 0] [glusterd-op-sm.c:311:glusterd_set_txn_opinfo] 0-management: Successfully set opinfo for transaction ID : f0d643c2-ef27-420a-bdae-5df63cb0ea93
[2016-03-22 12:20:24.007382] D [MSGID: 0] [glusterd-op-sm.c:318:glusterd_set_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:20:24.009896] D [MSGID: 0] [glusterd-mgmt-handler.c:874:glusterd_handle_mgmt_v3_unlock_fn] 0-management: Received volume unlock req from uuid: 356dabed-4314-48be-9eef-d2711d01270d
[2016-03-22 12:20:24.009969] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:20:24.010122] D [MSGID: 0] [glusterd-op-sm.c:7346:glusterd_op_sm_inject_event] 0-management: Enqueue event: 'GD_OP_EVENT_UNLOCK'
[2016-03-22 12:20:24.010167] D [MSGID: 0] [glusterd-op-sm.c:7428:glusterd_op_sm] 0-management: Dequeued event of type: 'GD_OP_EVENT_UNLOCK'
[2016-03-22 12:20:24.010199] D [MSGID: 0] [glusterd-op-sm.c:7431:glusterd_op_sm] 0-management: transaction ID = f0d643c2-ef27-420a-bdae-5df63cb0ea93
[2016-03-22 12:20:24.010234] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : f0d643c2-ef27-420a-bdae-5df63cb0ea93
[2016-03-22 12:20:24.010262] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:20:24.010301] D [MSGID: 0] [glusterd-locks.c:669:glusterd_mgmt_v3_unlock] 0-management: Trying to release lock of vol c_glusterfs for 356dabed-4314-48be-9eef-d2711d01270d as c_glusterfs_vol
[2016-03-22 12:20:24.010344] D [MSGID: 0] [glusterd-locks.c:714:glusterd_mgmt_v3_unlock] 0-management: Lock for vol c_glusterfs successfully released
[2016-03-22 12:20:24.010482] D [MSGID: 0] [glusterd-handler.c:2359:glusterd_op_mgmt_v3_unlock_send_resp] 0-management: Responded to mgmt_v3 unlock, ret: 0
[2016-03-22 12:20:24.010570] D [MSGID: 0] [glusterd-op-sm.c:3740:glusterd_op_ac_unlock] 0-management: Unlock Returned 0
[2016-03-22 12:20:24.010601] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Committed' to 'Default' due to event 'GD_OP_EVENT_UNLOCK'
[2016-03-22 12:20:24.010628] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:20:24.010662] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : f0d643c2-ef27-420a-bdae-5df63cb0ea93
[2016-03-22 12:20:24.010688] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:20:24.010766] D [MSGID: 0] [glusterd-op-sm.c:360:glusterd_clear_txn_opinfo] 0-management: Successfully cleared opinfo for transaction ID : f0d643c2-ef27-420a-bdae-5df63cb0ea93
[2016-03-22 12:20:24.010793] D [MSGID: 0] [glusterd-op-sm.c:364:glusterd_clear_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:20:24.180419] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:20:24.180474] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:20:24.180523] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=8 from gen=1 to slot->gen=2, fd=17, slot->fd=17
[2016-03-22 12:20:24.205224] D [MSGID: 0] [glusterd-handshake.c:409:glusterd_get_args_from_dict] 0-management: No brick name present
[2016-03-22 12:20:24.205271] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:20:24.205299] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:20:24.205439] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.1.144 
[2016-03-22 12:20:24.206093] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.1.144 is local address at interface efd
[2016-03-22 12:20:24.206147] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:20:24.206176] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:20:24.224165] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:20:24.224191] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:20:24.224232] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=9 from gen=1 to slot->gen=2, fd=18, slot->fd=18
[2016-03-22 12:20:26.374838] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:20:26.374874] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:20:26.374914] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=9 from gen=4 to slot->gen=5, fd=18, slot->fd=18
[2016-03-22 12:20:26.547030] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:20:26.547060] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:20:26.547099] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=9 from gen=7 to slot->gen=8, fd=18, slot->fd=18
[2016-03-22 12:20:26.547166] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:20:26.547187] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:20:26.547222] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=10 from gen=1 to slot->gen=2, fd=19, slot->fd=19
[2016-03-22 12:20:28.621644] D [MSGID: 0] [glusterd-handshake.c:409:glusterd_get_args_from_dict] 0-management: No brick name present
[2016-03-22 12:20:28.621699] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:20:28.621726] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:20:28.621839] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 127.0.0.1 
[2016-03-22 12:20:28.621877] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:20:28.621906] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:20:28.639117] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:20:28.639151] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:20:28.639197] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=11 from gen=1 to slot->gen=2, fd=20, slot->fd=20
[2016-03-22 12:20:30.696163] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:20:30.696223] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:20:30.696298] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=10 from gen=4 to slot->gen=5, fd=19, slot->fd=19
[2016-03-22 12:20:30.698109] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:20:30.698135] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:20:30.698168] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=9 from gen=10 to slot->gen=11, fd=18, slot->fd=18
[2016-03-22 12:20:42.664803] D [rpc-clnt-ping.c:300:rpc_clnt_start_ping] 0-management: returning as transport is already disconnected OR there are no frames (0 || 0)
[2016-03-22 12:22:10.671503] D [logging.c:1828:gf_log_flush_timeout_cbk] 0-logging-infra: Log timer timed out. About to flush outstanding messages if present
[2016-03-22 12:22:10.671627] D [logging.c:1790:__gf_log_inject_timer_event] 0-logging-infra: Starting timer now. Timeout = 120, current buf size = 5
[2016-03-22 12:24:10.680886] D [logging.c:1828:gf_log_flush_timeout_cbk] 0-logging-infra: Log timer timed out. About to flush outstanding messages if present
[2016-03-22 12:24:10.680943] D [logging.c:1790:__gf_log_inject_timer_event] 0-logging-infra: Starting timer now. Timeout = 120, current buf size = 5
[2016-03-22 12:24:26.796235] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:24:26.796304] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:24:26.796381] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=7 from gen=4 to slot->gen=5, fd=16, slot->fd=16
[2016-03-22 12:24:26.796235] D [socket.c:558:__socket_rwv] 0-management: EOF on socket
[2016-03-22 12:24:26.796520] W [socket.c:588:__socket_rwv] 0-management: readv on 10.32.0.48:24007 failed (No data available)
[2016-03-22 12:24:26.796539] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:24:26.796949] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8b729f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff8b6d3990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup-0x19b20)[0x3fff8b6cfaf0] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_notify-0x18e68)[0x3fff8b6d0808] (--> /usr/lib64/libgfrpc.so.0(rpc_transport_notify-0x1d8bc)[0x3fff8b6cb85c] ))))) 0-: 10.32.0.48:24007: ping timer event already removed
[2016-03-22 12:24:26.797007] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:24:26.797046] I [MSGID: 106004] [glusterd-handler.c:5065:__glusterd_peer_rpc_notify] 0-management: Peer <10.32.0.48> (<356dabed-4314-48be-9eef-d2711d01270d>), in state <Peer in Cluster>, has disconnected from glusterd.
[2016-03-22 12:24:26.797082] D [MSGID: 0] [glusterd-locks.c:669:glusterd_mgmt_v3_unlock] 0-management: Trying to release lock of vol c_glusterfs for 356dabed-4314-48be-9eef-d2711d01270d as c_glusterfs_vol
[2016-03-22 12:24:26.797388] W [glusterd-locks.c:681:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_big_locked_notify-0x14b5a0) [0x3fff880514f8] -->/usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(__glusterd_peer_rpc_notify-0x141264) [0x3fff8805bf54] -->/usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_mgmt_v3_unlock-0x9dd2c) [0x3fff8810497c] ) 0-management: Lock for vol c_glusterfs not held
[2016-03-22 12:24:26.797431] W [MSGID: 106118] [glusterd-handler.c:5087:__glusterd_peer_rpc_notify] 0-management: Lock not released for c_glusterfs
[2016-03-22 12:24:26.797467] D [MSGID: 0] [glusterd-utils.c:173:glusterd_lock] 0-management: Cluster lock held by 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:24:26.797505] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=2 from gen=1 to slot->gen=2, fd=13, slot->fd=13
[2016-03-22 12:24:37.683731] D [MSGID: 0] [common-utils.c:333:gf_resolve_ip6] 0-resolver: returning ip-10.32.0.48 (port-24007) for hostname: 10.32.0.48 and port: 24007
[2016-03-22 12:24:37.683796] D [socket.c:2833:socket_fix_ssl_opts] 0-management: disabling SSL for portmapper connection
[2016-03-22 12:24:37.683847] W [socket.c:869:__socket_keepalive] 0-socket: failed to set TCP_USER_TIMEOUT -1000 on socket 13, Invalid argument
[2016-03-22 12:24:37.683870] E [socket.c:2965:socket_connect] 0-management: Failed to set keep-alive: Invalid argument
[2016-03-22 12:25:08.756024] E [socket.c:2278:socket_connect_finish] 0-management: connection to 10.32.0.48:24007 failed (Connection refused)
[2016-03-22 12:25:08.756082] D [socket.c:681:__socket_shutdown] 0-management: shutdown() returned -1. Transport endpoint is not connected
[2016-03-22 12:25:08.756103] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:08.756527] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8b729f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff8b6d3990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup-0x19b20)[0x3fff8b6cfaf0] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_notify-0x18e68)[0x3fff8b6d0808] (--> /usr/lib64/libgfrpc.so.0(rpc_transport_notify-0x1d8bc)[0x3fff8b6cb85c] ))))) 0-: 10.32.0.48:24007: ping timer event already removed
[2016-03-22 12:25:08.756606] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:25:08.756592] I [MSGID: 106004] [glusterd-handler.c:5065:__glusterd_peer_rpc_notify] 0-management: Peer <10.32.0.48> (<356dabed-4314-48be-9eef-d2711d01270d>), in state <Peer in Cluster>, has disconnected from glusterd.
[2016-03-22 12:25:08.756604] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=2 from gen=4 to slot->gen=5, fd=13, slot->fd=13
[2016-03-22 12:25:10.687667] D [socket.c:2833:socket_fix_ssl_opts] 0-management: disabling SSL for portmapper connection
[2016-03-22 12:25:10.687736] W [socket.c:869:__socket_keepalive] 0-socket: failed to set TCP_USER_TIMEOUT -1000 on socket 13, Invalid argument
[2016-03-22 12:25:10.687759] E [socket.c:2965:socket_connect] 0-management: Failed to set keep-alive: Invalid argument
[2016-03-22 12:25:13.691797] D [socket.c:681:__socket_shutdown] 0-management: shutdown() returned -1. Transport endpoint is not connected
[2016-03-22 12:25:13.691843] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:13.692270] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8b729f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff8b6d3990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup-0x19b20)[0x3fff8b6cfaf0] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_notify-0x18e68)[0x3fff8b6d0808] (--> /usr/lib64/libgfrpc.so.0(rpc_transport_notify-0x1d8bc)[0x3fff8b6cb85c] ))))) 0-: 10.32.0.48:24007: ping timer event already removed
[2016-03-22 12:25:13.692337] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Peer in Cluster" repeated 2 times between [2016-03-22 12:24:26.797007] and [2016-03-22 12:25:13.692322]
[2016-03-22 12:25:13.692336] I [MSGID: 106004] [glusterd-handler.c:5065:__glusterd_peer_rpc_notify] 0-management: Peer <10.32.0.48> (<356dabed-4314-48be-9eef-d2711d01270d>), in state <Peer in Cluster>, has disconnected from glusterd.
[2016-03-22 12:25:13.692439] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=2 from gen=7 to slot->gen=8, fd=13, slot->fd=13
[2016-03-22 12:25:16.689050] D [socket.c:2833:socket_fix_ssl_opts] 0-management: disabling SSL for portmapper connection
[2016-03-22 12:25:16.689121] W [socket.c:869:__socket_keepalive] 0-socket: failed to set TCP_USER_TIMEOUT -1000 on socket 13, Invalid argument
[2016-03-22 12:25:16.689144] E [socket.c:2965:socket_connect] 0-management: Failed to set keep-alive: Invalid argument
[2016-03-22 12:25:16.689731] D [socket.c:681:__socket_shutdown] 0-management: shutdown() returned -1. Transport endpoint is not connected
[2016-03-22 12:25:16.689778] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:16.690209] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8b729f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff8b6d3990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup-0x19b20)[0x3fff8b6cfaf0] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_notify-0x18e68)[0x3fff8b6d0808] (--> /usr/lib64/libgfrpc.so.0(rpc_transport_notify-0x1d8bc)[0x3fff8b6cb85c] ))))) 0-: 10.32.0.48:24007: ping timer event already removed
[2016-03-22 12:25:16.690262] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [common-utils.c:333:gf_resolve_ip6] 0-resolver: returning ip-10.32.0.48 (port-24007) for hostname: 10.32.0.48 and port: 24007" repeated 2 times between [2016-03-22 12:24:37.683731] and [2016-03-22 12:25:16.689046]
[2016-03-22 12:25:16.690261] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:16.690353] I [MSGID: 106004] [glusterd-handler.c:5065:__glusterd_peer_rpc_notify] 0-management: Peer <10.32.0.48> (<356dabed-4314-48be-9eef-d2711d01270d>), in state <Peer in Cluster>, has disconnected from glusterd.
[2016-03-22 12:25:16.690390] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=2 from gen=10 to slot->gen=11, fd=13, slot->fd=13
[2016-03-22 12:25:19.691174] D [MSGID: 0] [common-utils.c:333:gf_resolve_ip6] 0-resolver: returning ip-10.32.0.48 (port-24007) for hostname: 10.32.0.48 and port: 24007
[2016-03-22 12:25:19.691234] D [socket.c:2833:socket_fix_ssl_opts] 0-management: disabling SSL for portmapper connection
[2016-03-22 12:25:19.691288] W [socket.c:869:__socket_keepalive] 0-socket: failed to set TCP_USER_TIMEOUT -1000 on socket 13, Invalid argument
[2016-03-22 12:25:19.691315] E [socket.c:2965:socket_connect] 0-management: Failed to set keep-alive: Invalid argument
[2016-03-22 12:25:19.691823] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.691883] D [MSGID: 0] [glusterd-handler.c:5042:__glusterd_peer_rpc_notify] 0-management: got RPC_CLNT_CONNECT
[2016-03-22 12:25:19.691917] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.692620] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8b729f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff8b6d3990] (--> /usr/lib64/libgfrpc.so.0(+0x1a188)[0x3fff8b6d4188] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_submit-0x1a4d8)[0x3fff8b6cf0a8] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_submit_request-0x129078)[0x3fff88074de8] ))))) 0-: 10.32.0.48:24007: ping timer event already removed
[2016-03-22 12:25:19.855792] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:25:19.855745] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.855761] D [MSGID: 0] [glusterd-handshake.c:2098:__glusterd_peer_dump_version_cbk] 0-management: Proceeding to op-version handshake with peer 10.32.0.48
[2016-03-22 12:25:19.855960] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.856492] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.856603] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.856649] D [MSGID: 0] [glusterd-handshake.c:1660:gd_validate_peer_op_version] 0-management: Peer 10.32.0.48 accepted
[2016-03-22 12:25:19.856905] I [MSGID: 106163] [glusterd-handshake.c:1193:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30706
[2016-03-22 12:25:19.857188] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:19.879210] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:25:19.857283] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:19.879208] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.879323] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:25:19.879316] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.879323] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_CONNECTED'
[2016-03-22 12:25:19.879389] D [MSGID: 0] [glusterd-handshake.c:1613:glusterd_event_connected_inject] 0-glusterd: returning 0
[2016-03-22 12:25:19.879438] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.879478] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_CONNECTED'
[2016-03-22 12:25:19.879507] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.879854] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:25:19.879538] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.879854] D [MSGID: 0] [glusterd-utils.c:2504:glusterd_add_volume_to_dict] 0-management: Returning with 0
[2016-03-22 12:25:19.880331] D [MSGID: 0] [glusterd-rpc-ops.c:1579:glusterd_rpc_friend_add] 0-glusterd: Returning 0
[2016-03-22 12:25:19.880364] D [MSGID: 0] [glusterd-sm.c:334:glusterd_ac_friend_add] 0-glusterd: Returning with 0
[2016-03-22 12:25:19.880392] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.880441] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Peer in Cluster' to 'Peer in Cluster' due to event 'GD_FRIEND_EVENT_CONNECTED'
[2016-03-22 12:25:19.880473] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:19.880502] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.880544] D [MSGID: 0] [glusterd-store.c:3802:glusterd_store_create_peer_dir] 0-glusterd: Returning with 0
[2016-03-22 12:25:19.880776] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:19.880933] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 2 times between [2016-03-22 12:25:19.880776] and [2016-03-22 12:25:19.880929]
[2016-03-22 12:25:19.880933] D [MSGID: 0] [glusterd-store.c:3931:glusterd_store_peer_write] 0-glusterd: Returning with 0
[2016-03-22 12:25:19.895999] D [MSGID: 0] [glusterd-store.c:3957:glusterd_store_perform_peer_store] 0-glusterd: Returning 0
[2016-03-22 12:25:19.896056] D [MSGID: 0] [glusterd-store.c:3978:glusterd_store_peerinfo] 0-glusterd: Returning with 0
[2016-03-22 12:25:19.896174] I [MSGID: 106490] [glusterd-handler.c:2539:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 356dabed-4314-48be-9eef-d2711d01270d
[2016-03-22 12:25:19.896635] E [MSGID: 101076] [common-utils.c:2954:gf_get_hostname_from_ip] 0-common-utils: Could not lookup hostname of 10.32.0.48 : Temporary failure in name resolution
[2016-03-22 12:25:19.896684] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.897014] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_RCVD_FRIEND_REQ'
[2016-03-22 12:25:19.897051] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.897108] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_RCVD_FRIEND_REQ'
[2016-03-22 12:25:19.897143] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.897202] D [MSGID: 0] [glusterd-utils.c:2830:import_prdict_dict] 0-glusterd: Returning with 0
[2016-03-22 12:25:19.897249] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:19.897290] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:19.897336] D [MSGID: 0] [glusterd-utils.c:2757:glusterd_compare_friend_volume] 0-management: quota checksum absent for volume c_glusterfs in peer 10.32.0.48's response
[2016-03-22 12:25:19.897388] D [MSGID: 0] [glusterd-utils.c:2776:glusterd_compare_friend_volume] 0-management: Returning with ret: 0, status: 1
[2016-03-22 12:25:19.897436] D [MSGID: 0] [glusterd-utils.c:4078:glusterd_compare_friend_data] 0-management: Returning with ret: 0, status: 1
[2016-03-22 12:25:19.910907] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_LOCAL_ACC'
[2016-03-22 12:25:19.911062] I [MSGID: 106493] [glusterd-handler.c:3780:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 10.32.0.48 (0), ret: 0
[2016-03-22 12:25:19.911094] D [MSGID: 0] [glusterd-sm.c:911:glusterd_ac_handle_friend_add_req] 0-glusterd: Returning with 0
[2016-03-22 12:25:19.911127] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.911160] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Peer in Cluster' to 'Peer in Cluster' due to event 'GD_FRIEND_EVENT_RCVD_FRIEND_REQ'
[2016-03-22 12:25:19.911188] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:19.911221] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.911263] D [MSGID: 0] [glusterd-store.c:3802:glusterd_store_create_peer_dir] 0-glusterd: Returning with 0
[2016-03-22 12:25:19.911501] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:19.911654] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 2 times between [2016-03-22 12:25:19.911501] and [2016-03-22 12:25:19.911649]
[2016-03-22 12:25:19.911653] D [MSGID: 0] [glusterd-store.c:3931:glusterd_store_peer_write] 0-glusterd: Returning with 0
[2016-03-22 12:25:19.926778] D [MSGID: 0] [glusterd-store.c:3957:glusterd_store_perform_peer_store] 0-glusterd: Returning 0
[2016-03-22 12:25:19.926829] D [MSGID: 0] [glusterd-store.c:3978:glusterd_store_peerinfo] 0-glusterd: Returning with 0
[2016-03-22 12:25:19.927038] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.927092] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_LOCAL_ACC'
[2016-03-22 12:25:19.927122] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.927198] D [MSGID: 0] [glusterd-peer-utils.c:615:gd_add_friend_to_dict] 0-management: Returning 0
[2016-03-22 12:25:19.927334] D [MSGID: 0] [glusterd-rpc-ops.c:1674:glusterd_rpc_friend_update] 0-glusterd: Returning 0
[2016-03-22 12:25:19.927363] D [MSGID: 0] [glusterd-sm.c:590:glusterd_ac_send_friend_update] 0-glusterd: Returning with 0
[2016-03-22 12:25:19.927449] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.927481] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Peer in Cluster' to 'Peer in Cluster' due to event 'GD_FRIEND_EVENT_LOCAL_ACC'
[2016-03-22 12:25:19.927509] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:19.927536] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.927585] D [MSGID: 0] [glusterd-store.c:3802:glusterd_store_create_peer_dir] 0-glusterd: Returning with 0
[2016-03-22 12:25:19.927819] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:19.928007] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 2 times between [2016-03-22 12:25:19.927819] and [2016-03-22 12:25:19.928002]
[2016-03-22 12:25:19.928006] D [MSGID: 0] [glusterd-store.c:3931:glusterd_store_peer_write] 0-glusterd: Returning with 0
[2016-03-22 12:25:19.943701] D [MSGID: 0] [glusterd-store.c:3957:glusterd_store_perform_peer_store] 0-glusterd: Returning 0
[2016-03-22 12:25:19.943766] D [MSGID: 0] [glusterd-store.c:3978:glusterd_store_peerinfo] 0-glusterd: Returning with 0
[2016-03-22 12:25:19.943967] D [MSGID: 0] [glusterd-utils.c:173:glusterd_lock] 0-management: Cluster lock held by 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:19.944072] D [MSGID: 0] [glusterd-snapshot-utils.c:3595:glusterd_is_snapd_enabled] 0-management: Key features.uss not present in the dict for volume c_glusterfs
[2016-03-22 12:25:19.944151] I [MSGID: 106493] [glusterd-rpc-ops.c:480:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 356dabed-4314-48be-9eef-d2711d01270d, host: 10.32.0.48, port: 0
[2016-03-22 12:25:19.944194] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.944235] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_RCVD_ACC'
[2016-03-22 12:25:19.944270] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.944304] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_RCVD_ACC'
[2016-03-22 12:25:19.944338] D [MSGID: 0] [glusterd-sm.c:203:glusterd_ac_none] 0-glusterd: Returning with 0
[2016-03-22 12:25:19.944371] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.944421] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Peer in Cluster' to 'Peer in Cluster' due to event 'GD_FRIEND_EVENT_RCVD_ACC'
[2016-03-22 12:25:19.944456] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:19.944489] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.944539] D [MSGID: 0] [glusterd-store.c:3802:glusterd_store_create_peer_dir] 0-glusterd: Returning with 0
[2016-03-22 12:25:19.944818] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:19.945000] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 2 times between [2016-03-22 12:25:19.944818] and [2016-03-22 12:25:19.944993]
[2016-03-22 12:25:19.944998] D [MSGID: 0] [glusterd-store.c:3931:glusterd_store_peer_write] 0-glusterd: Returning with 0
[2016-03-22 12:25:19.959949] D [MSGID: 0] [glusterd-store.c:3957:glusterd_store_perform_peer_store] 0-glusterd: Returning 0
[2016-03-22 12:25:19.960009] D [MSGID: 0] [glusterd-store.c:3978:glusterd_store_peerinfo] 0-glusterd: Returning with 0
[2016-03-22 12:25:19.960093] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:19.960139] I [MSGID: 106492] [glusterd-handler.c:2715:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 356dabed-4314-48be-9eef-d2711d01270d
[2016-03-22 12:25:19.960229] I [MSGID: 106502] [glusterd-handler.c:2760:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2016-03-22 12:25:20.030785] I [MSGID: 106493] [glusterd-rpc-ops.c:694:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 356dabed-4314-48be-9eef-d2711d01270d
[2016-03-22 12:25:20.057605] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:20.057652] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:20.057705] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=9 from gen=13 to slot->gen=14, fd=18, slot->fd=18
[2016-03-22 12:25:20.175439] I [MSGID: 106488] [glusterd-handler.c:1472:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req
[2016-03-22 12:25:20.175520] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:20.175548] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:20.177899] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:20.177945] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:20.177988] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=9 from gen=16 to slot->gen=17, fd=18, slot->fd=18
[2016-03-22 12:25:20.237011] I [MSGID: 106487] [glusterd-handler.c:1411:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2016-03-22 12:25:20.239914] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:20.239958] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:20.240003] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=9 from gen=19 to slot->gen=20, fd=18, slot->fd=18
[2016-03-22 12:25:20.290821] I [MSGID: 106487] [glusterd-handler.c:1411:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2016-03-22 12:25:20.293188] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:20.293253] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:20.293313] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=9 from gen=22 to slot->gen=23, fd=18, slot->fd=18
[2016-03-22 12:25:20.346484] D [MSGID: 0] [glusterd-op-sm.c:214:glusterd_generate_txn_id] 0-management: Transaction_id = e739fff3-8668-4c1a-9fdc-6d71196119bd
[2016-03-22 12:25:20.346558] D [MSGID: 0] [glusterd-op-sm.c:311:glusterd_set_txn_opinfo] 0-management: Successfully set opinfo for transaction ID : e739fff3-8668-4c1a-9fdc-6d71196119bd
[2016-03-22 12:25:20.346587] D [MSGID: 0] [glusterd-op-sm.c:318:glusterd_set_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.346620] D [MSGID: 0] [glusterd-syncop.c:1767:gd_sync_task_begin] 0-management: Transaction ID : e739fff3-8668-4c1a-9fdc-6d71196119bd
[2016-03-22 12:25:20.346655] D [MSGID: 0] [glusterd-syncop.c:1807:gd_sync_task_begin] 0-glusterd: Failed to get volume name
[2016-03-22 12:25:20.346716] D [MSGID: 0] [glusterd-op-sm.c:1695:glusterd_op_stage_status_volume] 0-management: Returning: 0
[2016-03-22 12:25:20.346749] D [MSGID: 0] [glusterd-op-sm.c:5515:glusterd_op_stage_validate] 0-management: OP = 18. Returning 0
[2016-03-22 12:25:20.347018] D [MSGID: 0] [glusterd-syncop.c:1312:gd_stage_op_phase] 0-management: Sent stage op req for 'Volume Status' to 1 peers
[2016-03-22 12:25:20.352046] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:20.352286] D [MSGID: 0] [glusterd-op-sm.c:7010:glusterd_op_bricks_select] 0-management: Returning 0
[2016-03-22 12:25:20.352323] D [MSGID: 0] [glusterd-syncop.c:1709:gd_brick_op_phase] 0-management: Sent op req to 0 bricks
[2016-03-22 12:25:20.352379] D [MSGID: 0] [glusterd-op-sm.c:3360:glusterd_op_status_volume] 0-management: Returning 0
[2016-03-22 12:25:20.352432] D [MSGID: 0] [glusterd-op-sm.c:5642:glusterd_op_commit_perform] 0-management: Returning 0
[2016-03-22 12:25:20.356031] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:20.356168] D [MSGID: 0] [glusterd-syncop.c:1449:gd_commit_op_phase] 0-management: Sent commit op req for 'Volume Status' to 1 peers
[2016-03-22 12:25:20.356207] D [MSGID: 0] [glusterd-op-sm.c:4487:glusterd_op_modify_op_ctx] 0-management: op_ctx modification not required for status operation being performed
[2016-03-22 12:25:20.356252] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : e739fff3-8668-4c1a-9fdc-6d71196119bd
[2016-03-22 12:25:20.356282] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.356328] D [MSGID: 0] [glusterd-op-sm.c:360:glusterd_clear_txn_opinfo] 0-management: Successfully cleared opinfo for transaction ID : e739fff3-8668-4c1a-9fdc-6d71196119bd
[2016-03-22 12:25:20.356363] D [MSGID: 0] [glusterd-op-sm.c:364:glusterd_clear_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.356626] D [MSGID: 0] [glusterd-rpc-ops.c:205:glusterd_op_send_cli_response] 0-management: Returning 0
[2016-03-22 12:25:20.359178] I [MSGID: 106499] [glusterd-handler.c:4267:__glusterd_handle_status_volume] 0-management: Received status volume req for volume c_glusterfs
[2016-03-22 12:25:20.359321] D [MSGID: 0] [glusterd-op-sm.c:214:glusterd_generate_txn_id] 0-management: Transaction_id = 00999cc1-70ab-410c-b2dd-b5e197de6f90
[2016-03-22 12:25:20.359366] D [MSGID: 0] [glusterd-op-sm.c:311:glusterd_set_txn_opinfo] 0-management: Successfully set opinfo for transaction ID : 00999cc1-70ab-410c-b2dd-b5e197de6f90
[2016-03-22 12:25:20.359394] D [MSGID: 0] [glusterd-op-sm.c:318:glusterd_set_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.359441] D [MSGID: 0] [glusterd-syncop.c:1767:gd_sync_task_begin] 0-management: Transaction ID : 00999cc1-70ab-410c-b2dd-b5e197de6f90
[2016-03-22 12:25:20.359486] D [MSGID: 0] [glusterd-locks.c:562:glusterd_mgmt_v3_lock] 0-management: Trying to acquire lock of vol c_glusterfs for 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2 as c_glusterfs_vol
[2016-03-22 12:25:20.360244] D [MSGID: 0] [glusterd-locks.c:618:glusterd_mgmt_v3_lock] 0-management: Lock for vol c_glusterfs successfully held by 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:20.360490] D [MSGID: 0] [glusterd-syncop.c:411:gd_syncop_mgmt_v3_lock] 0-glusterd: Returning 0
[2016-03-22 12:25:20.364417] D [MSGID: 0] [glusterd-syncop.c:1205:gd_lock_op_phase] 0-management: Sent lock op req for 'Volume Status' to 1 peers. Returning 0
[2016-03-22 12:25:20.364503] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:20.364538] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:20.364605] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:20.364642] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:20.364690] D [MSGID: 0] [glusterd-op-sm.c:1695:glusterd_op_stage_status_volume] 0-management: Returning: 0
[2016-03-22 12:25:20.364729] D [MSGID: 0] [glusterd-op-sm.c:5515:glusterd_op_stage_validate] 0-management: OP = 18. Returning 0
[2016-03-22 12:25:20.364869] D [MSGID: 0] [glusterd-syncop.c:1312:gd_stage_op_phase] 0-management: Sent stage op req for 'Volume Status' to 1 peers
[2016-03-22 12:25:20.368746] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:20.368879] D [MSGID: 0] [glusterd-op-sm.c:7010:glusterd_op_bricks_select] 0-management: Returning 0
[2016-03-22 12:25:20.368933] D [MSGID: 0] [glusterd-syncop.c:1709:gd_brick_op_phase] 0-management: Sent op req to 0 bricks
[2016-03-22 12:25:20.368994] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:20.369023] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:20.369231] D [MSGID: 0] [glusterd-snapshot-utils.c:3595:glusterd_is_snapd_enabled] 0-management: Key features.uss not present in the dict for volume c_glusterfs
[2016-03-22 12:25:20.369382] D [MSGID: 0] [glusterd-utils.c:4342:glusterd_add_node_to_dict] 0-management: Returning 0
[2016-03-22 12:25:20.369753] D [MSGID: 0] [glusterd-op-sm.c:3360:glusterd_op_status_volume] 0-management: Returning 0
[2016-03-22 12:25:20.369790] D [MSGID: 0] [glusterd-op-sm.c:5642:glusterd_op_commit_perform] 0-management: Returning 0
[2016-03-22 12:25:20.374733] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:20.375146] D [MSGID: 0] [glusterd-syncop.c:1449:gd_commit_op_phase] 0-management: Sent commit op req for 'Volume Status' to 1 peers
[2016-03-22 12:25:20.375193] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:20.375223] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:20.375268] D [MSGID: 0] [glusterd-op-sm.c:4244:glusterd_op_volume_dict_uuid_to_hostname] 0-management: Got uuid /opt/lvmdir/c2/brick
[2016-03-22 12:25:20.375305] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:25:20.375300] D [MSGID: 0] [glusterd-op-sm.c:4244:glusterd_op_volume_dict_uuid_to_hostname] 0-management: Got uuid /opt/lvmdir/c2/brick
[2016-03-22 12:25:20.375304] D [MSGID: 0] [glusterd-op-sm.c:4244:glusterd_op_volume_dict_uuid_to_hostname] 0-management: Got uuid 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:20.375476] D [MSGID: 0] [glusterd-op-sm.c:4256:glusterd_op_volume_dict_uuid_to_hostname] 0-management: 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2 -> localhost
[2016-03-22 12:25:20.375510] D [MSGID: 0] [glusterd-op-sm.c:4244:glusterd_op_volume_dict_uuid_to_hostname] 0-management: Got uuid 356dabed-4314-48be-9eef-d2711d01270d
[2016-03-22 12:25:20.375542] D [MSGID: 0] [glusterd-op-sm.c:4256:glusterd_op_volume_dict_uuid_to_hostname] 0-management: 356dabed-4314-48be-9eef-d2711d01270d -> 10.32.0.48
[2016-03-22 12:25:20.375572] D [MSGID: 0] [glusterd-op-sm.c:4270:glusterd_op_volume_dict_uuid_to_hostname] 0-management: Returning 0
[2016-03-22 12:25:20.375722] D [MSGID: 0] [glusterd-syncop.c:510:gd_syncop_mgmt_v3_unlock] 0-glusterd: Returning 0
[2016-03-22 12:25:20.378467] D [MSGID: 0] [glusterd-syncop.c:1558:gd_unlock_op_phase] 0-management: Sent unlock op req for 'Volume Status' to 1 peers. Returning 0
[2016-03-22 12:25:20.378529] D [MSGID: 0] [glusterd-locks.c:669:glusterd_mgmt_v3_unlock] 0-management: Trying to release lock of vol c_glusterfs for 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2 as c_glusterfs_vol
[2016-03-22 12:25:20.378572] D [MSGID: 0] [glusterd-locks.c:714:glusterd_mgmt_v3_unlock] 0-management: Lock for vol c_glusterfs successfully released
[2016-03-22 12:25:20.378610] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : 00999cc1-70ab-410c-b2dd-b5e197de6f90
[2016-03-22 12:25:20.378638] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.378672] D [MSGID: 0] [glusterd-op-sm.c:360:glusterd_clear_txn_opinfo] 0-management: Successfully cleared opinfo for transaction ID : 00999cc1-70ab-410c-b2dd-b5e197de6f90
[2016-03-22 12:25:20.378698] D [MSGID: 0] [glusterd-op-sm.c:364:glusterd_clear_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.378879] D [MSGID: 0] [glusterd-rpc-ops.c:205:glusterd_op_send_cli_response] 0-management: Returning 0
[2016-03-22 12:25:20.381072] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:20.381112] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:20.381167] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=9 from gen=25 to slot->gen=26, fd=18, slot->fd=18
[2016-03-22 12:25:20.432347] I [MSGID: 106484] [glusterd-brick-ops.c:836:__glusterd_handle_remove_brick] 0-management: Received rem brick req
[2016-03-22 12:25:20.432458] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:20.432487] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:20.432517] I [MSGID: 106062] [glusterd-brick-ops.c:914:__glusterd_handle_remove_brick] 0-management: request to change replica-count to 1
[2016-03-22 12:25:20.432577] D [MSGID: 0] [glusterd-brick-ops.c:1038:__glusterd_handle_remove_brick] 0-management: Remove brick count 1 brick: 10.32.0.48:/opt/lvmdir/c2/brick
[2016-03-22 12:25:20.432626] D [MSGID: 0] [glusterd-utils.c:884:glusterd_brickinfo_new] 0-management: Returning 0
[2016-03-22 12:25:20.432662] D [MSGID: 0] [glusterd-utils.c:1041:glusterd_brickinfo_new_from_brick] 0-management: Returning 0
[2016-03-22 12:25:20.432696] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.0.48 found.. state: 3
[2016-03-22 12:25:20.432724] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:20.432753] D [MSGID: 0] [glusterd-utils.c:1326:glusterd_volume_brickinfo_get] 0-management: Found brick 10.32.0.48:/opt/lvmdir/c2/brick in volume c_glusterfs
[2016-03-22 12:25:20.432825] D [MSGID: 0] [glusterd-utils.c:1335:glusterd_volume_brickinfo_get] 0-management: Returning 0
[2016-03-22 12:25:20.432853] D [MSGID: 0] [glusterd-utils.c:1359:glusterd_volume_brickinfo_get_by_brick] 0-glusterd: Returning 0
[2016-03-22 12:25:20.432978] D [MSGID: 0] [glusterd-op-sm.c:214:glusterd_generate_txn_id] 0-management: Transaction_id = 1d126f79-deb2-4cf1-bbd8-1651ee9183c3
[2016-03-22 12:25:20.433021] D [MSGID: 0] [glusterd-op-sm.c:311:glusterd_set_txn_opinfo] 0-management: Successfully set opinfo for transaction ID : 1d126f79-deb2-4cf1-bbd8-1651ee9183c3
[2016-03-22 12:25:20.433048] D [MSGID: 0] [glusterd-op-sm.c:318:glusterd_set_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.433078] D [MSGID: 0] [glusterd-syncop.c:1767:gd_sync_task_begin] 0-management: Transaction ID : 1d126f79-deb2-4cf1-bbd8-1651ee9183c3
[2016-03-22 12:25:20.433120] D [MSGID: 0] [glusterd-locks.c:562:glusterd_mgmt_v3_lock] 0-management: Trying to acquire lock of vol c_glusterfs for 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2 as c_glusterfs_vol
[2016-03-22 12:25:20.433817] D [MSGID: 0] [glusterd-locks.c:618:glusterd_mgmt_v3_lock] 0-management: Lock for vol c_glusterfs successfully held by 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:20.433989] D [MSGID: 0] [glusterd-syncop.c:411:gd_syncop_mgmt_v3_lock] 0-glusterd: Returning 0
[2016-03-22 12:25:20.437782] D [MSGID: 0] [glusterd-syncop.c:1205:gd_lock_op_phase] 0-management: Sent lock op req for 'Volume Remove brick' to 1 peers. Returning 0
[2016-03-22 12:25:20.437867] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:20.437895] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:20.437950] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:20.437978] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:20.438024] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:20.438053] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:20.438089] D [MSGID: 0] [glusterd-brick-ops.c:2084:glusterd_op_stage_remove_brick] 0-management: Returning 0
[2016-03-22 12:25:20.438117] D [MSGID: 0] [glusterd-op-sm.c:5515:glusterd_op_stage_validate] 0-management: OP = 9. Returning 0
[2016-03-22 12:25:20.438296] D [MSGID: 0] [glusterd-syncop.c:1312:gd_stage_op_phase] 0-management: Sent stage op req for 'Volume Remove brick' to 1 peers
[2016-03-22 12:25:20.441629] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:20.441757] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:20.441794] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:20.441826] I [MSGID: 106062] [glusterd-op-sm.c:5745:glusterd_bricks_select_remove_brick] 0-management: force flag is not set
[2016-03-22 12:25:20.441854] D [MSGID: 0] [glusterd-op-sm.c:7010:glusterd_op_bricks_select] 0-management: Returning 0
[2016-03-22 12:25:20.441882] D [MSGID: 0] [glusterd-syncop.c:1709:gd_brick_op_phase] 0-management: Sent op req to 0 bricks
[2016-03-22 12:25:20.442332] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:20.442386] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:20.442785] W [dict.c:592:dict_unref] (-->/usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_op_commit_perform-0x1350d4) [0x3fff8806875c] -->/usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_op_remove_brick-0xa98b0) [0x3fff880f86c0] -->/usr/lib64/libglusterfs.so.0(dict_unref-0xc08c0) [0x3fff8b71e520] ) 0-dict: dict is NULL [Invalid argument]
[2016-03-22 12:25:20.442864] D [MSGID: 0] [glusterd-utils.c:884:glusterd_brickinfo_new] 0-management: Returning 0
[2016-03-22 12:25:20.442900] D [MSGID: 0] [glusterd-utils.c:1041:glusterd_brickinfo_new_from_brick] 0-management: Returning 0
[2016-03-22 12:25:20.442932] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.0.48 found.. state: 3
[2016-03-22 12:25:20.442959] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:20.442987] D [MSGID: 0] [glusterd-utils.c:1326:glusterd_volume_brickinfo_get] 0-management: Found brick 10.32.0.48:/opt/lvmdir/c2/brick in volume c_glusterfs
[2016-03-22 12:25:20.443014] D [MSGID: 0] [glusterd-utils.c:1335:glusterd_volume_brickinfo_get] 0-management: Returning 0
[2016-03-22 12:25:20.443040] D [MSGID: 0] [glusterd-utils.c:1359:glusterd_volume_brickinfo_get_by_brick] 0-glusterd: Returning 0
[2016-03-22 12:25:20.443069] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.0.48 found.. state: 3
[2016-03-22 12:25:20.443112] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:20.443139] D [MSGID: 0] [glusterd-utils.c:925:glusterd_resolve_brick] 0-management: Returning 0
[2016-03-22 12:25:20.443458] D [MSGID: 0] [store.c:460:gf_store_handle_destroy] 0-: Returning 0
[2016-03-22 12:25:20.443488] D [MSGID: 0] [glusterd-store.c:620:glusterd_store_delete_brick] 0-management: Returning with 0
[2016-03-22 12:25:20.443534] D [MSGID: 0] [glusterd-utils.c:5542:glusterd_brick_stop] 0-management: returning 0 
[2016-03-22 12:25:20.443561] D [MSGID: 0] [glusterd-brick-ops.c:1486:glusterd_op_perform_remove_brick] 0-glusterd: Returning 0
[2016-03-22 12:25:20.443591] I [MSGID: 106062] [glusterd-brick-ops.c:2556:glusterd_op_remove_brick] 0-management: changing replica count 2 to 1 on volume c_glusterfs
[2016-03-22 12:25:20.443765] D [MSGID: 0] [glusterd-volgen.c:5109:generate_brick_volfiles] 0-management: Found a brick - 10.32.1.144:/opt/lvmdir/c2/brick
[2016-03-22 12:25:20.444467] D [MSGID: 0] [glusterd-volgen.c:1268:server_check_marker_off] 0-glusterd: Returning 0
[2016-03-22 12:25:20.445676] D [MSGID: 0] [glusterd-volgen.c:5120:generate_brick_volfiles] 0-management: Returning 0
[2016-03-22 12:25:20.446183] D [MSGID: 0] [glusterd-peer-utils.c:133:glusterd_peerinfo_find_by_hostname] 0-management: Unable to find friend: 10.32.1.144
[2016-03-22 12:25:20.446290] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.1.144 
[2016-03-22 12:25:20.446996] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.1.144 is local address at interface efd
[2016-03-22 12:25:20.447031] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:20.447060] D [MSGID: 0] [glusterd-utils.c:1326:glusterd_volume_brickinfo_get] 0-management: Found brick 10.32.1.144:/opt/lvmdir/c2/brick in volume c_glusterfs
[2016-03-22 12:25:20.447087] D [MSGID: 0] [glusterd-utils.c:1335:glusterd_volume_brickinfo_get] 0-management: Returning 0
[2016-03-22 12:25:20.450427] D [MSGID: 0] [glusterd-peer-utils.c:133:glusterd_peerinfo_find_by_hostname] 0-management: Unable to find friend: 10.32.1.144
[2016-03-22 12:25:20.450543] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.1.144 
[2016-03-22 12:25:20.451293] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.1.144 is local address at interface efd
[2016-03-22 12:25:20.451328] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:20.451358] D [MSGID: 0] [glusterd-utils.c:1326:glusterd_volume_brickinfo_get] 0-management: Found brick 10.32.1.144:/opt/lvmdir/c2/brick in volume c_glusterfs
[2016-03-22 12:25:20.451385] D [MSGID: 0] [glusterd-utils.c:1335:glusterd_volume_brickinfo_get] 0-management: Returning 0
[2016-03-22 12:25:20.453247] D [MSGID: 0] [glusterd-store.c:1058:glusterd_store_create_volume_dir] 0-management: Returning with 0
[2016-03-22 12:25:20.453446] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:20.462351] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 18 times between [2016-03-22 12:25:20.453446] and [2016-03-22 12:25:20.454521]
[2016-03-22 12:25:20.462349] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:25:20.462644] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:20.462674] D [MSGID: 0] [glusterd-store.c:440:glusterd_store_snapd_write] 0-management: Returning 0
[2016-03-22 12:25:20.477657] D [MSGID: 0] [glusterd-store.c:503:glusterd_store_perform_snapd_store] 0-management: Returning 0
[2016-03-22 12:25:20.477715] D [MSGID: 0] [glusterd-store.c:568:glusterd_store_snapd_info] 0-management: Returning with 0
[2016-03-22 12:25:20.477792] D [MSGID: 0] [glusterd-store.c:753:_storeopts] 0-management: Storing in volinfo:key= nfs.disable, val=on
[2016-03-22 12:25:20.477914] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:20.477952] D [MSGID: 0] [glusterd-store.c:753:_storeopts] 0-management: Storing in volinfo:key= network.ping-timeout, val=4
[2016-03-22 12:25:20.478031] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:20.478068] D [MSGID: 0] [glusterd-store.c:753:_storeopts] 0-management: Storing in volinfo:key= performance.readdir-ahead, val=on
[2016-03-22 12:25:20.478147] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:20.478177] D [MSGID: 0] [glusterd-store.c:1104:glusterd_store_volinfo_write] 0-management: Returning 0
[2016-03-22 12:25:20.478263] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:20.478906] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 8 times between [2016-03-22 12:25:20.478263] and [2016-03-22 12:25:20.478902]
[2016-03-22 12:25:20.478905] D [MSGID: 0] [glusterd-store.c:414:glusterd_store_brickinfo_write] 0-management: Returning 0
[2016-03-22 12:25:20.478998] D [MSGID: 0] [glusterd-store.c:464:glusterd_store_perform_brick_store] 0-management: Returning 0
[2016-03-22 12:25:20.479030] D [MSGID: 0] [glusterd-store.c:533:glusterd_store_brickinfo] 0-management: Returning with 0
[2016-03-22 12:25:20.479057] D [MSGID: 0] [glusterd-store.c:1325:glusterd_store_brickinfos] 0-management: Returning 0
[2016-03-22 12:25:20.479084] D [MSGID: 0] [glusterd-store.c:1439:glusterd_store_perform_volume_store] 0-management: Returning 0
[2016-03-22 12:25:20.509762] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:20.510015] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 3 times between [2016-03-22 12:25:20.509762] and [2016-03-22 12:25:20.510009]
[2016-03-22 12:25:20.510013] D [MSGID: 0] [glusterd-store.c:1383:glusterd_store_node_state_write] 0-management: Returning 0
[2016-03-22 12:25:20.525203] D [MSGID: 0] [glusterd-store.c:1411:glusterd_store_perform_node_state_store] 0-management: Returning 0
[2016-03-22 12:25:20.527841] D [MSGID: 0] [glusterd-utils.c:2052:glusterd_volume_compute_cksum] 0-management: Returning with 0
[2016-03-22 12:25:20.527894] D [MSGID: 0] [glusterd-store.c:1645:glusterd_store_volinfo] 0-management: Returning 0
[2016-03-22 12:25:20.527979] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already stopped
[2016-03-22 12:25:20.528688] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8b729f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff8b6d3990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff8b6cf430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff88141ab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff881428b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:25:20.528776] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:20.528807] D [MSGID: 0] [glusterd-nfs-svc.c:135:glusterd_nfssvc_stop] 0-management: Returning 0
[2016-03-22 12:25:20.529017] D [MSGID: 0] [glusterd-volgen.c:4809:build_nfs_graph] 0-glusterd: Returning 0
[2016-03-22 12:25:20.529912] D [MSGID: 0] [glusterd-nfs-svc.c:106:glusterd_nfssvc_manager] 0-management: Returning 0
[2016-03-22 12:25:20.529954] D [MSGID: 0] [glusterd-shd-svc.c:139:glusterd_shdsvc_manager] 0-management: Returning 0
[2016-03-22 12:25:20.530138] D [MSGID: 0] [glusterd-quotad-svc.c:111:glusterd_quotadsvc_manager] 0-management: Returning 0
[2016-03-22 12:25:20.530252] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: bitd already stopped
[2016-03-22 12:25:20.531039] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8b729f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff8b6d3990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff8b6cf430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff88141ab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff881428b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:25:20.531124] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:20.531155] D [MSGID: 0] [glusterd-bitd-svc.c:109:glusterd_bitdsvc_manager] 0-management: Returning 0
[2016-03-22 12:25:20.531286] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: scrub already stopped
[2016-03-22 12:25:20.531893] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8b729f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff8b6d3990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff8b6cf430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff88141ab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff881428b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:25:20.531986] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:20.532018] D [MSGID: 0] [glusterd-scrub-svc.c:104:glusterd_scrubsvc_manager] 0-management: Returning 0
[2016-03-22 12:25:20.532098] D [MSGID: 0] [glusterd-op-sm.c:5642:glusterd_op_commit_perform] 0-management: Returning 0
[2016-03-22 12:25:20.532282] D [MSGID: 0] [glusterd-handshake.c:409:glusterd_get_args_from_dict] 0-management: No brick name present
[2016-03-22 12:25:20.532321] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning -1
[2016-03-22 12:25:20.532499] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 127.0.0.1 
[2016-03-22 12:25:20.532727] D [MSGID: 0] [glusterd-handshake.c:414:glusterd_get_args_from_dict] 0-management: brick_name = /opt/lvmdir/c2/brick
[2016-03-22 12:25:20.532772] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning -1
[2016-03-22 12:25:20.532883] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.1.144 
[2016-03-22 12:25:20.533773] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.1.144 is local address at interface efd
[2016-03-22 12:25:20.533823] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning -1
[2016-03-22 12:25:20.533861] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:20.533934] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:20.534042] D [MSGID: 0] [glusterd-handshake.c:841:__server_getspec] 0-management: Look for missing snap creates for /opt/lvmdir/c2/brick
[2016-03-22 12:25:20.534277] D [MSGID: 0] [glusterd-handshake.c:409:glusterd_get_args_from_dict] 0-management: No brick name present
[2016-03-22 12:25:20.534318] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:20.534346] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:20.534469] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.1.144 
[2016-03-22 12:25:20.535027] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.1.144 is local address at interface efd
[2016-03-22 12:25:20.535064] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:20.535093] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:20.540280] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:20.540317] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:20.540359] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=10 from gen=7 to slot->gen=8, fd=19, slot->fd=19
[2016-03-22 12:25:20.637729] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:20.637861] D [MSGID: 0] [glusterd-syncop.c:1449:gd_commit_op_phase] 0-management: Sent commit op req for 'Volume Remove brick' to 1 peers
[2016-03-22 12:25:20.637906] D [MSGID: 0] [glusterd-op-sm.c:4660:glusterd_op_modify_op_ctx] 0-management: op_ctx modification not required
[2016-03-22 12:25:20.638035] D [MSGID: 0] [glusterd-syncop.c:510:gd_syncop_mgmt_v3_unlock] 0-glusterd: Returning 0
[2016-03-22 12:25:20.643081] D [MSGID: 0] [glusterd-syncop.c:1558:gd_unlock_op_phase] 0-management: Sent unlock op req for 'Volume Remove brick' to 1 peers. Returning 0
[2016-03-22 12:25:20.643152] D [MSGID: 0] [glusterd-locks.c:669:glusterd_mgmt_v3_unlock] 0-management: Trying to release lock of vol c_glusterfs for 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2 as c_glusterfs_vol
[2016-03-22 12:25:20.643196] D [MSGID: 0] [glusterd-locks.c:714:glusterd_mgmt_v3_unlock] 0-management: Lock for vol c_glusterfs successfully released
[2016-03-22 12:25:20.643234] D [MSGID: 0] [glusterd-op-sm.c:255:glusterd_get_txn_opinfo] 0-management: Successfully got opinfo for transaction ID : 1d126f79-deb2-4cf1-bbd8-1651ee9183c3
[2016-03-22 12:25:20.643262] D [MSGID: 0] [glusterd-op-sm.c:259:glusterd_get_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.643296] D [MSGID: 0] [glusterd-op-sm.c:360:glusterd_clear_txn_opinfo] 0-management: Successfully cleared opinfo for transaction ID : 1d126f79-deb2-4cf1-bbd8-1651ee9183c3
[2016-03-22 12:25:20.643323] D [MSGID: 0] [glusterd-op-sm.c:364:glusterd_clear_txn_opinfo] 0-management: Returning 0
[2016-03-22 12:25:20.643441] D [MSGID: 0] [glusterd-rpc-ops.c:205:glusterd_op_send_cli_response] 0-management: Returning 0
[2016-03-22 12:25:20.645955] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:20.646072] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:20.646117] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=9 from gen=28 to slot->gen=29, fd=18, slot->fd=18
[2016-03-22 12:25:20.691532] I [MSGID: 106487] [glusterd-handler.c:1411:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2016-03-22 12:25:20.693843] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:20.693895] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:20.693941] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=9 from gen=31 to slot->gen=32, fd=18, slot->fd=18
[2016-03-22 12:25:20.747422] I [MSGID: 106487] [glusterd-handler.c:1290:__glusterd_handle_cli_deprobe] 0-glusterd: Received CLI deprobe req
[2016-03-22 12:25:20.747516] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.0.48 found.. state: 3
[2016-03-22 12:25:20.747555] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:20.747594] D [MSGID: 0] [glusterd-peer-utils.c:350:glusterd_chk_peers_connected_befriended] 0-management: Returning TRUE
[2016-03-22 12:25:20.747632] D [MSGID: 0] [glusterd-utils.c:6763:glusterd_friend_contains_vol_bricks] 0-management: Returning 0
[2016-03-22 12:25:20.747674] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:20.747719] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_INIT_REMOVE_FRIEND'
[2016-03-22 12:25:20.747758] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:20.747795] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_INIT_REMOVE_FRIEND'
[2016-03-22 12:25:20.747833] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:20.748026] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:25:20.747873] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:20.748025] D [MSGID: 0] [glusterd-rpc-ops.c:1628:glusterd_rpc_friend_remove] 0-glusterd: Returning 0
[2016-03-22 12:25:20.748113] D [MSGID: 0] [glusterd-sm.c:491:glusterd_ac_send_friend_remove_req] 0-glusterd: Returning with 0
[2016-03-22 12:25:20.749256] I [MSGID: 106493] [glusterd-rpc-ops.c:598:__glusterd_friend_remove_cbk] 0-glusterd: Received ACC from uuid: 356dabed-4314-48be-9eef-d2711d01270d, host: 10.32.1.144, port: 0
[2016-03-22 12:25:20.749335] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:20.749375] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_REMOVE_FRIEND'
[2016-03-22 12:25:20.749507] D [MSGID: 0] [glusterd-handler.c:4009:glusterd_xfer_cli_deprobe_resp] 0-management: Responded to CLI, ret: 0
[2016-03-22 12:25:20.749547] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:20.749582] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_REMOVE_FRIEND'
[2016-03-22 12:25:20.749617] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:20.749654] D [MSGID: 0] [glusterd-utils.c:6763:glusterd_friend_contains_vol_bricks] 0-management: Returning 0
[2016-03-22 12:25:20.749687] D [MSGID: 0] [glusterd-utils.c:6799:glusterd_friend_remove_cleanup_vols] 0-management: Returning 0
[2016-03-22 12:25:20.749883] D [MSGID: 0] [glusterd-utils.c:173:glusterd_lock] 0-management: Cluster lock held by 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:20.749954] D [MSGID: 0] [glusterd-sm.c:188:glusterd_broadcast_friend_delete] 0-glusterd: Returning with 0
[2016-03-22 12:25:20.750140] D [MSGID: 0] [store.c:460:gf_store_handle_destroy] 0-: Returning 0
[2016-03-22 12:25:20.750183] D [MSGID: 0] [glusterd-store.c:3777:glusterd_store_delete_peerinfo] 0-management: Returning with 0
[2016-03-22 12:25:20.751574] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:20.751629] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:20.751680] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=9 from gen=34 to slot->gen=35, fd=18, slot->fd=18
[2016-03-22 12:25:20.805016] I [MSGID: 106487] [glusterd-handler.c:1411:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2016-03-22 12:25:20.807872] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:20.807908] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:20.807955] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=9 from gen=37 to slot->gen=38, fd=18, slot->fd=18
[2016-03-22 12:25:20.859605] I [MSGID: 106487] [glusterd-handler.c:1411:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2016-03-22 12:25:20.862457] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:20.862505] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:20.862548] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=9 from gen=40 to slot->gen=41, fd=18, slot->fd=18
[2016-03-22 12:25:20.915030] I [MSGID: 106487] [glusterd-handler.c:1178:__glusterd_handle_cli_probe] 0-glusterd: Received CLI probe req 10.32.0.48 24007
[2016-03-22 12:25:20.915200] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.0.48 
[2016-03-22 12:25:20.917175] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.0.48 " repeated 2 times between [2016-03-22 12:25:20.915200] and [2016-03-22 12:25:20.916600]
[2016-03-22 12:25:20.917174] D [MSGID: 0] [common-utils.c:3149:gf_is_local_addr] 0-management: 10.32.0.48 is not local
[2016-03-22 12:25:20.917324] D [MSGID: 0] [glusterd-peer-utils.c:133:glusterd_peerinfo_find_by_hostname] 0-management: Unable to find friend: 10.32.0.48
[2016-03-22 12:25:20.917435] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:25:20.917432] D [MSGID: 0] [glusterd-peer-utils.c:133:glusterd_peerinfo_find_by_hostname] 0-management: Unable to find friend: 10.32.0.48
[2016-03-22 12:25:20.917435] D [MSGID: 0] [glusterd-peer-utils.c:247:glusterd_peerinfo_find] 0-management: Unable to find hostname: 10.32.0.48
[2016-03-22 12:25:20.917519] I [MSGID: 106129] [glusterd-handler.c:3600:glusterd_probe_begin] 0-glusterd: Unable to find peerinfo for host: 10.32.0.48 (24007)
[2016-03-22 12:25:20.917552] D [MSGID: 0] [glusterd-peer-utils.c:458:glusterd_peer_hostname_new] 0-glusterd: Returning 0
[2016-03-22 12:25:20.917585] D [MSGID: 0] [glusterd-utils.c:5974:glusterd_sm_tr_log_init] 0-glusterd: returning 0
[2016-03-22 12:25:20.917626] D [MSGID: 0] [glusterd-store.c:3802:glusterd_store_create_peer_dir] 0-glusterd: Returning with 0
[2016-03-22 12:25:20.924767] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:25:20.924987] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:20.925173] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 2 times between [2016-03-22 12:25:20.924987] and [2016-03-22 12:25:20.925168]
[2016-03-22 12:25:20.925172] D [MSGID: 0] [glusterd-store.c:3931:glusterd_store_peer_write] 0-glusterd: Returning with 0
[2016-03-22 12:25:20.940836] D [MSGID: 0] [glusterd-store.c:3957:glusterd_store_perform_peer_store] 0-glusterd: Returning 0
[2016-03-22 12:25:20.940876] D [MSGID: 0] [glusterd-store.c:3978:glusterd_store_peerinfo] 0-glusterd: Returning with 0
[2016-03-22 12:25:20.940954] D [MSGID: 0] [glusterd-handler.c:3391:glusterd_transport_inet_options_build] 0-glusterd: Returning 0
[2016-03-22 12:25:20.941004] I [rpc-clnt.c:984:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2016-03-22 12:25:20.941026] D [rpc-clnt.c:996:rpc_clnt_connection_init] 0-management: setting ping-timeout to 30
[2016-03-22 12:25:20.941055] D [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3.7.6/rpc-transport/socket.so
[2016-03-22 12:25:20.941177] D [socket.c:3845:socket_init] 0-management: Configued transport.tcp-user-timeout=-1
[2016-03-22 12:25:20.941204] D [socket.c:3928:socket_init] 0-management: SSL support on the I/O path is NOT enabled
[2016-03-22 12:25:20.941223] D [socket.c:3931:socket_init] 0-management: SSL support for glusterd is NOT enabled
[2016-03-22 12:25:20.941241] D [socket.c:3948:socket_init] 0-management: using system polling thread
[2016-03-22 12:25:20.941838] D [MSGID: 0] [common-utils.c:333:gf_resolve_ip6] 0-resolver: returning ip-10.32.0.48 (port-24007) for hostname: 10.32.0.48 and port: 24007
[2016-03-22 12:25:20.941868] D [socket.c:2833:socket_fix_ssl_opts] 0-management: disabling SSL for portmapper connection
[2016-03-22 12:25:20.941916] W [socket.c:869:__socket_keepalive] 0-socket: failed to set TCP_USER_TIMEOUT -1000 on socket 19, Invalid argument
[2016-03-22 12:25:20.941941] E [socket.c:2965:socket_connect] 0-management: Failed to set keep-alive: Invalid argument
[2016-03-22 12:25:20.942167] D [MSGID: 0] [glusterd-handler.c:3323:glusterd_rpc_create] 0-management: returning 0
[2016-03-22 12:25:20.942196] I [MSGID: 106498] [glusterd-handler.c:3528:glusterd_friend_add] 0-management: connect returned 0
[2016-03-22 12:25:20.942224] D [MSGID: 0] [glusterd-handler.c:3642:glusterd_probe_begin] 0-glusterd: returning 100
[2016-03-22 12:25:20.942906] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Establishing Connection
[2016-03-22 12:25:20.942962] D [MSGID: 0] [glusterd-handler.c:5042:__glusterd_peer_rpc_notify] 0-management: got RPC_CLNT_CONNECT
[2016-03-22 12:25:20.942995] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Establishing Connection
[2016-03-22 12:25:20.943683] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8b729f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff8b6d3990] (--> /usr/lib64/libgfrpc.so.0(+0x1a188)[0x3fff8b6d4188] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_submit-0x1a4d8)[0x3fff8b6cf0a8] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_submit_request-0x129078)[0x3fff88074de8] ))))) 0-: 10.32.0.48:24007: ping timer event already removed
[2016-03-22 12:25:20.943897] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:25:20.943881] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Establishing Connection
[2016-03-22 12:25:20.943896] D [MSGID: 0] [glusterd-handshake.c:2098:__glusterd_peer_dump_version_cbk] 0-management: Proceeding to op-version handshake with peer 10.32.0.48
[2016-03-22 12:25:20.943983] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Establishing Connection
[2016-03-22 12:25:21.894501] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:25:21.894471] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Establishing Connection
[2016-03-22 12:25:21.894499] D [MSGID: 0] [glusterd-handshake.c:1660:gd_validate_peer_op_version] 0-management: Peer 10.32.0.48 accepted
[2016-03-22 12:25:21.910151] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Establishing Connection
[2016-03-22 12:25:21.910227] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:25:21.910220] D [MSGID: 0] [glusterd-peer-utils.c:995:glusterd_peerinfo_find_by_generation] 0-management: Friend found... state: Establishing Connection
[2016-03-22 12:25:21.910226] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_CONNECTED'
[2016-03-22 12:25:21.910296] D [MSGID: 0] [glusterd-handshake.c:1613:glusterd_event_connected_inject] 0-glusterd: returning 0
[2016-03-22 12:25:21.910336] D [MSGID: 0] [glusterd-peer-utils.c:235:glusterd_peerinfo_find] 0-management: Unable to find peer by uuid: 00000000-0000-0000-0000-000000000000
[2016-03-22 12:25:21.910366] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.0.48 found.. state: 0
[2016-03-22 12:25:21.910394] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_CONNECTED'
[2016-03-22 12:25:21.910437] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.0.48 found.. state: 0
[2016-03-22 12:25:21.910555] D [MSGID: 0] [glusterd-rpc-ops.c:1477:glusterd_rpc_probe] 0-glusterd: Returning 0
[2016-03-22 12:25:21.910589] D [MSGID: 0] [glusterd-sm.c:413:glusterd_ac_friend_probe] 0-glusterd: Returning with 0
[2016-03-22 12:25:21.910620] D [MSGID: 0] [glusterd-peer-utils.c:235:glusterd_peerinfo_find] 0-management: Unable to find peer by uuid: 00000000-0000-0000-0000-000000000000
[2016-03-22 12:25:21.910648] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.0.48 found.. state: 0
[2016-03-22 12:25:21.910678] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Establishing Connection' to 'Establishing Connection' due to event 'GD_FRIEND_EVENT_CONNECTED'
[2016-03-22 12:25:21.910706] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:21.910750] D [MSGID: 0] [glusterd-peer-utils.c:235:glusterd_peerinfo_find] 0-management: Unable to find peer by uuid: 00000000-0000-0000-0000-000000000000
[2016-03-22 12:25:21.910779] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.0.48 found.. state: 0
[2016-03-22 12:25:21.910821] D [MSGID: 0] [glusterd-store.c:3802:glusterd_store_create_peer_dir] 0-glusterd: Returning with 0
[2016-03-22 12:25:21.911055] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:21.911197] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 2 times between [2016-03-22 12:25:21.911055] and [2016-03-22 12:25:21.911193]
[2016-03-22 12:25:21.911197] D [MSGID: 0] [glusterd-store.c:3931:glusterd_store_peer_write] 0-glusterd: Returning with 0
[2016-03-22 12:25:21.925322] D [MSGID: 0] [glusterd-store.c:3957:glusterd_store_perform_peer_store] 0-glusterd: Returning 0
[2016-03-22 12:25:21.925383] D [MSGID: 0] [glusterd-store.c:3978:glusterd_store_peerinfo] 0-glusterd: Returning with 0
[2016-03-22 12:25:21.935869] I [MSGID: 106511] [glusterd-rpc-ops.c:256:__glusterd_probe_cbk] 0-management: Received probe resp from uuid: 356dabed-4314-48be-9eef-d2711d01270d, host: 10.32.0.48
[2016-03-22 12:25:21.935941] D [MSGID: 0] [glusterd-peer-utils.c:209:glusterd_peerinfo_find_by_uuid] 0-management: Friend with uuid: 356dabed-4314-48be-9eef-d2711d01270d, not found
[2016-03-22 12:25:21.935974] D [MSGID: 0] [glusterd-peer-utils.c:235:glusterd_peerinfo_find] 0-management: Unable to find peer by uuid: 356dabed-4314-48be-9eef-d2711d01270d
[2016-03-22 12:25:21.936004] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.0.48 found.. state: 0
[2016-03-22 12:25:21.936039] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_INIT_FRIEND_REQ'
[2016-03-22 12:25:21.936066] I [MSGID: 106511] [glusterd-rpc-ops.c:416:__glusterd_probe_cbk] 0-glusterd: Received resp to probe req
[2016-03-22 12:25:21.936098] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Establishing Connection
[2016-03-22 12:25:21.936126] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_INIT_FRIEND_REQ'
[2016-03-22 12:25:21.936154] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Establishing Connection
[2016-03-22 12:25:21.936514] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:25:21.936184] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Establishing Connection
[2016-03-22 12:25:21.936513] D [MSGID: 0] [glusterd-utils.c:2504:glusterd_add_volume_to_dict] 0-management: Returning with 0
[2016-03-22 12:25:21.936973] D [MSGID: 0] [glusterd-rpc-ops.c:1579:glusterd_rpc_friend_add] 0-glusterd: Returning 0
[2016-03-22 12:25:21.937008] D [MSGID: 0] [glusterd-sm.c:334:glusterd_ac_friend_add] 0-glusterd: Returning with 0
[2016-03-22 12:25:21.937039] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Establishing Connection
[2016-03-22 12:25:21.937076] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Establishing Connection' to 'Probe Sent to Peer' due to event 'GD_FRIEND_EVENT_INIT_FRIEND_REQ'
[2016-03-22 12:25:21.937104] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:21.937132] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Probe Sent to Peer
[2016-03-22 12:25:21.937192] D [MSGID: 0] [glusterd-store.c:3802:glusterd_store_create_peer_dir] 0-glusterd: Returning with 0
[2016-03-22 12:25:21.937232] D [MSGID: 0] [store.c:460:gf_store_handle_destroy] 0-: Returning 0
[2016-03-22 12:25:21.944313] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:25:21.944583] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:21.944742] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 2 times between [2016-03-22 12:25:21.944583] and [2016-03-22 12:25:21.944736]
[2016-03-22 12:25:21.944740] D [MSGID: 0] [glusterd-store.c:3931:glusterd_store_peer_write] 0-glusterd: Returning with 0
[2016-03-22 12:25:21.958916] D [MSGID: 0] [glusterd-store.c:3957:glusterd_store_perform_peer_store] 0-glusterd: Returning 0
[2016-03-22 12:25:21.958977] D [MSGID: 0] [glusterd-store.c:3978:glusterd_store_peerinfo] 0-glusterd: Returning with 0
[2016-03-22 12:25:21.959155] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Probe Sent to Peer
[2016-03-22 12:25:22.047236] I [MSGID: 106493] [glusterd-rpc-ops.c:480:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 356dabed-4314-48be-9eef-d2711d01270d, host: 10.32.0.48, port: 0
[2016-03-22 12:25:22.047303] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Probe Sent to Peer
[2016-03-22 12:25:22.047340] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_RCVD_ACC'
[2016-03-22 12:25:22.047461] D [MSGID: 0] [glusterd-handler.c:3915:glusterd_xfer_cli_probe_resp] 0-management: Responded to CLI, ret: 0
[2016-03-22 12:25:22.047496] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Probe Sent to Peer
[2016-03-22 12:25:22.047524] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_RCVD_ACC'
[2016-03-22 12:25:22.047552] D [MSGID: 0] [glusterd-sm.c:203:glusterd_ac_none] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.047580] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Probe Sent to Peer
[2016-03-22 12:25:22.047613] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Probe Sent to Peer' to 'Accepted peer request' due to event 'GD_FRIEND_EVENT_RCVD_ACC'
[2016-03-22 12:25:22.047641] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:22.047670] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Accepted peer request
[2016-03-22 12:25:22.047716] D [MSGID: 0] [glusterd-store.c:3802:glusterd_store_create_peer_dir] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.047967] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:22.048122] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 2 times between [2016-03-22 12:25:22.047967] and [2016-03-22 12:25:22.048117]
[2016-03-22 12:25:22.048121] D [MSGID: 0] [glusterd-store.c:3931:glusterd_store_peer_write] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.049726] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:22.049758] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:22.049800] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=9 from gen=43 to slot->gen=44, fd=18, slot->fd=18
[2016-03-22 12:25:22.062619] D [MSGID: 0] [glusterd-store.c:3957:glusterd_store_perform_peer_store] 0-glusterd: Returning 0
[2016-03-22 12:25:22.062680] D [MSGID: 0] [glusterd-store.c:3978:glusterd_store_peerinfo] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.084069] I [MSGID: 106163] [glusterd-handshake.c:1193:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 30706
[2016-03-22 12:25:22.084352] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:22.122170] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
[2016-03-22 12:25:22.084459] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:22.122168] I [MSGID: 106490] [glusterd-handler.c:2884:__glusterd_handle_probe_query] 0-glusterd: Received probe from uuid: 356dabed-4314-48be-9eef-d2711d01270d
[2016-03-22 12:25:22.122797] E [MSGID: 101076] [common-utils.c:2954:gf_get_hostname_from_ip] 0-common-utils: Could not lookup hostname of 10.32.0.48 : Temporary failure in name resolution
[2016-03-22 12:25:22.122851] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Accepted peer request
[2016-03-22 12:25:22.122967] I [MSGID: 106493] [glusterd-handler.c:2947:__glusterd_handle_probe_query] 0-glusterd: Responded to 10.32.0.48, op_ret: 0, op_errno: 0, ret: 0
[2016-03-22 12:25:22.140622] I [MSGID: 106490] [glusterd-handler.c:2539:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 356dabed-4314-48be-9eef-d2711d01270d
[2016-03-22 12:25:22.141062] E [MSGID: 101076] [common-utils.c:2954:gf_get_hostname_from_ip] 0-common-utils: Could not lookup hostname of 10.32.0.48 : Temporary failure in name resolution
[2016-03-22 12:25:22.141111] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Accepted peer request
[2016-03-22 12:25:22.141312] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_RCVD_FRIEND_REQ'
[2016-03-22 12:25:22.141346] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Accepted peer request
[2016-03-22 12:25:22.141376] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_RCVD_FRIEND_REQ'
[2016-03-22 12:25:22.141430] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Accepted peer request
[2016-03-22 12:25:22.141483] D [MSGID: 0] [glusterd-utils.c:2830:import_prdict_dict] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.141527] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:22.141556] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:22.141595] D [MSGID: 0] [glusterd-utils.c:2757:glusterd_compare_friend_volume] 0-management: quota checksum absent for volume c_glusterfs in peer 10.32.0.48's response
[2016-03-22 12:25:22.141625] D [MSGID: 0] [glusterd-utils.c:2776:glusterd_compare_friend_volume] 0-management: Returning with ret: 0, status: 1
[2016-03-22 12:25:22.141654] D [MSGID: 0] [glusterd-utils.c:4078:glusterd_compare_friend_data] 0-management: Returning with ret: 0, status: 1
[2016-03-22 12:25:22.156228] D [MSGID: 0] [glusterd-sm.c:1163:glusterd_friend_sm_inject_event] 0-glusterd: Enqueue event: 'GD_FRIEND_EVENT_LOCAL_ACC'
[2016-03-22 12:25:22.156488] I [MSGID: 106493] [glusterd-handler.c:3780:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 10.32.0.48 (0), ret: 0
[2016-03-22 12:25:22.156540] D [MSGID: 0] [glusterd-sm.c:911:glusterd_ac_handle_friend_add_req] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.156585] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Accepted peer request
[2016-03-22 12:25:22.156629] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Accepted peer request' to 'Accepted peer request' due to event 'GD_FRIEND_EVENT_RCVD_FRIEND_REQ'
[2016-03-22 12:25:22.156677] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:22.156707] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Accepted peer request
[2016-03-22 12:25:22.156750] D [MSGID: 0] [glusterd-store.c:3802:glusterd_store_create_peer_dir] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.157082] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:22.157270] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 2 times between [2016-03-22 12:25:22.157082] and [2016-03-22 12:25:22.157265]
[2016-03-22 12:25:22.157269] D [MSGID: 0] [glusterd-store.c:3931:glusterd_store_peer_write] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.172238] D [MSGID: 0] [glusterd-store.c:3957:glusterd_store_perform_peer_store] 0-glusterd: Returning 0
[2016-03-22 12:25:22.172299] D [MSGID: 0] [glusterd-store.c:3978:glusterd_store_peerinfo] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.172516] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Accepted peer request
[2016-03-22 12:25:22.172548] D [MSGID: 0] [glusterd-sm.c:1255:glusterd_friend_sm] 0-glusterd: Dequeued event of type: 'GD_FRIEND_EVENT_LOCAL_ACC'
[2016-03-22 12:25:22.172577] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Accepted peer request
[2016-03-22 12:25:22.172655] D [MSGID: 0] [glusterd-peer-utils.c:615:gd_add_friend_to_dict] 0-management: Returning 0
[2016-03-22 12:25:22.172815] D [MSGID: 0] [glusterd-rpc-ops.c:1674:glusterd_rpc_friend_update] 0-glusterd: Returning 0
[2016-03-22 12:25:22.172863] D [MSGID: 0] [glusterd-sm.c:590:glusterd_ac_send_friend_update] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.172933] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Accepted peer request
[2016-03-22 12:25:22.172966] D [MSGID: 0] [glusterd-utils.c:6023:glusterd_sm_tr_log_transition_add] 0-management: Transitioning from 'Accepted peer request' to 'Peer in Cluster' due to event 'GD_FRIEND_EVENT_LOCAL_ACC'
[2016-03-22 12:25:22.172994] D [MSGID: 0] [glusterd-utils.c:6025:glusterd_sm_tr_log_transition_add] 0-management: returning 0
[2016-03-22 12:25:22.173022] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:22.173108] D [MSGID: 0] [glusterd-store.c:3802:glusterd_store_create_peer_dir] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.173340] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:22.173513] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 2 times between [2016-03-22 12:25:22.173340] and [2016-03-22 12:25:22.173508]
[2016-03-22 12:25:22.173512] D [MSGID: 0] [glusterd-store.c:3931:glusterd_store_peer_write] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.189738] D [MSGID: 0] [glusterd-store.c:3957:glusterd_store_perform_peer_store] 0-glusterd: Returning 0
[2016-03-22 12:25:22.189800] D [MSGID: 0] [glusterd-store.c:3978:glusterd_store_peerinfo] 0-glusterd: Returning with 0
[2016-03-22 12:25:22.189901] I [MSGID: 106493] [glusterd-rpc-ops.c:694:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 356dabed-4314-48be-9eef-d2711d01270d
[2016-03-22 12:25:22.191749] D [MSGID: 0] [glusterd-utils.c:173:glusterd_lock] 0-management: Cluster lock held by 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:22.191965] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:22.192000] I [MSGID: 106492] [glusterd-handler.c:2715:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 356dabed-4314-48be-9eef-d2711d01270d
[2016-03-22 12:25:22.192062] I [MSGID: 106502] [glusterd-handler.c:2760:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2016-03-22 12:25:22.192554] D [MSGID: 0] [glusterd-snapshot-utils.c:3595:glusterd_is_snapd_enabled] 0-management: Key features.uss not present in the dict for volume c_glusterfs
[2016-03-22 12:25:23.099807] I [MSGID: 106487] [glusterd-handler.c:1411:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2016-03-22 12:25:23.102392] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:23.102461] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:23.102510] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=9 from gen=46 to slot->gen=47, fd=18, slot->fd=18
[2016-03-22 12:25:23.152537] I [MSGID: 106487] [glusterd-handler.c:1411:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2016-03-22 12:25:23.154802] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:23.154862] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:23.154913] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=9 from gen=49 to slot->gen=50, fd=18, slot->fd=18
[2016-03-22 12:25:23.204648] I [MSGID: 106482] [glusterd-brick-ops.c:442:__glusterd_handle_add_brick] 0-management: Received add brick req
[2016-03-22 12:25:23.204748] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:23.204779] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:23.204817] I [MSGID: 106062] [glusterd-brick-ops.c:494:__glusterd_handle_add_brick] 0-management: replica-count is 2
[2016-03-22 12:25:23.204850] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:23.204883] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:23.204915] I [MSGID: 106447] [glusterd-brick-ops.c:240:gd_addbr_validate_replica_count] 0-management: Changing the type of volume c_glusterfs from 'distribute' to 'replica'
[2016-03-22 12:25:23.205023] D [MSGID: 0] [glusterd-locks.c:562:glusterd_mgmt_v3_lock] 0-management: Trying to acquire lock of vol c_glusterfs for 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2 as c_glusterfs_vol
[2016-03-22 12:25:23.205694] D [MSGID: 0] [glusterd-locks.c:618:glusterd_mgmt_v3_lock] 0-management: Lock for vol c_glusterfs successfully held by 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
[2016-03-22 12:25:23.205734] D [MSGID: 0] [glusterd-locks.c:512:glusterd_multiple_mgmt_v3_lock] 0-management: Returning 0
[2016-03-22 12:25:23.215903] D [MSGID: 0] [glusterd-mgmt.c:554:glusterd_mgmt_v3_initiate_lockdown] 0-management: Sent lock op req for Add brick to 1 peers. Returning 0
[2016-03-22 12:25:23.215989] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:23.216018] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:23.216086] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:23.216115] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:23.216154] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:23.216196] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:23.216289] D [MSGID: 0] [glusterd-utils.c:884:glusterd_brickinfo_new] 0-management: Returning 0
[2016-03-22 12:25:23.216330] D [MSGID: 0] [glusterd-utils.c:1041:glusterd_brickinfo_new_from_brick] 0-management: Returning 0
[2016-03-22 12:25:23.216414] D [MSGID: 0] [glusterd-utils.c:512:glusterd_volinfo_new] 0-management: Returning 0
[2016-03-22 12:25:23.216455] D [MSGID: 0] [glusterd-utils.c:815:glusterd_volume_brickinfos_delete] 0-management: Returning 0
[2016-03-22 12:25:23.216487] D [MSGID: 0] [store.c:460:gf_store_handle_destroy] 0-: Returning 0
[2016-03-22 12:25:23.216518] D [MSGID: 0] [glusterd-utils.c:859:glusterd_volinfo_delete] 0-management: Returning 0
[2016-03-22 12:25:23.216551] D [MSGID: 0] [glusterd-utils.c:884:glusterd_brickinfo_new] 0-management: Returning 0
[2016-03-22 12:25:23.216587] D [MSGID: 0] [glusterd-utils.c:1041:glusterd_brickinfo_new_from_brick] 0-management: Returning 0
[2016-03-22 12:25:23.216621] D [MSGID: 0] [glusterd-utils.c:512:glusterd_volinfo_new] 0-management: Returning 0
[2016-03-22 12:25:23.216654] D [MSGID: 0] [glusterd-utils.c:815:glusterd_volume_brickinfos_delete] 0-management: Returning 0
[2016-03-22 12:25:23.216683] D [MSGID: 0] [store.c:460:gf_store_handle_destroy] 0-: Returning 0
[2016-03-22 12:25:23.216712] D [MSGID: 0] [glusterd-utils.c:859:glusterd_volinfo_delete] 0-management: Returning 0
[2016-03-22 12:25:23.216744] D [MSGID: 0] [glusterd-utils.c:884:glusterd_brickinfo_new] 0-management: Returning 0
[2016-03-22 12:25:23.216777] D [MSGID: 0] [glusterd-utils.c:1041:glusterd_brickinfo_new_from_brick] 0-management: Returning 0
[2016-03-22 12:25:23.216808] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.0.48 found.. state: 3
[2016-03-22 12:25:23.216836] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:23.216863] D [MSGID: 0] [glusterd-utils.c:925:glusterd_resolve_brick] 0-management: Returning 0
[2016-03-22 12:25:23.216891] D [MSGID: 0] [glusterd-peer-utils.c:200:glusterd_peerinfo_find_by_uuid] 0-management: Friend found... state: Peer in Cluster
[2016-03-22 12:25:23.216919] D [MSGID: 0] [glusterd-utils.c:5656:glusterd_new_brick_validate] 0-management: returning 0 
[2016-03-22 12:25:23.216955] D [MSGID: 0] [glusterd-brick-ops.c:1741:glusterd_op_stage_add_brick] 0-management: Returning 0
[2016-03-22 12:25:23.216983] D [MSGID: 0] [glusterd-mgmt.c:198:gd_mgmt_v3_pre_validate_fn] 0-management: OP = 8. Returning 0
[2016-03-22 12:25:23.246278] D [MSGID: 0] [glusterd-mgmt.c:893:glusterd_mgmt_v3_pre_validate] 0-management: Sent pre valaidation req for Add brick to 1 peers. Returning 0
[2016-03-22 12:25:23.246364] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:23.246397] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:23.246454] I [MSGID: 106062] [glusterd-brick-ops.c:1268:glusterd_op_perform_add_bricks] 0-management: replica-count is set 2
[2016-03-22 12:25:23.246490] I [MSGID: 106062] [glusterd-brick-ops.c:1273:glusterd_op_perform_add_bricks] 0-management: type is set 2, need to change it
[2016-03-22 12:25:23.246565] D [MSGID: 0] [glusterd-utils.c:884:glusterd_brickinfo_new] 0-management: Returning 0
[2016-03-22 12:25:23.246603] D [MSGID: 0] [glusterd-utils.c:1041:glusterd_brickinfo_new_from_brick] 0-management: Returning 0
[2016-03-22 12:25:23.246647] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.0.48 found.. state: 3
[2016-03-22 12:25:23.246680] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:23.246711] D [MSGID: 0] [glusterd-utils.c:925:glusterd_resolve_brick] 0-management: Returning 0
[2016-03-22 12:25:23.246744] D [MSGID: 0] [glusterd-brick-ops.c:113:add_brick_at_right_order] 0-management: brick:/opt/lvmdir/c2/brick index=1, count=0
[2016-03-22 12:25:23.246938] D [MSGID: 0] [glusterd-volgen.c:5109:generate_brick_volfiles] 0-management: Found a brick - 10.32.1.144:/opt/lvmdir/c2/brick
[2016-03-22 12:25:23.247904] D [MSGID: 0] [glusterd-volgen.c:1268:server_check_marker_off] 0-glusterd: Returning 0
[2016-03-22 12:25:23.249144] D [MSGID: 0] [glusterd-volgen.c:5109:generate_brick_volfiles] 0-management: Found a brick - 10.32.0.48:/opt/lvmdir/c2/brick
[2016-03-22 12:25:23.249989] D [MSGID: 0] [glusterd-volgen.c:1268:server_check_marker_off] 0-glusterd: Returning 0
[2016-03-22 12:25:23.251099] D [MSGID: 0] [glusterd-volgen.c:5120:generate_brick_volfiles] 0-management: Returning 0
[2016-03-22 12:25:23.251178] D [MSGID: 0] [glusterd-utils.c:884:glusterd_brickinfo_new] 0-management: Returning 0
[2016-03-22 12:25:23.251219] D [MSGID: 0] [glusterd-utils.c:1041:glusterd_brickinfo_new_from_brick] 0-management: Returning 0
[2016-03-22 12:25:23.251256] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.0.48 found.. state: 3
[2016-03-22 12:25:23.251288] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:23.251322] D [MSGID: 0] [glusterd-utils.c:1326:glusterd_volume_brickinfo_get] 0-management: Found brick 10.32.0.48:/opt/lvmdir/c2/brick in volume c_glusterfs
[2016-03-22 12:25:23.251352] D [MSGID: 0] [glusterd-utils.c:1335:glusterd_volume_brickinfo_get] 0-management: Returning 0
[2016-03-22 12:25:23.251382] D [MSGID: 0] [glusterd-utils.c:1359:glusterd_volume_brickinfo_get_by_brick] 0-glusterd: Returning 0
[2016-03-22 12:25:23.251429] D [MSGID: 0] [glusterd-utils.c:4539:glusterd_brick_start] 0-management: returning 0 
[2016-03-22 12:25:23.251462] D [MSGID: 0] [glusterd-brick-ops.c:1436:glusterd_op_perform_add_bricks] 0-glusterd: Returning 0
[2016-03-22 12:25:23.251520] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already stopped
[2016-03-22 12:25:23.252151] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8b729f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff8b6d3990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff8b6cf430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff88141ab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff881428b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:25:23.252229] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:23.252263] D [MSGID: 0] [glusterd-nfs-svc.c:135:glusterd_nfssvc_stop] 0-management: Returning 0
[2016-03-22 12:25:23.252483] D [MSGID: 0] [glusterd-volgen.c:4809:build_nfs_graph] 0-glusterd: Returning 0
[2016-03-22 12:25:23.253071] D [MSGID: 0] [glusterd-nfs-svc.c:106:glusterd_nfssvc_manager] 0-management: Returning 0
[2016-03-22 12:25:23.253508] D [MSGID: 0] [glusterd-volgen.c:881:_xl_link_children] 0-management: glustershd:c_glusterfs-replicate-0
[2016-03-22 12:25:23.254961] D [MSGID: 0] [glusterd-shd-svc.c:78:glusterd_shdsvc_create_volfile] 0-management: Returning 0
[2016-03-22 12:25:23.255088] D [MSGID: 0] [glusterd-proc-mgmt.c:87:glusterd_proc_stop] 0-management: Stopping glustershd daemon running in pid: 2694
[2016-03-22 12:25:23.261701] D [socket.c:558:__socket_rwv] 0-glustershd: EOF on socket
[2016-03-22 12:25:23.261777] W [socket.c:588:__socket_rwv] 0-glustershd: readv on /var/run/gluster/8426fb5fe867c36346144bff93f74572.socket failed (No data available)
[2016-03-22 12:25:23.261773] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:23.261803] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:23.261821] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:23.261896] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=3 from gen=10 to slot->gen=11, fd=7, slot->fd=7
[2016-03-22 12:25:23.262275] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8b729f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff8b6d3990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup-0x19b20)[0x3fff8b6cfaf0] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_notify-0x18e68)[0x3fff8b6d0808] (--> /usr/lib64/libgfrpc.so.0(rpc_transport_notify-0x1d8bc)[0x3fff8b6cb85c] ))))) 0-: /var/run/gluster/8426fb5fe867c36346144bff93f74572.socket: ping timer event already removed
[2016-03-22 12:25:24.255889] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8b729f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff8b6d3990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff8b6cf430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff88141ab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff881428b8] ))))) 0-: /var/run/gluster/8426fb5fe867c36346144bff93f74572.socket: ping timer event already removed
[2016-03-22 12:25:24.255982] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:24.256078] D [MSGID: 0] [glusterd-svc-mgmt.c:195:glusterd_svc_start] 0-management: Starting glustershd service
[2016-03-22 12:25:24.266854] D [MSGID: 0] [glusterd-svc-mgmt.c:208:glusterd_svc_start] 0-management: Returning 0
[2016-03-22 12:25:24.267005] D [MSGID: 0] [glusterd-shd-svc.c:177:glusterd_shdsvc_start] 0-management: Returning 0
[2016-03-22 12:25:24.267196] W [socket.c:3009:socket_connect] 0-glustershd: Ignore failed connection attempt on /var/run/gluster/8426fb5fe867c36346144bff93f74572.socket, (No such file or directory) 
[2016-03-22 12:25:24.267311] D [MSGID: 0] [glusterd-shd-svc.c:139:glusterd_shdsvc_manager] 0-management: Returning 0
[2016-03-22 12:25:24.267576] D [MSGID: 0] [glusterd-quotad-svc.c:111:glusterd_quotadsvc_manager] 0-management: Returning 0
[2016-03-22 12:25:24.267710] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: bitd already stopped
[2016-03-22 12:25:24.268420] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8b729f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff8b6d3990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff8b6cf430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff88141ab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff881428b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:25:24.268535] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:24.268572] D [MSGID: 0] [glusterd-bitd-svc.c:109:glusterd_bitdsvc_manager] 0-management: Returning 0
[2016-03-22 12:25:24.268725] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: scrub already stopped
[2016-03-22 12:25:24.269205] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8b729f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff8b6d3990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff8b6cf430] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_conn_disconnect-0x6210c)[0x3fff88141ab4] (--> /usr/lib64/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_svc_stop-0x613f8)[0x3fff881428b8] ))))) 0-: : ping timer event already removed
[2016-03-22 12:25:24.269300] D [MSGID: 0] [glusterd-svc-mgmt.c:227:glusterd_svc_stop] 0-management: Returning 0
[2016-03-22 12:25:24.269333] D [MSGID: 0] [glusterd-scrub-svc.c:104:glusterd_scrubsvc_manager] 0-management: Returning 0
[2016-03-22 12:25:24.269373] D [MSGID: 0] [glusterd-mgmt.c:294:gd_mgmt_v3_commit_fn] 0-management: OP = 8. Returning 0
[2016-03-22 12:25:24.269434] D [MSGID: 0] [glusterd-utils.c:7507:glusterd_aggr_brick_mount_dirs] 0-management: No brick_count present
[2016-03-22 12:25:24.269955] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=5 from gen=4 to slot->gen=5, fd=12, slot->fd=12
[2016-03-22 12:25:24.269988] D [MSGID: 0] [glusterd-svc-mgmt.c:315:glusterd_svc_common_rpc_notify] 0-management: glustershd has connected with glusterd.
[2016-03-22 12:25:24.270125] W [socket.c:588:__socket_rwv] 0-glustershd: readv on /var/run/gluster/8426fb5fe867c36346144bff93f74572.socket failed (Invalid argument)
[2016-03-22 12:25:24.270160] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:24.270746] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8b729f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff8b6d3990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup-0x19b20)[0x3fff8b6cfaf0] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_notify-0x18e68)[0x3fff8b6d0808] (--> /usr/lib64/libgfrpc.so.0(rpc_transport_notify-0x1d8bc)[0x3fff8b6cb85c] ))))) 0-: /var/run/gluster/8426fb5fe867c36346144bff93f74572.socket: ping timer event already removed
[2016-03-22 12:25:24.271190] I [MSGID: 106006] [glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: glustershd has disconnected from glusterd.
[2016-03-22 12:25:24.271233] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=3 from gen=13 to slot->gen=14, fd=7, slot->fd=7
[2016-03-22 12:25:24.299163] D [MSGID: 0] [glusterd-handshake.c:409:glusterd_get_args_from_dict] 0-management: No brick name present
[2016-03-22 12:25:24.299211] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning -1
[2016-03-22 12:25:24.299395] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 127.0.0.1 
[2016-03-22 12:25:24.310882] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:24.310913] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:24.310955] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=5 from gen=7 to slot->gen=8, fd=12, slot->fd=12
[2016-03-22 12:25:24.383088] D [MSGID: 0] [glusterd-utils.c:7507:glusterd_aggr_brick_mount_dirs] 0-management: No brick_count present
[2016-03-22 12:25:24.383262] D [MSGID: 0] [glusterd-mgmt.c:1427:glusterd_mgmt_v3_commit] 0-management: Sent commit req for Add brick to 1 peers. Returning 0
[2016-03-22 12:25:24.383358] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:24.383444] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:24.383645] D [MSGID: 0] [glusterd-volgen.c:5109:generate_brick_volfiles] 0-management: Found a brick - 10.32.1.144:/opt/lvmdir/c2/brick
[2016-03-22 12:25:24.384550] D [MSGID: 0] [glusterd-volgen.c:1268:server_check_marker_off] 0-glusterd: Returning 0
[2016-03-22 12:25:24.385721] D [MSGID: 0] [glusterd-volgen.c:5109:generate_brick_volfiles] 0-management: Found a brick - 10.32.0.48:/opt/lvmdir/c2/brick
[2016-03-22 12:25:24.386468] D [MSGID: 0] [glusterd-volgen.c:1268:server_check_marker_off] 0-glusterd: Returning 0
[2016-03-22 12:25:24.387367] D [MSGID: 0] [glusterd-volgen.c:5120:generate_brick_volfiles] 0-management: Returning 0
[2016-03-22 12:25:24.387859] D [MSGID: 0] [glusterd-peer-utils.c:133:glusterd_peerinfo_find_by_hostname] 0-management: Unable to find friend: 10.32.1.144
[2016-03-22 12:25:24.387966] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.1.144 
[2016-03-22 12:25:24.388778] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.1.144 is local address at interface efd
[2016-03-22 12:25:24.388827] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:24.388869] D [MSGID: 0] [glusterd-utils.c:1326:glusterd_volume_brickinfo_get] 0-management: Found brick 10.32.1.144:/opt/lvmdir/c2/brick in volume c_glusterfs
[2016-03-22 12:25:24.388900] D [MSGID: 0] [glusterd-utils.c:1335:glusterd_volume_brickinfo_get] 0-management: Returning 0
[2016-03-22 12:25:24.388936] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.0.48 found.. state: 3
[2016-03-22 12:25:24.388967] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:24.388998] D [MSGID: 0] [glusterd-utils.c:1326:glusterd_volume_brickinfo_get] 0-management: Found brick 10.32.0.48:/opt/lvmdir/c2/brick in volume c_glusterfs
[2016-03-22 12:25:24.389028] D [MSGID: 0] [glusterd-utils.c:1335:glusterd_volume_brickinfo_get] 0-management: Returning 0
[2016-03-22 12:25:24.390980] D [MSGID: 0] [glusterd-peer-utils.c:133:glusterd_peerinfo_find_by_hostname] 0-management: Unable to find friend: 10.32.1.144
[2016-03-22 12:25:24.391087] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.1.144 
[2016-03-22 12:25:24.391667] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.1.144 is local address at interface efd
[2016-03-22 12:25:24.391704] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:24.391736] D [MSGID: 0] [glusterd-utils.c:1326:glusterd_volume_brickinfo_get] 0-management: Found brick 10.32.1.144:/opt/lvmdir/c2/brick in volume c_glusterfs
[2016-03-22 12:25:24.391767] D [MSGID: 0] [glusterd-utils.c:1335:glusterd_volume_brickinfo_get] 0-management: Returning 0
[2016-03-22 12:25:24.391818] D [MSGID: 0] [glusterd-peer-utils.c:646:gd_peerinfo_find_from_hostname] 0-management: Friend 10.32.0.48 found.. state: 3
[2016-03-22 12:25:24.391850] D [MSGID: 0] [glusterd-peer-utils.c:168:glusterd_hostname_to_uuid] 0-management: returning 0
[2016-03-22 12:25:24.391881] D [MSGID: 0] [glusterd-utils.c:1326:glusterd_volume_brickinfo_get] 0-management: Found brick 10.32.0.48:/opt/lvmdir/c2/brick in volume c_glusterfs
[2016-03-22 12:25:24.391911] D [MSGID: 0] [glusterd-utils.c:1335:glusterd_volume_brickinfo_get] 0-management: Returning 0
[2016-03-22 12:25:24.393953] D [MSGID: 0] [glusterd-store.c:1058:glusterd_store_create_volume_dir] 0-management: Returning with 0
[2016-03-22 12:25:24.394166] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:24.395466] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 19 times between [2016-03-22 12:25:24.394166] and [2016-03-22 12:25:24.395462]
[2016-03-22 12:25:24.395465] D [MSGID: 0] [glusterd-store.c:440:glusterd_store_snapd_write] 0-management: Returning 0
[2016-03-22 12:25:24.413146] D [MSGID: 0] [glusterd-store.c:503:glusterd_store_perform_snapd_store] 0-management: Returning 0
[2016-03-22 12:25:24.413216] D [MSGID: 0] [glusterd-store.c:568:glusterd_store_snapd_info] 0-management: Returning with 0
[2016-03-22 12:25:24.413300] D [MSGID: 0] [glusterd-store.c:753:_storeopts] 0-management: Storing in volinfo:key= nfs.disable, val=on
[2016-03-22 12:25:24.413440] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:24.413481] D [MSGID: 0] [glusterd-store.c:753:_storeopts] 0-management: Storing in volinfo:key= network.ping-timeout, val=4
[2016-03-22 12:25:24.413571] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:24.413611] D [MSGID: 0] [glusterd-store.c:753:_storeopts] 0-management: Storing in volinfo:key= performance.readdir-ahead, val=on
[2016-03-22 12:25:24.413695] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:24.413728] D [MSGID: 0] [glusterd-store.c:1104:glusterd_store_volinfo_write] 0-management: Returning 0
[2016-03-22 12:25:24.413830] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:24.414505] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 8 times between [2016-03-22 12:25:24.413830] and [2016-03-22 12:25:24.414500]
[2016-03-22 12:25:24.414504] D [MSGID: 0] [glusterd-store.c:414:glusterd_store_brickinfo_write] 0-management: Returning 0
[2016-03-22 12:25:24.414591] D [MSGID: 0] [glusterd-store.c:464:glusterd_store_perform_brick_store] 0-management: Returning 0
[2016-03-22 12:25:24.414623] D [MSGID: 0] [glusterd-store.c:533:glusterd_store_brickinfo] 0-management: Returning with 0
[2016-03-22 12:25:24.414722] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:24.421818] D [MSGID: 0] [store.c:421:gf_store_handle_new] 0-: Returning 0
[2016-03-22 12:25:24.422099] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:24.422587] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 7 times between [2016-03-22 12:25:24.422099] and [2016-03-22 12:25:24.422582]
[2016-03-22 12:25:24.422586] D [MSGID: 0] [glusterd-store.c:414:glusterd_store_brickinfo_write] 0-management: Returning 0
[2016-03-22 12:25:24.422671] D [MSGID: 0] [glusterd-store.c:464:glusterd_store_perform_brick_store] 0-management: Returning 0
[2016-03-22 12:25:24.422702] D [MSGID: 0] [glusterd-store.c:533:glusterd_store_brickinfo] 0-management: Returning with 0
[2016-03-22 12:25:24.422731] D [MSGID: 0] [glusterd-store.c:1325:glusterd_store_brickinfos] 0-management: Returning 0
[2016-03-22 12:25:24.422762] D [MSGID: 0] [glusterd-store.c:1439:glusterd_store_perform_volume_store] 0-management: Returning 0
[2016-03-22 12:25:24.464704] D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0
[2016-03-22 12:25:24.464947] D [logging.c:1952:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "D [MSGID: 0] [store.c:375:gf_store_save_value] 0-management: returning: 0" repeated 3 times between [2016-03-22 12:25:24.464704] and [2016-03-22 12:25:24.464942]
[2016-03-22 12:25:24.464946] D [MSGID: 0] [glusterd-store.c:1383:glusterd_store_node_state_write] 0-management: Returning 0
[2016-03-22 12:25:24.480129] D [MSGID: 0] [glusterd-store.c:1411:glusterd_store_perform_node_state_store] 0-management: Returning 0
[2016-03-22 12:25:24.480812] D [MSGID: 0] [glusterd-utils.c:2052:glusterd_volume_compute_cksum] 0-management: Returning with 0
[2016-03-22 12:25:24.480851] D [MSGID: 0] [glusterd-store.c:1645:glusterd_store_volinfo] 0-management: Returning 0
[2016-03-22 12:25:24.481146] D [MSGID: 0] [glusterd-handshake.c:409:glusterd_get_args_from_dict] 0-management: No brick name present
[2016-03-22 12:25:24.481194] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:24.481221] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:24.481418] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.1.144 
[2016-03-22 12:25:24.482230] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.1.144 is local address at interface efd
[2016-03-22 12:25:24.482303] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:24.482334] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:24.482660] D [MSGID: 0] [glusterd-handshake.c:409:glusterd_get_args_from_dict] 0-management: No brick name present
[2016-03-22 12:25:24.482707] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning -1
[2016-03-22 12:25:24.482823] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 127.0.0.1 
[2016-03-22 12:25:24.483036] D [MSGID: 0] [glusterd-handshake.c:414:glusterd_get_args_from_dict] 0-management: brick_name = /opt/lvmdir/c2/brick
[2016-03-22 12:25:24.483070] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning -1
[2016-03-22 12:25:24.483207] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 10.32.1.144 
[2016-03-22 12:25:24.484328] D [MSGID: 0] [common-utils.c:3027:gf_interface_search] 0-management: 10.32.1.144 is local address at interface efd
[2016-03-22 12:25:24.484417] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning -1
[2016-03-22 12:25:24.484452] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:24.484480] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:24.484576] D [MSGID: 0] [glusterd-handshake.c:841:__server_getspec] 0-management: Look for missing snap creates for /opt/lvmdir/c2/brick
[2016-03-22 12:25:24.485263] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:24.485294] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:24.485378] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=5 from gen=10 to slot->gen=11, fd=12, slot->fd=12
[2016-03-22 12:25:24.497136] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:24.497167] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:24.497212] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=5 from gen=13 to slot->gen=14, fd=12, slot->fd=12
[2016-03-22 12:25:24.597215] D [MSGID: 0] [glusterd-mgmt.c:1647:glusterd_mgmt_v3_post_validate] 0-management: Sent post valaidation req for Add brick to 1 peers. Returning 0
[2016-03-22 12:25:24.601235] D [MSGID: 0] [glusterd-mgmt.c:1828:glusterd_mgmt_v3_release_peer_locks] 0-management: Sent unlock op req for Add brick to 1 peers. Returning 0
[2016-03-22 12:25:24.601308] D [MSGID: 0] [glusterd-locks.c:669:glusterd_mgmt_v3_unlock] 0-management: Trying to release lock of vol c_glusterfs for 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2 as c_glusterfs_vol
[2016-03-22 12:25:24.601354] D [MSGID: 0] [glusterd-locks.c:714:glusterd_mgmt_v3_unlock] 0-management: Lock for vol c_glusterfs successfully released
[2016-03-22 12:25:24.601390] D [MSGID: 0] [glusterd-locks.c:451:glusterd_multiple_mgmt_v3_unlock] 0-management: Returning 0
[2016-03-22 12:25:24.601561] D [MSGID: 0] [glusterd-rpc-ops.c:205:glusterd_op_send_cli_response] 0-management: Returning 0
[2016-03-22 12:25:24.603589] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:24.603636] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:24.603685] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=9 from gen=52 to slot->gen=53, fd=18, slot->fd=18
[2016-03-22 12:25:24.661699] I [MSGID: 106487] [glusterd-handler.c:1411:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2016-03-22 12:25:24.664343] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:24.664380] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:24.664433] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=5 from gen=16 to slot->gen=17, fd=12, slot->fd=12
[2016-03-22 12:25:24.688154] I [MSGID: 106488] [glusterd-handler.c:1472:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req
[2016-03-22 12:25:24.688250] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:24.688289] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:24.690900] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:24.690942] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:24.690992] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=5 from gen=19 to slot->gen=20, fd=12, slot->fd=12
[2016-03-22 12:25:24.745324] D [MSGID: 0] [glusterd-handshake.c:409:glusterd_get_args_from_dict] 0-management: No brick name present
[2016-03-22 12:25:24.745380] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:24.745424] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:24.745586] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 127.0.0.1 
[2016-03-22 12:25:24.745631] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:24.745664] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:24.768833] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:24.768862] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:24.768903] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=12 from gen=1 to slot->gen=2, fd=21, slot->fd=21
[2016-03-22 12:25:24.848051] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:24.848096] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:24.848158] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=12 from gen=4 to slot->gen=5, fd=21, slot->fd=21
[2016-03-22 12:25:27.692390] D [MSGID: 0] [glusterd-svc-mgmt.c:315:glusterd_svc_common_rpc_notify] 0-management: glustershd has connected with glusterd.
[2016-03-22 12:25:28.249810] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:28.249864] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:28.249937] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=9 from gen=55 to slot->gen=56, fd=18, slot->fd=18
[2016-03-22 12:25:28.252294] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:28.252326] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:28.252365] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=5 from gen=22 to slot->gen=23, fd=12, slot->fd=12
[2016-03-22 12:25:28.307216] D [MSGID: 0] [glusterd-handshake.c:409:glusterd_get_args_from_dict] 0-management: No brick name present
[2016-03-22 12:25:28.307265] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:28.307293] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:28.307441] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 127.0.0.1 
[2016-03-22 12:25:28.307479] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:28.307509] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:28.326043] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:28.326071] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:28.326111] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=13 from gen=1 to slot->gen=2, fd=22, slot->fd=22
[2016-03-22 12:25:32.241181] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:32.241253] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:32.241353] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=9 from gen=58 to slot->gen=59, fd=18, slot->fd=18
[2016-03-22 12:25:32.244059] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:32.244123] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:32.244170] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=5 from gen=25 to slot->gen=26, fd=12, slot->fd=12
[2016-03-22 12:25:32.455289] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:32.455344] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:32.455394] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=5 from gen=28 to slot->gen=29, fd=12, slot->fd=12
[2016-03-22 12:25:37.762196] I [MSGID: 106487] [glusterd-handler.c:1411:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2016-03-22 12:25:37.765123] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:37.765172] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:37.765220] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=5 from gen=31 to slot->gen=32, fd=12, slot->fd=12
[2016-03-22 12:25:37.790360] I [MSGID: 106488] [glusterd-handler.c:1472:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req
[2016-03-22 12:25:37.790476] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:37.790505] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:37.793712] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:37.793766] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:37.793837] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=5 from gen=34 to slot->gen=35, fd=12, slot->fd=12
[2016-03-22 12:25:38.537127] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:38.537174] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:38.537221] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=5 from gen=37 to slot->gen=38, fd=12, slot->fd=12
[2016-03-22 12:25:42.496714] D [MSGID: 0] [glusterd-handshake.c:409:glusterd_get_args_from_dict] 0-management: No brick name present
[2016-03-22 12:25:42.496771] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:42.496815] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:42.497001] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 127.0.0.1 
[2016-03-22 12:25:42.497040] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:42.497101] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:42.514431] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:42.514465] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:42.514508] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=13 from gen=4 to slot->gen=5, fd=22, slot->fd=22
[2016-03-22 12:25:44.576164] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:44.576227] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:44.576332] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=9 from gen=61 to slot->gen=62, fd=18, slot->fd=18
[2016-03-22 12:25:44.578500] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:44.578538] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:44.578579] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=5 from gen=40 to slot->gen=41, fd=12, slot->fd=12
[2016-03-22 12:25:50.442388] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:50.442452] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:50.442529] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=7 from gen=7 to slot->gen=8, fd=16, slot->fd=16
[2016-03-22 12:25:50.694362] D [rpc-clnt-ping.c:300:rpc_clnt_start_ping] 0-management: returning as transport is already disconnected OR there are no frames (0 || 0)
[2016-03-22 12:25:50.694877] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8b729f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff8b6d3990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_disconnect-0x1a198)[0x3fff8b6cf430] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_unref-0x1a098)[0x3fff8b6cf548] (--> /usr/lib64/libgfrpc.so.0(+0x1a224)[0x3fff8b6d4224] ))))) 0-: 10.32.0.48:24007: ping timer event already removed
[2016-03-22 12:25:50.695014] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:50.695353] D [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8b729f58] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x15f98)[0x3fff8b6d3990] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup-0x19b20)[0x3fff8b6cfaf0] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_notify-0x18e68)[0x3fff8b6d0808] (--> /usr/lib64/libgfrpc.so.0(rpc_transport_notify-0x1d8bc)[0x3fff8b6cb85c] ))))) 0-: 10.32.0.48:24007: ping timer event already removed
[2016-03-22 12:25:50.695416] D [MSGID: 0] [glusterd-peer-utils.c:1005:glusterd_peerinfo_find_by_generation] 0-management: Friend with generation: 3, not found
[2016-03-22 12:25:50.695469] D [MSGID: 106129] [glusterd-handler.c:5032:__glusterd_peer_rpc_notify] 0-management: Could not find peer 10.32.0.48(356dabed-4314-48be-9eef-d2711d01270d) [No such file or directory]
[2016-03-22 12:25:50.695527] I [MSGID: 101053] [mem-pool.c:616:mem_pool_destroy] 0-management: size=588 max=3 total=23
[2016-03-22 12:25:50.695571] I [MSGID: 101053] [mem-pool.c:616:mem_pool_destroy] 0-management: size=124 max=3 total=23
[2016-03-22 12:25:50.695686] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=2 from gen=13 to slot->gen=14, fd=13, slot->fd=13
[2016-03-22 12:25:51.695073] D [rpc-clnt-ping.c:300:rpc_clnt_start_ping] 0-management: returning as transport is already disconnected OR there are no frames (0 || 0)
[2016-03-22 12:25:57.389572] I [MSGID: 106487] [glusterd-handler.c:1411:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2016-03-22 12:25:57.392222] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:57.392257] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:57.392298] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=2 from gen=16 to slot->gen=17, fd=12, slot->fd=12
[2016-03-22 12:25:57.416585] I [MSGID: 106488] [glusterd-handler.c:1472:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req
[2016-03-22 12:25:57.416674] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:25:57.416702] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:25:57.419023] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:57.419057] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:57.419098] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=2 from gen=19 to slot->gen=20, fd=12, slot->fd=12
[2016-03-22 12:25:58.113234] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:25:58.113277] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:25:58.113318] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=2 from gen=22 to slot->gen=23, fd=12, slot->fd=12
[2016-03-22 12:26:01.487395] D [MSGID: 0] [glusterd-handshake.c:409:glusterd_get_args_from_dict] 0-management: No brick name present
[2016-03-22 12:26:01.487467] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:26:01.487528] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:26:01.487658] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 127.0.0.1 
[2016-03-22 12:26:01.487703] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:26:01.487735] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:26:01.505308] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:26:01.505342] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:26:01.505382] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=7 from gen=10 to slot->gen=11, fd=16, slot->fd=16
[2016-03-22 12:26:03.566175] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:26:03.566231] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:26:03.566305] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=5 from gen=43 to slot->gen=44, fd=13, slot->fd=13
[2016-03-22 12:26:03.568572] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:26:03.568609] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:26:03.568672] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=2 from gen=25 to slot->gen=26, fd=12, slot->fd=12
[2016-03-22 12:26:03.897189] I [MSGID: 106487] [glusterd-handler.c:1411:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2016-03-22 12:26:03.899886] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:26:03.899923] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:26:03.899964] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=2 from gen=28 to slot->gen=29, fd=12, slot->fd=12
[2016-03-22 12:26:03.924940] I [MSGID: 106488] [glusterd-handler.c:1472:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req
[2016-03-22 12:26:03.925053] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:26:03.925082] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:26:03.928263] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:26:03.928301] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:26:03.928342] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=2 from gen=31 to slot->gen=32, fd=12, slot->fd=12
[2016-03-22 12:26:04.619314] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:26:04.619361] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:26:04.619422] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=2 from gen=34 to slot->gen=35, fd=12, slot->fd=12
[2016-03-22 12:26:06.921969] D [MSGID: 0] [glusterd-handshake.c:409:glusterd_get_args_from_dict] 0-management: No brick name present
[2016-03-22 12:26:06.922024] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:26:06.922062] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:26:06.922211] D [MSGID: 0] [common-utils.c:3133:gf_is_local_addr] 0-management: 127.0.0.1 
[2016-03-22 12:26:06.922249] D [MSGID: 0] [glusterd-utils.c:1424:glusterd_volinfo_find] 0-management: Volume c_glusterfs found
[2016-03-22 12:26:06.922277] D [MSGID: 0] [glusterd-utils.c:1431:glusterd_volinfo_find] 0-management: Returning 0
[2016-03-22 12:26:06.939433] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:26:06.939469] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:26:06.939511] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=7 from gen=13 to slot->gen=14, fd=16, slot->fd=16
[2016-03-22 12:26:09.242716] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:26:09.242776] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:26:09.242853] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=5 from gen=46 to slot->gen=47, fd=13, slot->fd=13
[2016-03-22 12:26:09.244915] D [socket.c:558:__socket_rwv] 0-socket.management: EOF on socket
[2016-03-22 12:26:09.244948] D [socket.c:2355:socket_event_handler] 0-transport: disconnecting now
[2016-03-22 12:26:09.245286] D [MSGID: 0] [event-epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=2 from gen=37 to slot->gen=38, fd=12, slot->fd=12
[2016-03-22 12:26:10.696664] D [logging.c:1828:gf_log_flush_timeout_cbk] 0-logging-infra: Log timer timed out. About to flush outstanding messages if present
[2016-03-22 12:26:10.696760] D [logging.c:1790:__gf_log_inject_timer_event] 0-logging-infra: Starting timer now. Timeout = 120, current buf size = 5
[2016-03-22 12:28:10.706666] D [logging.c:1828:gf_log_flush_timeout_cbk] 0-logging-infra: Log timer timed out. About to flush outstanding messages if present
[2016-03-22 12:28:10.706727] D [logging.c:1790:__gf_log_inject_timer_event] 0-logging-infra: Starting timer now. Timeout = 120, current buf size = 5
[2016-03-22 12:30:10.716362] D [logging.c:1828:gf_log_flush_timeout_cbk] 0-logging-infra: Log timer timed out. About to flush outstanding messages if present
[2016-03-22 12:30:10.716433] D [logging.c:1790:__gf_log_inject_timer_event] 0-logging-infra: Starting timer now. Timeout = 120, current buf size = 5
[2016-03-22 12:32:10.725984] D [logging.c:1828:gf_log_flush_timeout_cbk] 0-logging-infra: Log timer timed out. About to flush outstanding messages if present
[2016-03-22 12:32:10.726039] D [logging.c:1790:__gf_log_inject_timer_event] 0-logging-infra: Starting timer now. Timeout = 120, current buf size = 5
$ 

EVOA_8300-10_6B> lhsh 002500 cat /var/log/glusterfs/glfsheal-c_glusterfs.log

160322-13:32:12 10.67.29.90 16.0c CPP_MOM-CPP-LSV184-gen1_gen1_COMPLETE stopfile=/tmp/13910
$ lhsh 002500 cat /var/log/glusterfs/glfsheal-c_glusterfs.log
[2016-03-22 12:20:28.608592] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:20:28.635700] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2016-03-22 12:20:28.636860] I [MSGID: 104045] [glfs-master.c:95:notify] 0-gfapi: New graph 30303235-3030-2d33-3932-372d32303136 (0) coming up
[2016-03-22 12:20:28.636924] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-416: parent translators are ready, attempting connect on transport
[2016-03-22 12:20:28.637951] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-417: parent translators are ready, attempting connect on transport
[2016-03-22 12:20:28.639050] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-417: changing port to 49719 (from 0)
[2016-03-22 12:20:28.639097] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-416: changing port to 49729 (from 0)
[2016-03-22 12:20:28.640589] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-416: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:20:28.640685] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-417: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:20:28.641187] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-417: Connected to c_glusterfs-client-417, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:20:28.641217] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-417: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:20:28.641321] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-416: Connected to c_glusterfs-client-416, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:20:28.641352] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-416: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:20:28.641473] I [MSGID: 108005] [afr-common.c:3841:afr_notify] 0-c_glusterfs-replicate-0: Subvolume 'c_glusterfs-client-416' came back up; going online.
[2016-03-22 12:20:28.641700] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-416: Server lk version = 1
[2016-03-22 12:20:28.672159] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-417: Server lk version = 1
[2016-03-22 12:20:28.674089] I [MSGID: 108031] [afr-common.c:1782:afr_local_discovery_cbk] 0-c_glusterfs-replicate-0: selecting local read_child c_glusterfs-client-417
[2016-03-22 12:20:28.674797] I [MSGID: 104041] [glfs-resolve.c:862:__glfs_active_subvol] 0-c_glusterfs: switched to graph 30303235-3030-2d33-3932-372d32303136 (0)
[2016-03-22 12:25:24.731785] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:24.764457] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2016-03-22 12:25:24.766942] I [MSGID: 104045] [glfs-master.c:95:notify] 0-gfapi: New graph 30303235-3030-2d39-3837-362d32303136 (0) coming up
[2016-03-22 12:25:24.767026] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-417: parent translators are ready, attempting connect on transport
[2016-03-22 12:25:24.768251] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-418: parent translators are ready, attempting connect on transport
[2016-03-22 12:25:24.768761] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-417: changing port to 49719 (from 0)
[2016-03-22 12:25:24.770263] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-417: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:25:24.770427] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-418: changing port to 49730 (from 0)
[2016-03-22 12:25:24.771204] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-417: Connected to c_glusterfs-client-417, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:25:24.771244] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-417: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:25:24.771627] I [MSGID: 108005] [afr-common.c:3841:afr_notify] 0-c_glusterfs-replicate-0: Subvolume 'c_glusterfs-client-417' came back up; going online.
[2016-03-22 12:25:24.771816] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-417: Server lk version = 1
[2016-03-22 12:25:24.772075] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-418: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:25:24.772848] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-418: Connected to c_glusterfs-client-418, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:25:24.772882] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-418: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:25:24.806371] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-418: Server lk version = 1
[2016-03-22 12:25:24.808735] I [MSGID: 108031] [afr-common.c:1782:afr_local_discovery_cbk] 0-c_glusterfs-replicate-0: selecting local read_child c_glusterfs-client-417
[2016-03-22 12:25:24.812363] I [MSGID: 104041] [glfs-resolve.c:862:__glfs_active_subvol] 0-c_glusterfs: switched to graph 30303235-3030-2d39-3837-362d32303136 (0)
[2016-03-22 12:25:28.293106] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:28.323065] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2016-03-22 12:25:28.324287] I [MSGID: 104045] [glfs-master.c:95:notify] 0-gfapi: New graph 30303235-3030-2d31-3035-33342d323031 (0) coming up
[2016-03-22 12:25:28.324358] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-417: parent translators are ready, attempting connect on transport
[2016-03-22 12:25:28.325507] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-418: parent translators are ready, attempting connect on transport
[2016-03-22 12:25:28.325966] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-417: changing port to 49719 (from 0)
[2016-03-22 12:25:28.327317] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-417: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:25:28.327871] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-417: Connected to c_glusterfs-client-417, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:25:28.327902] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-417: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:25:28.328078] I [MSGID: 108005] [afr-common.c:3841:afr_notify] 0-c_glusterfs-replicate-0: Subvolume 'c_glusterfs-client-417' came back up; going online.
[2016-03-22 12:25:28.328150] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-417: Server lk version = 1
[2016-03-22 12:25:28.329970] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-418: changing port to 49730 (from 0)
[2016-03-22 12:25:28.332380] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-418: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:25:28.333134] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-418: Connected to c_glusterfs-client-418, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:25:28.333164] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-418: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:25:28.367088] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-418: Server lk version = 1
[2016-03-22 12:25:28.368944] I [MSGID: 108031] [afr-common.c:1782:afr_local_discovery_cbk] 0-c_glusterfs-replicate-0: selecting local read_child c_glusterfs-client-417
[2016-03-22 12:25:28.370725] I [MSGID: 104041] [glfs-resolve.c:862:__glfs_active_subvol] 0-c_glusterfs: switched to graph 30303235-3030-2d31-3035-33342d323031 (0)
[2016-03-22 12:25:42.478440] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:42.511385] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2016-03-22 12:25:42.512617] I [MSGID: 104045] [glfs-master.c:95:notify] 0-gfapi: New graph 30303235-3030-2d31-3233-30352d323031 (0) coming up
[2016-03-22 12:25:42.512690] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-417: parent translators are ready, attempting connect on transport
[2016-03-22 12:25:42.513808] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-418: parent translators are ready, attempting connect on transport
[2016-03-22 12:25:42.514332] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-417: changing port to 49719 (from 0)
[2016-03-22 12:25:42.515996] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-417: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:25:42.516227] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-418: changing port to 49730 (from 0)
[2016-03-22 12:25:42.516524] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-417: Connected to c_glusterfs-client-417, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:25:42.516563] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-417: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:25:42.516726] I [MSGID: 108005] [afr-common.c:3841:afr_notify] 0-c_glusterfs-replicate-0: Subvolume 'c_glusterfs-client-417' came back up; going online.
[2016-03-22 12:25:42.516809] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-417: Server lk version = 1
[2016-03-22 12:25:42.517910] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-418: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:25:42.518619] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-418: Connected to c_glusterfs-client-418, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:25:42.518650] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-418: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:25:42.551048] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-418: Server lk version = 1
[2016-03-22 12:25:42.553321] I [MSGID: 108031] [afr-common.c:1782:afr_local_discovery_cbk] 0-c_glusterfs-replicate-0: selecting local read_child c_glusterfs-client-417
[2016-03-22 12:25:42.554266] I [MSGID: 104041] [glfs-resolve.c:862:__glfs_active_subvol] 0-c_glusterfs: switched to graph 30303235-3030-2d31-3233-30352d323031 (0)
[2016-03-22 12:26:01.473931] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:26:01.502238] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2016-03-22 12:26:01.503491] I [MSGID: 104045] [glfs-master.c:95:notify] 0-gfapi: New graph 30303235-3030-2d31-3733-30302d323031 (0) coming up
[2016-03-22 12:26:01.503554] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-417: parent translators are ready, attempting connect on transport
[2016-03-22 12:26:01.504737] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-418: parent translators are ready, attempting connect on transport
[2016-03-22 12:26:01.505235] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-417: changing port to 49719 (from 0)
[2016-03-22 12:26:01.506551] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-417: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:26:01.506646] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-418: changing port to 49730 (from 0)
[2016-03-22 12:26:01.507082] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-417: Connected to c_glusterfs-client-417, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:26:01.507114] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-417: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:26:01.507282] I [MSGID: 108005] [afr-common.c:3841:afr_notify] 0-c_glusterfs-replicate-0: Subvolume 'c_glusterfs-client-417' came back up; going online.
[2016-03-22 12:26:01.507369] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-417: Server lk version = 1
[2016-03-22 12:26:01.508177] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-418: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:26:01.508861] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-418: Connected to c_glusterfs-client-418, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:26:01.508891] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-418: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:26:01.541521] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-418: Server lk version = 1
[2016-03-22 12:26:01.543635] I [MSGID: 108031] [afr-common.c:1782:afr_local_discovery_cbk] 0-c_glusterfs-replicate-0: selecting local read_child c_glusterfs-client-417
[2016-03-22 12:26:01.544690] I [MSGID: 104041] [glfs-resolve.c:862:__glfs_active_subvol] 0-c_glusterfs: switched to graph 30303235-3030-2d31-3733-30302d323031 (0)
[2016-03-22 12:26:06.908603] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:26:06.936348] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2016-03-22 12:26:06.937570] I [MSGID: 104045] [glfs-master.c:95:notify] 0-gfapi: New graph 30303235-3030-2d31-3735-36312d323031 (0) coming up
[2016-03-22 12:26:06.937632] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-417: parent translators are ready, attempting connect on transport
[2016-03-22 12:26:06.938852] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-418: parent translators are ready, attempting connect on transport
[2016-03-22 12:26:06.939343] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-417: changing port to 49719 (from 0)
[2016-03-22 12:26:06.940706] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-417: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:26:06.940839] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-418: changing port to 49730 (from 0)
[2016-03-22 12:26:06.941242] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-417: Connected to c_glusterfs-client-417, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:26:06.941275] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-417: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:26:06.941450] I [MSGID: 108005] [afr-common.c:3841:afr_notify] 0-c_glusterfs-replicate-0: Subvolume 'c_glusterfs-client-417' came back up; going online.
[2016-03-22 12:26:06.941542] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-417: Server lk version = 1
[2016-03-22 12:26:06.942313] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-418: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:26:06.943027] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-418: Connected to c_glusterfs-client-418, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:26:06.943059] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-418: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:26:06.974816] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-418: Server lk version = 1
[2016-03-22 12:26:06.977015] I [MSGID: 108031] [afr-common.c:1782:afr_local_discovery_cbk] 0-c_glusterfs-replicate-0: selecting local read_child c_glusterfs-client-417
[2016-03-22 12:26:06.978601] I [MSGID: 104041] [glfs-resolve.c:862:__glfs_active_subvol] 0-c_glusterfs: switched to graph 30303235-3030-2d31-3735-36312d323031 (0)
$ 

EVOA_8300-10_6B> lhsh 002500 cat /var/log/glusterfs/glustershd.log

160322-13:32:13 10.67.29.90 16.0c CPP_MOM-CPP-LSV184-gen1_gen1_COMPLETE stopfile=/tmp/13910
$ lhsh 002500 cat /var/log/glusterfs/glustershd.log
[2016-03-22 12:20:08.798347] I [MSGID: 100030] [glusterfsd.c:2318:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.6 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/glustershd -p /system/glusterd/glustershd/run/glustershd.pid -l /var/log/glusterfs/glustershd.log -S /var/run/gluster/8426fb5fe867c36346144bff93f74572.socket --xlator-option *replicate*.node-uuid=74c9e3bb-8657-4e74-9888-d1c1d76bb1f2)
[2016-03-22 12:20:08.826061] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:20:08.831126] I [graph.c:269:gf_add_cmdline_options] 0-c_glusterfs-replicate-0: adding option 'node-uuid' for volume 'c_glusterfs-replicate-0' with value '74c9e3bb-8657-4e74-9888-d1c1d76bb1f2'
[2016-03-22 12:20:08.835108] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2016-03-22 12:20:08.836296] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-416: parent translators are ready, attempting connect on transport
[2016-03-22 12:20:08.837257] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-417: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
  1: volume c_glusterfs-client-416
  2:     type protocol/client
  3:     option ping-timeout 4
  4:     option remote-host 10.32.0.48
  5:     option remote-subvolume /opt/lvmdir/c2/brick
  6:     option transport-type socket
  7:     option username 06afe471-c710-4493-9ab0-b74b9e66f01f
  8:     option password 6440d9dd-6ccb-44a1-a0eb-c6898ca84178
  9: end-volume
 10:  
 11: volume c_glusterfs-client-417
 12:     type protocol/client
 13:     option ping-timeout 4
 14:     option remote-host 10.32.1.144
 15:     option remote-subvolume /opt/lvmdir/c2/brick
 16:     option transport-type socket
 17:     option username 06afe471-c710-4493-9ab0-b74b9e66f01f
 18:     option password 6440d9dd-6ccb-44a1-a0eb-c6898ca84178
 19: end-volume
 20:  
 21: volume c_glusterfs-replicate-0
 22:     type cluster/replicate
 23:     option node-uuid 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
 24:     option background-self-heal-count 0
[2016-03-22 12:20:08.839042] E [MSGID: 114058] [client-handshake.c:1524:client_query_portmap_cbk] 0-c_glusterfs-client-417: failed to get the port number for remote subvolume. Please run 'gluster volume status' on server to see if brick process is running.
 25:     option metadata-self-heal on
 26:     option data-self-heal on
[2016-03-22 12:20:08.839183] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-416: changing port to 49729 (from 0)
[2016-03-22 12:20:08.839274] I [MSGID: 114018] [client.c:2042:client_rpc_notify] 0-c_glusterfs-client-417: disconnected from c_glusterfs-client-417. Client process will keep trying to connect to glusterd until brick's port is available
 27:     option entry-self-heal on
 28:     option self-heal-daemon enable
 29:     option iam-self-heal-daemon yes
 30:     subvolumes c_glusterfs-client-416 c_glusterfs-client-417
 31: end-volume
 32:  
 33: volume glustershd
 34:     type debug/io-stats
 35:     subvolumes c_glusterfs-replicate-0
 36: end-volume
 37:  
+------------------------------------------------------------------------------+
[2016-03-22 12:20:08.842975] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-416: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:20:08.843832] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-416: Connected to c_glusterfs-client-416, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:20:12.415372] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-416: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:20:12.415515] I [MSGID: 108005] [afr-common.c:3841:afr_notify] 0-c_glusterfs-replicate-0: Subvolume 'c_glusterfs-client-416' came back up; going online.
[2016-03-22 12:20:12.415838] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-416: Server lk version = 1
[2016-03-22 12:20:16.374650] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-417: changing port to 49719 (from 0)
[2016-03-22 12:20:16.376044] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-417: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:20:16.382873] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-417: Connected to c_glusterfs-client-417, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:20:16.382946] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-417: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:20:16.384687] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-417: Server lk version = 1
[2016-03-22 12:20:16.464388] I [MSGID: 108026] [afr-self-heal-common.c:651:afr_log_selfheal] 0-c_glusterfs-replicate-0: Completed data selfheal on 0d40ab37-e9c2-423d-a81d-8090a6b7596c. source=0 sinks=1 
[2016-03-22 12:24:28.465472] W [socket.c:588:__socket_rwv] 0-c_glusterfs-client-416: readv on 10.32.0.48:49729 failed (No data available)
[2016-03-22 12:24:28.465550] I [MSGID: 114018] [client.c:2042:client_rpc_notify] 0-c_glusterfs-client-416: disconnected from c_glusterfs-client-416. Client process will keep trying to connect to glusterd until brick's port is available
[2016-03-22 12:25:20.453009] I [glusterfsd-mgmt.c:57:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2016-03-22 12:25:20.532755] I [glusterfsd-mgmt.c:1513:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2016-03-22 12:25:23.255282] W [glusterfsd.c:1236:cleanup_and_exit] (-->/lib64/libpthread.so.0() [0x80f02fb730] -->/usr/sbin/glusterfs(glusterfs_sigwaiter-0x1bdfc) [0x10009b04] -->/usr/sbin/glusterfs(cleanup_and_exit-0x1c02c) [0x100098bc] ) 0-: received signum (15), shutting down
[2016-03-22 12:25:24.275843] I [MSGID: 100030] [glusterfsd.c:2318:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.6 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/glustershd -p /system/glusterd/glustershd/run/glustershd.pid -l /var/log/glusterfs/glustershd.log -S /var/run/gluster/8426fb5fe867c36346144bff93f74572.socket --xlator-option *replicate*.node-uuid=74c9e3bb-8657-4e74-9888-d1c1d76bb1f2)
[2016-03-22 12:25:24.298338] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:25:24.303579] I [graph.c:269:gf_add_cmdline_options] 0-c_glusterfs-replicate-0: adding option 'node-uuid' for volume 'c_glusterfs-replicate-0' with value '74c9e3bb-8657-4e74-9888-d1c1d76bb1f2'
[2016-03-22 12:25:24.307753] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2016-03-22 12:25:24.309025] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-417: parent translators are ready, attempting connect on transport
[2016-03-22 12:25:24.310217] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-418: parent translators are ready, attempting connect on transport
[2016-03-22 12:25:24.310793] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-417: changing port to 49719 (from 0)
Final graph:
+------------------------------------------------------------------------------+
  1: volume c_glusterfs-client-417
  2:     type protocol/client
  3:     option ping-timeout 4
  4:     option remote-host 10.32.1.144
  5:     option remote-subvolume /opt/lvmdir/c2/brick
  6:     option transport-type socket
  7:     option username 06afe471-c710-4493-9ab0-b74b9e66f01f
  8:     option password 6440d9dd-6ccb-44a1-a0eb-c6898ca84178
  9: end-volume
 10:  
 11: volume c_glusterfs-client-418
 12:     type protocol/client
 13:     option ping-timeout 4
 14:     option remote-host 10.32.0.48
 15:     option remote-subvolume /opt/lvmdir/c2/brick
 16:     option transport-type socket
 17:     option username 06afe471-c710-4493-9ab0-b74b9e66f01f
 18:     option password 6440d9dd-6ccb-44a1-a0eb-c6898ca84178
 19: end-volume
 20:  
 21: volume c_glusterfs-replicate-0
 22:     type cluster/replicate
 23:     option node-uuid 74c9e3bb-8657-4e74-9888-d1c1d76bb1f2
 24:     option background-self-heal-count 0
 25:     option metadata-self-heal on
 26:     option data-self-heal on
 27:     option entry-self-heal on
 28:     option self-heal-daemon enable
 29:     option iam-self-heal-daemon yes
 30:     subvolumes c_glusterfs-client-417 c_glusterfs-client-418
 31: end-volume
 32:  
 33: volume glustershd
 34:     type debug/io-stats
 35:     subvolumes c_glusterfs-replicate-0
 36: end-volume
 37:  
+------------------------------------------------------------------------------+
[2016-03-22 12:25:24.312361] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-417: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:25:24.312985] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-417: Connected to c_glusterfs-client-417, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:25:24.313032] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-417: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:25:24.313197] I [MSGID: 108005] [afr-common.c:3841:afr_notify] 0-c_glusterfs-replicate-0: Subvolume 'c_glusterfs-client-417' came back up; going online.
[2016-03-22 12:25:24.313382] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-417: Server lk version = 1
[2016-03-22 12:25:24.313605] E [MSGID: 114058] [client-handshake.c:1524:client_query_portmap_cbk] 0-c_glusterfs-client-418: failed to get the port number for remote subvolume. Please run 'gluster volume status' on server to see if brick process is running.
[2016-03-22 12:25:24.313735] I [MSGID: 114018] [client.c:2042:client_rpc_notify] 0-c_glusterfs-client-418: disconnected from c_glusterfs-client-418. Client process will keep trying to connect to glusterd until brick's port is available
[2016-03-22 12:25:24.393972] I [glusterfsd-mgmt.c:57:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2016-03-22 12:25:24.483079] I [glusterfsd-mgmt.c:1513:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2016-03-22 12:25:28.282241] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-418: changing port to 49730 (from 0)
[2016-03-22 12:25:28.283876] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-418: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:25:28.284722] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-418: Connected to c_glusterfs-client-418, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:25:28.284769] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-418: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:25:28.285301] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-418: Server lk version = 1
[2016-03-22 12:25:28.318997] I [MSGID: 108026] [afr-self-heal-common.c:651:afr_log_selfheal] 0-c_glusterfs-replicate-0: Completed data selfheal on df00d265-4b67-415d-95a7-f9e4cac83029. source=0 sinks=1 
$ 

EVOA_8300-10_6B> lhsh 002500 cat /var/log/glusterfs/mnt-c.log

160322-13:32:15 10.67.29.90 16.0c CPP_MOM-CPP-LSV184-gen1_gen1_COMPLETE stopfile=/tmp/13910
$ lhsh 002500 cat /var/log/glusterfs/mnt-c.log
[2016-03-22 12:20:24.185354] I [MSGID: 100030] [glusterfsd.c:2318:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.6 (args: /usr/sbin/glusterfs --volfile-server=10.32.1.144 --volfile-id=c_glusterfs /mnt/c)
[2016-03-22 12:20:24.204581] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-03-22 12:20:24.220545] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2016-03-22 12:20:24.221729] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-416: parent translators are ready, attempting connect on transport
[2016-03-22 12:20:24.222776] I [MSGID: 114020] [client.c:2118:notify] 0-c_glusterfs-client-417: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
  1: volume c_glusterfs-client-416
  2:     type protocol/client
  3:     option ping-timeout 4
  4:     option remote-host 10.32.0.48
  5:     option remote-subvolume /opt/lvmdir/c2/brick
  6:     option transport-type socket
  7:     option username 06afe471-c710-4493-9ab0-b74b9e66f01f
  8:     option password 6440d9dd-6ccb-44a1-a0eb-c6898ca84178
  9:     option send-gids true
 10: end-volume
 11:  
 12: volume c_glusterfs-client-417
 13:     type protocol/client
 14:     option ping-timeout 4
 15:     option remote-host 10.32.1.144
 16:     option remote-subvolume /opt/lvmdir/c2/brick
 17:     option transport-type socket
 18:     option username 06afe471-c710-4493-9ab0-b74b9e66f01f
 19:     option password 6440d9dd-6ccb-44a1-a0eb-c6898ca84178
 20:     option send-gids true
 21: end-volume
 22:  
 23: volume c_glusterfs-replicate-0
 24:     type cluster/replicate
 25:     subvolumes c_glusterfs-client-416 c_glusterfs-client-417
 26: end-volume
 27:  
 28: volume c_glusterfs-dht
 29:     type cluster/distribute
 30:     subvolumes c_glusterfs-replicate-0
 31: end-volume
 32:  
 33: volume c_glusterfs-write-behind
 34:     type performance/write-behind
 35:     subvolumes c_glusterfs-dht
 36: end-volume
 37:  
 38: volume c_glusterfs-read-ahead
 39:     type performance/read-ahead
 40:     subvolumes c_glusterfs-write-behind
 41: end-volume
 42:  
 43: volume c_glusterfs-readdir-ahead
 44:     type performance/readdir-ahead
 45:     subvolumes c_glusterfs-read-ahead
 46: end-volume
 47:  
 48: volume c_glusterfs-io-cache
 49:     type performance/io-cache
 50:     subvolumes c_glusterfs-readdir-ahead
 51: end-volume
 52:  
 53: volume c_glusterfs-quick-read
 54:     type performance/quick-read
 55:     subvolumes c_glusterfs-io-cache
 56: end-volume
 57:  
 58: volume c_glusterfs-open-behind
 59:     type performance/open-behind
[2016-03-22 12:20:24.223965] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-417: changing port to 49719 (from 0)
 60:     subvolumes c_glusterfs-quick-read
 61: end-volume
 62:  
 63: volume c_glusterfs-md-cache
 64:     type performance/md-cache
 65:     subvolumes c_glusterfs-open-behind
 66: end-volume
[2016-03-22 12:20:24.224186] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 0-c_glusterfs-client-416: changing port to 49729 (from 0)
 67:  
 68: volume c_glusterfs
 69:     type debug/io-stats
 70:     option latency-measurement off
 71:     option count-fop-hits off
 72:     subvolumes c_glusterfs-md-cache
 73: end-volume
 74:  
 75: volume meta-autoload
 76:     type meta
 77:     subvolumes c_glusterfs
 78: end-volume
 79:  
+------------------------------------------------------------------------------+
[2016-03-22 12:20:24.225529] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-417: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:20:24.226109] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-c_glusterfs-client-416: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:20:24.226168] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-417: Connected to c_glusterfs-client-417, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:20:24.226210] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-417: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:20:24.226359] I [MSGID: 108005] [afr-common.c:3841:afr_notify] 0-c_glusterfs-replicate-0: Subvolume 'c_glusterfs-client-417' came back up; going online.
[2016-03-22 12:20:24.226442] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-417: Server lk version = 1
[2016-03-22 12:20:24.226838] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-c_glusterfs-client-416: Connected to c_glusterfs-client-416, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:20:24.226867] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-c_glusterfs-client-416: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:20:24.235897] I [fuse-bridge.c:5137:fuse_graph_setup] 0-fuse: switched to graph 0
[2016-03-22 12:20:24.236164] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-c_glusterfs-client-416: Server lk version = 1
[2016-03-22 12:20:24.236272] I [fuse-bridge.c:4030:fuse_init] 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.22 kernel 7.22
[2016-03-22 12:20:24.238137] I [MSGID: 108031] [afr-common.c:1782:afr_local_discovery_cbk] 0-c_glusterfs-replicate-0: selecting local read_child c_glusterfs-client-417
[2016-03-22 12:20:24.479120] I [MSGID: 108026] [afr-self-heal-common.c:651:afr_log_selfheal] 0-c_glusterfs-replicate-0: Completed data selfheal on 2c169915-6363-46ef-b0d4-e49859f64ab6. source=0 sinks=1 
[2016-03-22 12:24:28.465488] W [socket.c:588:__socket_rwv] 0-c_glusterfs-client-416: readv on 10.32.0.48:49729 failed (No data available)
[2016-03-22 12:24:28.465552] I [MSGID: 114018] [client.c:2042:client_rpc_notify] 0-c_glusterfs-client-416: disconnected from c_glusterfs-client-416. Client process will keep trying to connect to glusterd until brick's port is available
[2016-03-22 12:24:29.588062] I [MSGID: 108031] [afr-common.c:1782:afr_local_discovery_cbk] 0-c_glusterfs-replicate-0: selecting local read_child c_glusterfs-client-417
[2016-03-22 12:25:00.311359] W [MSGID: 114031] [client-rpc-fops.c:1845:client3_3_xattrop_cbk] 0-c_glusterfs-client-417: remote operation failed. Path: /configuration/oamrdncy.cfg (df00d265-4b67-415d-95a7-f9e4cac83029) [No space left on device]
[2016-03-22 12:25:00.332902] W [MSGID: 114031] [client-rpc-fops.c:1914:client3_3_fxattrop_cbk] 0-c_glusterfs-client-417: remote operation failed [No space left on device]
[2016-03-22 12:25:20.453138] I [glusterfsd-mgmt.c:57:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2016-03-22 12:25:20.538603] I [MSGID: 114020] [client.c:2118:notify] 2-c_glusterfs-client-417: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
  1: volume c_glusterfs-client-417
  2:     type protocol/client
  3:     option ping-timeout 4
  4:     option remote-host 10.32.1.144
  5:     option remote-subvolume /opt/lvmdir/c2/brick
  6:     option transport-type socket
  7:     option username 06afe471-c710-4493-9ab0-b74b9e66f01f
  8:     option password 6440d9dd-6ccb-44a1-a0eb-c6898ca84178
  9:     option send-gids true
 10: end-volume
 11:  
 12: volume c_glusterfs-dht
 13:     type cluster/distribute
 14:     subvolumes c_glusterfs-client-417
 15: end-volume
 16:  
 17: volume c_glusterfs-write-behind
 18:     type performance/write-behind
 19:     subvolumes c_glusterfs-dht
 20: end-volume
 21:  
 22: volume c_glusterfs-read-ahead
 23:     type performance/read-ahead
 24:     subvolumes c_glusterfs-write-behind
 25: end-volume
 26:  
 27: volume c_glusterfs-readdir-ahead
 28:     type performance/readdir-ahead
 29:     subvolumes c_glusterfs-read-ahead
 30: end-volume
 31:  
 32: volume c_glusterfs-io-cache
 33:     type performance/io-cache
 34:     subvolumes c_glusterfs-readdir-ahead
 35: end-volume
 36:  
 37: volume c_glusterfs-quick-read
 38:     type performance/quick-read
 39:     subvolumes c_glusterfs-io-cache
 40: end-volume
 41:  
 42: volume c_glusterfs-open-behind
 43:     type performance/open-behind
 44:     subvolumes c_glusterfs-quick-read
 45: end-volume
 46:  
 47: volume c_glusterfs-md-cache
 48:     type performance/md-cache
 49:     subvolumes c_glusterfs-open-behind
 50: end-volume
 51:  
 52: volume c_glusterfs
 53:     type debug/io-stats
 54:     option latency-measurement off
 55:     option count-fop-hits off
 56:     subvolumes c_glusterfs-md-cache
 57: end-volume
 58:  
[2016-03-22 12:25:20.540158] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 2-c_glusterfs-client-417: changing port to 49719 (from 0)
 59: volume meta-autoload
 60:     type meta
 61:     subvolumes c_glusterfs
 62: end-volume
 63:  
+------------------------------------------------------------------------------+
[2016-03-22 12:25:20.541589] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 2-c_glusterfs-client-417: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:25:20.542143] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 2-c_glusterfs-client-417: Connected to c_glusterfs-client-417, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:25:20.542175] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 2-c_glusterfs-client-417: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:25:20.552716] I [fuse-bridge.c:5137:fuse_graph_setup] 0-fuse: switched to graph 2
[2016-03-22 12:25:20.552887] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 2-c_glusterfs-client-417: Server lk version = 1
[2016-03-22 12:25:20.556229] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a465c) to newfd (0x3fff9011144c) (inode-gfid:bd2ba80d-cd65-43a1-b160-54fd7e33cef7)(old-subvolume:meta-autoload-0 new-subvolume:meta-autoload-2)
[2016-03-22 12:25:20.558200] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4584) to newfd (0x3fff901114b8) (inode-gfid:9891c01a-1b1a-45c9-8000-b989f663b8ad)(old-subvolume:meta-autoload-0 new-subvolume:meta-autoload-2)
[2016-03-22 12:25:20.560143] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4734) to newfd (0x3fff90111524) (inode-gfid:bbc827a7-88bb-4d18-ac62-0941ffb0992a)(old-subvolume:meta-autoload-0 new-subvolume:meta-autoload-2)
[2016-03-22 12:25:20.561983] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a44ac) to newfd (0x3fff90111590) (inode-gfid:a55f863d-645b-4a05-a4e9-82128f52fbe5)(old-subvolume:meta-autoload-0 new-subvolume:meta-autoload-2)
[2016-03-22 12:25:20.572480] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a46c8) to newfd (0x3fff901115fc) (inode-gfid:0d40ab37-e9c2-423d-a81d-8090a6b7596c)(old-subvolume:meta-autoload-0 new-subvolume:meta-autoload-2)
[2016-03-22 12:25:20.574355] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a47a0) to newfd (0x3fff90111668) (inode-gfid:0e9f1dd8-fcd6-4cda-9afe-91565a73acf9)(old-subvolume:meta-autoload-0 new-subvolume:meta-autoload-2)
[2016-03-22 12:25:20.576158] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a45f0) to newfd (0x3fff901116d4) (inode-gfid:708a857c-7d0e-4104-804a-37c3155e8ffa)(old-subvolume:meta-autoload-0 new-subvolume:meta-autoload-2)
[2016-03-22 12:25:20.578111] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4518) to newfd (0x3fff90111740) (inode-gfid:0e7a15bc-b5ac-4ea5-98dc-c8b74ac33596)(old-subvolume:meta-autoload-0 new-subvolume:meta-autoload-2)
[2016-03-22 12:25:20.579811] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a480c) to newfd (0x3fff901117ac) (inode-gfid:629a8c8b-d97c-44a3-be72-3ce0a81e2796)(old-subvolume:meta-autoload-0 new-subvolume:meta-autoload-2)
[2016-03-22 12:25:20.581887] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4878) to newfd (0x3fff90111818) (inode-gfid:3f2b622b-62cc-443b-9f1e-aa19ceaa9a6f)(old-subvolume:meta-autoload-0 new-subvolume:meta-autoload-2)
[2016-03-22 12:25:20.583703] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a48e4) to newfd (0x3fff90111884) (inode-gfid:a8e96c61-b1df-465e-991b-1ca7c8575f30)(old-subvolume:meta-autoload-0 new-subvolume:meta-autoload-2)
[2016-03-22 12:25:20.585267] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4950) to newfd (0x3fff901118f0) (inode-gfid:298de4ee-7635-45b1-913a-3144af23db9a)(old-subvolume:meta-autoload-0 new-subvolume:meta-autoload-2)
[2016-03-22 12:25:20.586887] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a49bc) to newfd (0x3fff9011195c) (inode-gfid:42111812-ae43-4133-9a4b-5a16d7891bf6)(old-subvolume:meta-autoload-0 new-subvolume:meta-autoload-2)
[2016-03-22 12:25:20.588472] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4a28) to newfd (0x3fff901119c8) (inode-gfid:b8e0811b-4714-49d8-8d5e-5a18a548985d)(old-subvolume:meta-autoload-0 new-subvolume:meta-autoload-2)
[2016-03-22 12:25:20.590065] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4a94) to newfd (0x3fff90111a34) (inode-gfid:b6f8b679-887d-4510-b24a-e8ca5034d336)(old-subvolume:meta-autoload-0 new-subvolume:meta-autoload-2)
[2016-03-22 12:25:20.591864] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4b00) to newfd (0x3fff90111aa0) (inode-gfid:3a970ce9-275e-47ae-80be-84045320ab7f)(old-subvolume:meta-autoload-0 new-subvolume:meta-autoload-2)
[2016-03-22 12:25:20.593467] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4b6c) to newfd (0x3fff90111b0c) (inode-gfid:2d2577c5-9f61-412f-aa4e-9290159a2fba)(old-subvolume:meta-autoload-0 new-subvolume:meta-autoload-2)
[2016-03-22 12:25:20.595466] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4bd8) to newfd (0x3fff90111b78) (inode-gfid:fed16f0b-286d-4ee7-8153-c36c5e9281ce)(old-subvolume:meta-autoload-0 new-subvolume:meta-autoload-2)
[2016-03-22 12:25:20.598827] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4c44) to newfd (0x3fff90111be4) (inode-gfid:6c18054b-d4cf-47ef-9cb1-3a7ba0e093fc)(old-subvolume:meta-autoload-0 new-subvolume:meta-autoload-2)
[2016-03-22 12:25:20.600603] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4cb0) to newfd (0x3fff90111c50) (inode-gfid:5d0f3428-b2ea-4972-91fb-d29b0df26169)(old-subvolume:meta-autoload-0 new-subvolume:meta-autoload-2)
[2016-03-22 12:25:20.602575] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4d1c) to newfd (0x3fff90111cbc) (inode-gfid:0903f32c-98c2-4424-9aec-c9118e807fea)(old-subvolume:meta-autoload-0 new-subvolume:meta-autoload-2)
[2016-03-22 12:25:20.604222] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4d88) to newfd (0x3fff90111d28) (inode-gfid:a1878f97-1294-4529-b424-fbcdf9689c25)(old-subvolume:meta-autoload-0 new-subvolume:meta-autoload-2)
[2016-03-22 12:25:20.606087] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4df4) to newfd (0x3fff90111d94) (inode-gfid:5d2acdfc-3475-4f8c-a8a5-4306183e38c5)(old-subvolume:meta-autoload-0 new-subvolume:meta-autoload-2)
[2016-03-22 12:25:20.607658] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4e60) to newfd (0x3fff90111e00) (inode-gfid:9b82588d-8933-4143-a56f-2a272121c36d)(old-subvolume:meta-autoload-0 new-subvolume:meta-autoload-2)
[2016-03-22 12:25:20.609141] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4ecc) to newfd (0x3fff90111e6c) (inode-gfid:249ad0ee-17df-4ff9-8fe1-4a3239f325d0)(old-subvolume:meta-autoload-0 new-subvolume:meta-autoload-2)
[2016-03-22 12:25:20.610755] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4f38) to newfd (0x3fff90111ed8) (inode-gfid:0c5b1280-9f3c-4bfd-982a-786feb910d08)(old-subvolume:meta-autoload-0 new-subvolume:meta-autoload-2)
[2016-03-22 12:25:20.610910] I [MSGID: 114021] [client.c:2127:notify] 0-c_glusterfs-client-416: current graph is no longer active, destroying rpc_client 
[2016-03-22 12:25:20.611005] I [MSGID: 114021] [client.c:2127:notify] 0-c_glusterfs-client-417: current graph is no longer active, destroying rpc_client 
[2016-03-22 12:25:20.611105] I [MSGID: 114018] [client.c:2042:client_rpc_notify] 0-c_glusterfs-client-417: disconnected from c_glusterfs-client-417. Client process will keep trying to connect to glusterd until brick's port is available
[2016-03-22 12:25:20.611142] E [MSGID: 108006] [afr-common.c:3880:afr_notify] 0-c_glusterfs-replicate-0: All subvolumes are down. Going offline until atleast one of them comes back up.
[2016-03-22 12:25:23.835796] I [MSGID: 109066] [dht-rename.c:1411:dht_rename] 2-c_glusterfs-dht: renaming /logfiles/security/CELLO_SECURITYEVENT_LOG.xml (hash=c_glusterfs-client-417/cache=c_glusterfs-client-417) => /logfiles/security/X912244659_CELLO_SECURITYEVENT_LOG_20160322T122523.push (hash=c_glusterfs-client-417/cache=<nul>)
[2016-03-22 12:25:24.393820] I [glusterfsd-mgmt.c:57:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2016-03-22 12:25:24.495211] I [MSGID: 114020] [client.c:2118:notify] 4-c_glusterfs-client-417: parent translators are ready, attempting connect on transport
[2016-03-22 12:25:24.496582] I [MSGID: 114020] [client.c:2118:notify] 4-c_glusterfs-client-418: parent translators are ready, attempting connect on transport
[2016-03-22 12:25:24.497057] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 4-c_glusterfs-client-417: changing port to 49719 (from 0)
Final graph:
+------------------------------------------------------------------------------+
  1: volume c_glusterfs-client-417
  2:     type protocol/client
  3:     option ping-timeout 4
  4:     option remote-host 10.32.1.144
  5:     option remote-subvolume /opt/lvmdir/c2/brick
  6:     option transport-type socket
  7:     option username 06afe471-c710-4493-9ab0-b74b9e66f01f
  8:     option password 6440d9dd-6ccb-44a1-a0eb-c6898ca84178
  9:     option send-gids true
 10: end-volume
 11:  
 12: volume c_glusterfs-client-418
 13:     type protocol/client
 14:     option ping-timeout 4
 15:     option remote-host 10.32.0.48
 16:     option remote-subvolume /opt/lvmdir/c2/brick
 17:     option transport-type socket
 18:     option username 06afe471-c710-4493-9ab0-b74b9e66f01f
 19:     option password 6440d9dd-6ccb-44a1-a0eb-c6898ca84178
 20:     option send-gids true
 21: end-volume
 22:  
 23: volume c_glusterfs-replicate-0
 24:     type cluster/replicate
 25:     subvolumes c_glusterfs-client-417 c_glusterfs-client-418
 26: end-volume
 27:  
 28: volume c_glusterfs-dht
 29:     type cluster/distribute
 30:     subvolumes c_glusterfs-replicate-0
 31: end-volume
 32:  
 33: volume c_glusterfs-write-behind
 34:     type performance/write-behind
 35:     subvolumes c_glusterfs-dht
 36: end-volume
 37:  
 38: volume c_glusterfs-read-ahead
 39:     type performance/read-ahead
 40:     subvolumes c_glusterfs-write-behind
 41: end-volume
 42:  
 43: volume c_glusterfs-readdir-ahead
 44:     type performance/readdir-ahead
 45:     subvolumes c_glusterfs-read-ahead
 46: end-volume
 47:  
 48: volume c_glusterfs-io-cache
 49:     type performance/io-cache
 50:     subvolumes c_glusterfs-readdir-ahead
 51: end-volume
 52:  
 53: volume c_glusterfs-quick-read
 54:     type performance/quick-read
 55:     subvolumes c_glusterfs-io-cache
 56: end-volume
 57:  
 58: volume c_glusterfs-open-behind
 59:     type performance/open-behind
 60:     subvolumes c_glusterfs-quick-read
 61: end-volume
 62:  
 63: volume c_glusterfs-md-cache
 64:     type performance/md-cache
 65:     subvolumes c_glusterfs-open-behind
 66: end-volume
 67:  
 68: volume c_glusterfs
 69:     type debug/io-stats
 70:     option latency-measurement off
 71:     option count-fop-hits off
[2016-03-22 12:25:24.498333] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 4-c_glusterfs-client-417: Using Program GlusterFS 3.3, Num (1298437), Version (330)
 72:     subvolumes c_glusterfs-md-cache
 73: end-volume
 74:  
 75: volume meta-autoload
 76:     type meta
 77:     subvolumes c_glusterfs
 78: end-volume
 79:  
+------------------------------------------------------------------------------+
[2016-03-22 12:25:24.498883] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 4-c_glusterfs-client-417: Connected to c_glusterfs-client-417, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:25:24.498928] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 4-c_glusterfs-client-417: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:25:24.499098] I [MSGID: 108005] [afr-common.c:3841:afr_notify] 4-c_glusterfs-replicate-0: Subvolume 'c_glusterfs-client-417' came back up; going online.
[2016-03-22 12:25:24.499174] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 4-c_glusterfs-client-417: Server lk version = 1
[2016-03-22 12:25:24.600051] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 4-c_glusterfs-client-418: changing port to 49730 (from 0)
[2016-03-22 12:25:24.602645] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 4-c_glusterfs-client-418: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2016-03-22 12:25:24.603483] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 4-c_glusterfs-client-418: Connected to c_glusterfs-client-418, attached to remote volume '/opt/lvmdir/c2/brick'.
[2016-03-22 12:25:24.603515] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 4-c_glusterfs-client-418: Server and Client lk-version numbers are not same, reopening the fds
[2016-03-22 12:25:24.612990] I [fuse-bridge.c:5137:fuse_graph_setup] 0-fuse: switched to graph 4
[2016-03-22 12:25:24.613189] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 4-c_glusterfs-client-418: Server lk version = 1
[2016-03-22 12:25:24.726984] I [MSGID: 108031] [afr-common.c:1782:afr_local_discovery_cbk] 4-c_glusterfs-replicate-0: selecting local read_child c_glusterfs-client-417
[2016-03-22 12:25:24.743752] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a465c) to newfd (0x3fff886fc3dc) (inode-gfid:bd2ba80d-cd65-43a1-b160-54fd7e33cef7)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.749359] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4584) to newfd (0x3fff886fc448) (inode-gfid:9891c01a-1b1a-45c9-8000-b989f663b8ad)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.758180] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4734) to newfd (0x3fff886fc4b4) (inode-gfid:bbc827a7-88bb-4d18-ac62-0941ffb0992a)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.763524] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a44ac) to newfd (0x3fff886fc520) (inode-gfid:a55f863d-645b-4a05-a4e9-82128f52fbe5)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.790914] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a46c8) to newfd (0x3fff886fc58c) (inode-gfid:0d40ab37-e9c2-423d-a81d-8090a6b7596c)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.800665] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a47a0) to newfd (0x3fff886fc5f8) (inode-gfid:0e9f1dd8-fcd6-4cda-9afe-91565a73acf9)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.805338] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a45f0) to newfd (0x3fff886fc664) (inode-gfid:708a857c-7d0e-4104-804a-37c3155e8ffa)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.811870] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4518) to newfd (0x3fff886fc6d0) (inode-gfid:0e7a15bc-b5ac-4ea5-98dc-c8b74ac33596)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.818811] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a480c) to newfd (0x3fff886fc73c) (inode-gfid:629a8c8b-d97c-44a3-be72-3ce0a81e2796)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.826607] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4878) to newfd (0x3fff886fc7a8) (inode-gfid:3f2b622b-62cc-443b-9f1e-aa19ceaa9a6f)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.830584] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a48e4) to newfd (0x3fff886fc814) (inode-gfid:a8e96c61-b1df-465e-991b-1ca7c8575f30)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.834745] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4950) to newfd (0x3fff886fc880) (inode-gfid:298de4ee-7635-45b1-913a-3144af23db9a)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.840327] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a49bc) to newfd (0x3fff886fc8ec) (inode-gfid:42111812-ae43-4133-9a4b-5a16d7891bf6)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.846991] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4a28) to newfd (0x3fff886fc958) (inode-gfid:b8e0811b-4714-49d8-8d5e-5a18a548985d)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.853031] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4a94) to newfd (0x3fff886fc9c4) (inode-gfid:b6f8b679-887d-4510-b24a-e8ca5034d336)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.859827] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4b00) to newfd (0x3fff886fca30) (inode-gfid:3a970ce9-275e-47ae-80be-84045320ab7f)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.864127] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4b6c) to newfd (0x3fff886fca9c) (inode-gfid:2d2577c5-9f61-412f-aa4e-9290159a2fba)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.875065] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4bd8) to newfd (0x3fff886fcb08) (inode-gfid:fed16f0b-286d-4ee7-8153-c36c5e9281ce)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.879096] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4c44) to newfd (0x3fff886fcb74) (inode-gfid:6c18054b-d4cf-47ef-9cb1-3a7ba0e093fc)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.882974] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4cb0) to newfd (0x3fff886fcbe0) (inode-gfid:5d0f3428-b2ea-4972-91fb-d29b0df26169)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.889333] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4d1c) to newfd (0x3fff886fcc4c) (inode-gfid:0903f32c-98c2-4424-9aec-c9118e807fea)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.893179] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4d88) to newfd (0x3fff886fccb8) (inode-gfid:a1878f97-1294-4529-b424-fbcdf9689c25)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.907716] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4df4) to newfd (0x3fff886fcd24) (inode-gfid:5d2acdfc-3475-4f8c-a8a5-4306183e38c5)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.911601] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4e60) to newfd (0x3fff886fcd90) (inode-gfid:9b82588d-8933-4143-a56f-2a272121c36d)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.915221] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4ecc) to newfd (0x3fff886fcdfc) (inode-gfid:249ad0ee-17df-4ff9-8fe1-4a3239f325d0)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.918736] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff900a4f38) to newfd (0x3fff886fce68) (inode-gfid:0c5b1280-9f3c-4bfd-982a-786feb910d08)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.922303] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff90111f44) to newfd (0x3fff886fced4) (inode-gfid:8bf1250f-46e6-47e0-b774-4aa2eef4ec07)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.925664] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff9011201c) to newfd (0x3fff886fcf40) (inode-gfid:af386522-099e-4b83-8243-72859bc14e9e)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.929045] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff90112088) to newfd (0x3fff886fcfac) (inode-gfid:c2900b92-f9cd-4370-9e24-1c8cca0b0199)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.932952] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff901120f4) to newfd (0x3fff886fd018) (inode-gfid:ed70040f-63fd-40ea-be83-889318ce52a2)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.936473] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff90112160) to newfd (0x3fff886fd084) (inode-gfid:d0c218c7-6dbb-4b91-a229-8d82af4a05b9)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.940142] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff901121cc) to newfd (0x3fff886fd0f0) (inode-gfid:a28bca90-c040-4793-9123-bfaf4817ca50)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.945273] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff90112238) to newfd (0x3fff886fd15c) (inode-gfid:361a64cf-d6b9-42ab-ac90-326618b15114)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.948581] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff901122a4) to newfd (0x3fff886fd1c8) (inode-gfid:957f2a81-536b-49d0-9e23-370753cb3014)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.953213] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff90112310) to newfd (0x3fff886fd234) (inode-gfid:adbc4a69-24aa-42d4-bcd2-f49ca8bf81e7)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.956360] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff9011237c) to newfd (0x3fff886fd2a0) (inode-gfid:53d57e35-6980-4a3a-9415-5025b859989c)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.963725] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff901123e8) to newfd (0x3fff886fd30c) (inode-gfid:7e33cd4b-d8d5-40ea-b208-19e6f2160edc)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.966981] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff90112454) to newfd (0x3fff886fd378) (inode-gfid:3ea4c25b-63df-4231-b9f0-a5c22f5c752d)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.970133] I [fuse-bridge.c:4348:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x3fff901124c0) to newfd (0x3fff886fd3e4) (inode-gfid:73847da7-c388-4602-9361-c2390d95a744)(old-subvolume:meta-autoload-2 new-subvolume:meta-autoload-4)
[2016-03-22 12:25:24.970921] I [MSGID: 114021] [client.c:2127:notify] 2-c_glusterfs-client-417: current graph is no longer active, destroying rpc_client 
[2016-03-22 12:25:24.971063] I [MSGID: 114018] [client.c:2042:client_rpc_notify] 2-c_glusterfs-client-417: disconnected from c_glusterfs-client-417. Client process will keep trying to connect to glusterd until brick's port is available
[2016-03-22 12:25:24.991543] W [MSGID: 108008] [afr-self-heal-name.c:359:afr_selfheal_name_gfid_mismatch_check] 4-c_glusterfs-replicate-0: GFID mismatch for <gfid:90643798-3dc6-4ae3-ac34-0993e14d14e2>/CELLO_SECURITYEVENT_LOG.xml 1609541d-60c0-4199-ac52-78c8d21cf9a4 on c_glusterfs-client-418 and 9e927ed6-f456-4ec1-9724-c86e17bd028a on c_glusterfs-client-417
[2016-03-22 12:25:24.992468] W [fuse-bridge.c:462:fuse_entry_cbk] 0-glusterfs-fuse: 3059: LOOKUP() /logfiles/security/CELLO_SECURITYEVENT_LOG.xml => -1 (Input/output error)
[2016-03-22 12:25:25.000089] W [fuse-bridge.c:462:fuse_entry_cbk] 0-glusterfs-fuse: 3066: LOOKUP() /logfiles/security/CELLO_SECURITYEVENT_LOG.xml => -1 (Input/output error)
[2016-03-22 12:25:25.008339] W [fuse-bridge.c:462:fuse_entry_cbk] 0-glusterfs-fuse: 3067: LOOKUP() /logfiles/security/CELLO_SECURITYEVENT_LOG.xml => -1 (Input/output error)
[2016-03-22 12:25:37.728818] W [fuse-bridge.c:462:fuse_entry_cbk] 0-glusterfs-fuse: 3640: LOOKUP() /logfiles/security/CELLO_SECURITYEVENT_LOG.xml => -1 (Input/output error)
The message "W [MSGID: 108008] [afr-self-heal-name.c:359:afr_selfheal_name_gfid_mismatch_check] 4-c_glusterfs-replicate-0: GFID mismatch for <gfid:90643798-3dc6-4ae3-ac34-0993e14d14e2>/CELLO_SECURITYEVENT_LOG.xml 1609541d-60c0-4199-ac52-78c8d21cf9a4 on c_glusterfs-client-418 and 9e927ed6-f456-4ec1-9724-c86e17bd028a on c_glusterfs-client-417" repeated 3 times between [2016-03-22 12:25:24.991543] and [2016-03-22 12:25:37.728008]
[2016-03-22 12:25:44.620728] I [MSGID: 108026] [afr-self-heal-common.c:651:afr_log_selfheal] 4-c_glusterfs-replicate-0: Completed data selfheal on 9e927ed6-f456-4ec1-9724-c86e17bd028a. source=0 sinks=1 
[2016-03-22 12:25:44.623126] I [MSGID: 108026] [afr-self-heal-metadata.c:56:__afr_selfheal_metadata_do] 4-c_glusterfs-replicate-0: performing metadata selfheal on 9e927ed6-f456-4ec1-9724-c86e17bd028a
[2016-03-22 12:25:44.627742] I [MSGID: 108026] [afr-self-heal-common.c:651:afr_log_selfheal] 4-c_glusterfs-replicate-0: Completed metadata selfheal on 9e927ed6-f456-4ec1-9724-c86e17bd028a. source=0 sinks=1 
[2016-03-22 12:25:57.359788] W [MSGID: 108008] [afr-self-heal-name.c:359:afr_selfheal_name_gfid_mismatch_check] 4-c_glusterfs-replicate-0: GFID mismatch for <gfid:d9cd3be0-3fa4-4d1e-8a8d-a8523535ef0a>/nameroot.ior 50f131be-29bf-49bf-a843-7d7417ea3fcc on c_glusterfs-client-418 and 087044af-daae-414a-bd60-bf528890e0d4 on c_glusterfs-client-417
[2016-03-22 12:25:57.360569] W [fuse-bridge.c:462:fuse_entry_cbk] 0-glusterfs-fuse: 5818: LOOKUP() /public_html/cello/ior_files/nameroot.ior => -1 (Input/output error)
[2016-03-22 12:26:03.608607] I [MSGID: 108026] [afr-self-heal-common.c:651:afr_log_selfheal] 4-c_glusterfs-replicate-0: Completed data selfheal on 087044af-daae-414a-bd60-bf528890e0d4. source=0 sinks=1 
[2016-03-22 12:26:03.610891] I [MSGID: 108026] [afr-self-heal-metadata.c:56:__afr_selfheal_metadata_do] 4-c_glusterfs-replicate-0: performing metadata selfheal on 087044af-daae-414a-bd60-bf528890e0d4
[2016-03-22 12:26:03.615325] I [MSGID: 108026] [afr-self-heal-common.c:651:afr_log_selfheal] 4-c_glusterfs-replicate-0: Completed metadata selfheal on 087044af-daae-414a-bd60-bf528890e0d4. source=0 sinks=1 
[2016-03-22 12:26:03.867425] W [MSGID: 108008] [afr-self-heal-name.c:359:afr_selfheal_name_gfid_mismatch_check] 4-c_glusterfs-replicate-0: GFID mismatch for <gfid:69eee88c-ddb6-4c27-b990-48123191b112>/corbasecurity 0e689d54-2822-4620-afe2-0109d4c45efc on c_glusterfs-client-418 and 288fe1cf-e5b0-4c40-813d-52d652afd4f2 on c_glusterfs-client-417
[2016-03-22 12:26:03.868290] W [fuse-bridge.c:462:fuse_entry_cbk] 0-glusterfs-fuse: 5905: LOOKUP() /security/corbasecurity => -1 (Input/output error)
[2016-03-22 12:26:09.283448] I [MSGID: 108026] [afr-self-heal-common.c:651:afr_log_selfheal] 4-c_glusterfs-replicate-0: Completed data selfheal on 288fe1cf-e5b0-4c40-813d-52d652afd4f2. source=0 sinks=1 
[2016-03-22 12:26:09.285648] I [MSGID: 108026] [afr-self-heal-metadata.c:56:__afr_selfheal_metadata_do] 4-c_glusterfs-replicate-0: performing metadata selfheal on 288fe1cf-e5b0-4c40-813d-52d652afd4f2
[2016-03-22 12:26:09.290060] I [MSGID: 108026] [afr-self-heal-common.c:651:afr_log_selfheal] 4-c_glusterfs-replicate-0: Completed metadata selfheal on 288fe1cf-e5b0-4c40-813d-52d652afd4f2. source=0 sinks=1 
[2016-03-22 12:26:10.159558] W [MSGID: 114031] [client-rpc-fops.c:2971:client3_3_lookup_cbk] 4-c_glusterfs-client-418: remote operation failed. Path: /logfiles/security/X912244659_CELLO_SECURITYEVENT_LOG_20160322T122523.push (1609541d-60c0-4199-ac52-78c8d21cf9a4) [No such file or directory]
[2016-03-22 12:26:10.161950] W [MSGID: 114031] [client-rpc-fops.c:2971:client3_3_lookup_cbk] 4-c_glusterfs-client-418: remote operation failed. Path: <gfid:1609541d-60c0-4199-ac52-78c8d21cf9a4> (1609541d-60c0-4199-ac52-78c8d21cf9a4) [No such file or directory]
[2016-03-22 12:26:10.163057] W [MSGID: 114031] [client-rpc-fops.c:2971:client3_3_lookup_cbk] 4-c_glusterfs-client-418: remote operation failed. Path: /logfiles/security/X912244659_CELLO_SECURITYEVENT_LOG_20160322T122523.push (1609541d-60c0-4199-ac52-78c8d21cf9a4) [No such file or directory]
[2016-03-22 12:25:00.361787] W [MSGID: 114031] [client-rpc-fops.c:1845:client3_3_xattrop_cbk] 0-c_glusterfs-client-417: remote operation failed. Path: /configuration/oamrdncy.cfg (df00d265-4b67-415d-95a7-f9e4cac83029) [No space left on device]
[2016-03-22 12:25:00.387858] W [MSGID: 114031] [client-rpc-fops.c:1914:client3_3_fxattrop_cbk] 0-c_glusterfs-client-417: remote operation failed [No space left on device]
$ 

EVOA_8300-10_6B> lhsh 002500 ls /var/log/glusterfs/

160322-13:32:24 10.67.29.90 16.0c CPP_MOM-CPP-LSV184-gen1_gen1_COMPLETE stopfile=/tmp/13910
$ lhsh 002500 ls /var/log/glusterfs/
bricks
cli.log
cmd_history.log
etc-glusterfs-glusterd.vol.log
glfsheal-c_glusterfs.log
glustershd.log
mnt-c.log
$ 

Log close: 160322-133249 - /home/uabbiha/SMX/Integration-Step3/HU57862/RGUST16/glusterfs-22Mar.txt
