[Gluster-users] Strange Logs

Christian Reiss email at christian-reiss.de
Fri Feb 14 11:10:28 UTC 2020


Hey folks,

my logs are constantly (every few secs, continuously) swamped with

[2020-02-14 11:05:20.258542] I [MSGID: 114046] 
[client-handshake.c:1106:client_setvolume_cbk] 0-ssd_storage-client-0: 
Connected to ssd_storage-client-0, attached to remote volume 
'/gluster_bricks/node01.company.com/gluster'.
[2020-02-14 11:05:20.258559] I [MSGID: 108005] 
[afr-common.c:5280:__afr_handle_child_up_event] 
0-ssd_storage-replicate-0: Subvolume 'ssd_storage-client-0' came back 
up; going online.
[2020-02-14 11:05:20.258920] I [rpc-clnt.c:1963:rpc_clnt_reconfig] 
0-ssd_storage-client-2: changing port to 49152 (from 0)
[2020-02-14 11:05:20.259132] I [socket.c:864:__socket_shutdown] 
0-ssd_storage-client-2: intentional socket shutdown(11)
[2020-02-14 11:05:20.260010] I [MSGID: 114057] 
[client-handshake.c:1376:select_server_supported_programs] 
0-ssd_storage-client-1: Using Program GlusterFS 4.x v1, Num (1298437), 
Version (400)
[2020-02-14 11:05:20.261077] I [MSGID: 114046] 
[client-handshake.c:1106:client_setvolume_cbk] 0-ssd_storage-client-1: 
Connected to ssd_storage-client-1, attached to remote volume 
'/gluster_bricks/node02.company.com/gluster'.
[2020-02-14 11:05:20.261089] I [MSGID: 108002] 
[afr-common.c:5647:afr_notify] 0-ssd_storage-replicate-0: Client-quorum 
is met
[2020-02-14 11:05:20.262005] I [MSGID: 114057] 
[client-handshake.c:1376:select_server_supported_programs] 
0-ssd_storage-client-2: Using Program GlusterFS 4.x v1, Num (1298437), 
Version (400)
[2020-02-14 11:05:20.262685] I [MSGID: 114046] 
[client-handshake.c:1106:client_setvolume_cbk] 0-ssd_storage-client-2: 
Connected to ssd_storage-client-2, attached to remote volume 
'/gluster_bricks/node03.company.com/gluster'.
[2020-02-14 11:05:20.263909] I [MSGID: 108031] 
[afr-common.c:2580:afr_local_discovery_cbk] 0-ssd_storage-replicate-0: 
selecting local read_child ssd_storage-client-0
[2020-02-14 11:05:20.264124] I [MSGID: 104041] 
[glfs-resolve.c:954:__glfs_active_subvol] 0-ssd_storage: switched to 
graph 6e6f6465-3031-2e64-632d-6475732e6461 (0)
  [2020-02-14 11:05:22.407851] I [MSGID: 114007] 
[client.c:2478:client_check_remote_host] 0-ssd_storage-snapd-client: 
Remote host is not set. Assuming the volfile server as remote host 
[Invalid argument]
[2020-02-14 11:05:22.409711] I [MSGID: 104045] [glfs-master.c:80:notify] 
0-gfapi: New graph 6e6f6465-3031-2e64-632d-6475732e6461 (0) coming up
[2020-02-14 11:05:22.409738] I [MSGID: 114020] [client.c:2436:notify] 
0-ssd_storage-client-0: parent translators are ready, attempting connect 
on transport
[2020-02-14 11:05:22.412949] I [MSGID: 114020] [client.c:2436:notify] 
0-ssd_storage-client-1: parent translators are ready, attempting connect 
on transport
[2020-02-14 11:05:22.413130] I [rpc-clnt.c:1963:rpc_clnt_reconfig] 
0-ssd_storage-client-0: changing port to 49152 (from 0)
[2020-02-14 11:05:22.413154] I [socket.c:864:__socket_shutdown] 
0-ssd_storage-client-0: intentional socket shutdown(10)
[2020-02-14 11:05:22.415534] I [MSGID: 114020] [client.c:2436:notify] 
0-ssd_storage-client-2: parent translators are ready, attempting connect 
on transport
[2020-02-14 11:05:22.417836] I [MSGID: 114057] 
[client-handshake.c:1376:select_server_supported_programs] 
0-ssd_storage-client-0: Using Program GlusterFS 4.x v1, Num (1298437), 
Version (400)
[2020-02-14 11:05:22.418036] I [rpc-clnt.c:1963:rpc_clnt_reconfig] 
0-ssd_storage-client-1: changing port to 49152 (from 0)
[2020-02-14 11:05:22.418095] I [socket.c:864:__socket_shutdown] 
0-ssd_storage-client-1: intentional socket shutdown(12)
[2020-02-14 11:05:22.420029] I [MSGID: 114020] [client.c:2436:notify] 
0-ssd_storage-snapd-client: parent translators are ready, attempting 
connect on transport
[2020-02-14 11:05:22.420533] E [MSGID: 101075] 
[common-utils.c:505:gf_resolve_ip6] 0-resolver: getaddrinfo failed 
(family:2) (Name or service not known)
[2020-02-14 11:05:22.420545] E 
[name.c:266:af_inet_client_get_remote_sockaddr] 
0-ssd_storage-snapd-client: DNS resolution failed on host 
/var/run/glusterd.socket
Final graph:
+------------------------------------------------------------------------------+
   1: volume ssd_storage-client-0
   2:     type protocol/client
   3:     option opversion 70000
   4:     option clnt-lk-version 1
   5:     option volfile-checksum 0
   6:     option volfile-key ssd_storage
   7:     option client-version 7.0
   8:     option process-name gfapi.glfsheal
   9:     option process-uuid 
CTX_ID:50cec79e-6028-4e6f-b8ed-dda9db36b2d0-GRAPH_ID:0-PID:24926-HOST:node01.company.com-PC_NAME:ssd_storage-client-0-RECON_NO:-0
  10:     option fops-version 1298437
  11:     option ping-timeout 42
  12:     option remote-host node01.company.com
  13:     option remote-subvolume /gluster_bricks/node01.company.com/gluster
  14:     option transport-type socket
  15:     option transport.address-family inet
  16:     option username 96bcf4d4-932f-4654-86c3-470a081d5021
  17:     option password 069e7ee9-b17d-4228-a612-b0f33588a9ec
  18:     option transport.socket.ssl-enabled off
  19:     option transport.tcp-user-timeout 0
  20:     option transport.socket.keepalive-time 20
  21:     option transport.socket.keepalive-interval 2
  22:     option transport.socket.keepalive-count 9
  23:     option send-gids true
  24: end-volume
  25:
  26: volume ssd_storage-client-1
  27:     type protocol/client
  28:     option ping-timeout 42
  29:     option remote-host node02.company.com
  30:     option remote-subvolume /gluster_bricks/node02.company.com/gluster
  31:     option transport-type socket
  32:     option transport.address-family inet
  33:     option username 96bcf4d4-932f-4654-86c3-470a081d5021
  34:     option password 069e7ee9-b17d-4228-a612-b0f33588a9ec
  35:     option transport.socket.ssl-enabled off
  36:     option transport.tcp-user-timeout 0
  37:     option transport.socket.keepalive-time 20
  38:     option transport.socket.keepalive-interval 2
  39:     option transport.socket.keepalive-count 9
  40:     option send-gids true
  41: end-volume
  42:
  43: volume ssd_storage-client-2
  44:     type protocol/client
  45:     option ping-timeout 42
  46:     option remote-host node03.company.com
  47:     option remote-subvolume /gluster_bricks/node03.company.com/gluster
  48:     option transport-type socket
  49:     option transport.address-family inet
  50:     option username 96bcf4d4-932f-4654-86c3-470a081d5021
  51:     option password 069e7ee9-b17d-4228-a612-b0f33588a9ec
  52:     option transport.socket.ssl-enabled off
  53:     option transport.tcp-user-timeout 0
  54:     option transport.socket.keepalive-time 20
  55:     option transport.socket.keepalive-interval 2
  56:     option transport.socket.keepalive-count 9
  57:     option send-gids true
  58: end-volume
  59:
  60: volume ssd_storage-replicate-0
  61:     type cluster/replicate
  62:     option background-self-heal-count 0
  63:     option afr-pending-xattr 
ssd_storage-client-0,ssd_storage-client-1,ssd_storage-client-2
  64:     option metadata-self-heal on
  65:     option data-self-heal on
  66:     option entry-self-heal on
  67:     option data-self-heal-algorithm full
  68:     option use-compound-fops off
  69:     subvolumes ssd_storage-client-0 ssd_storage-client-1 
ssd_storage-client-2
  70: end-volume
  71:
  72: volume ssd_storage-dht
  73:     type cluster/distribute
  74:     option readdir-optimize on
  75:     option lock-migration off
  76:     option force-migration off
  77:     subvolumes ssd_storage-replicate-0
  78: end-volume
  79:
  80: volume ssd_storage-utime
  81:     type features/utime
  82:     option noatime on
  83:     subvolumes ssd_storage-dht
  84: end-volume
  85:
  86: volume ssd_storage-write-behind
  87:     type performance/write-behind
  88:     subvolumes ssd_storage-utime
  89: end-volume
  90:
  91: volume ssd_storage-read-ahead
  92:     type performance/read-ahead
  93:     subvolumes ssd_storage-write-behind
  94: end-volume
  95:
  96: volume ssd_storage-readdir-ahead
  97:     type performance/readdir-ahead
  98:     option parallel-readdir off
  99:     option rda-request-size 131072
100:     option rda-cache-limit 10MB
101:     subvolumes ssd_storage-read-ahead
102: end-volume
103:
104: volume ssd_storage-io-cache
105:     type performance/io-cache
106:     subvolumes ssd_storage-readdir-ahead
107: end-volume
108:
109: volume ssd_storage-open-behind
110:     type performance/open-behind
111:     subvolumes ssd_storage-io-cache
112: end-volume
113:
114: volume ssd_storage-quick-read
115:     type performance/quick-read
116:     subvolumes ssd_storage-open-behind
117: end-volume
118:
119: volume ssd_storage-md-cache
120:     type performance/md-cache
121:     subvolumes ssd_storage-quick-read
122: end-volume
123:
124: volume ssd_storage-snapd-client
125:     type protocol/client
126:     option remote-host /var/run/glusterd.socket
127:     option ping-timeout 42
128:     option remote-subvolume snapd-ssd_storage
129:     option transport-type socket
130:     option transport.address-family inet
131:     option username 96bcf4d4-932f-4654-86c3-470a081d5021
132:     option password 069e7ee9-b17d-4228-a612-b0f33588a9ec
133:     option transport.socket.ssl-enabled off
134:     option transport.tcp-user-timeout 0
135:     option transport.socket.keepalive-time 20
136:     option transport.socket.keepalive-interval 2
137:     option transport.socket.keepalive-count 9
138:     option send-gids true
139: end-volume
140:
141: volume ssd_storage-snapview-client
142:     type features/snapview-client
143:     option snapshot-directory .snaps
144:     option show-snapshot-directory on
145:     subvolumes ssd_storage-md-cache ssd_storage-snapd-client
146: end-volume
147:
148: volume ssd_storage
149:     type debug/io-stats
150:     option log-level INFO
151:     option threads 16
152:     option latency-measurement off
153:     option count-fop-hits off
154:     option global-threading off
155:     subvolumes ssd_storage-snapview-client
156: end-volume
157:
158: volume meta-autoload
159:     type meta
160:     subvolumes ssd_storage
161: end-volume
162:
+------------------------------------------------------------------------------+
[2020-02-14 11:05:22.421366] I [MSGID: 114046] 
[client-handshake.c:1106:client_setvolume_cbk] 0-ssd_storage-client-0: 
Connected to ssd_storage-client-0, attached to remote volume 
'/gluster_bricks/node01.company.com/gluster'.
[2020-02-14 11:05:22.421379] I [MSGID: 108005] 
[afr-common.c:5280:__afr_handle_child_up_event] 
0-ssd_storage-replicate-0: Subvolume 'ssd_storage-client-0' came back 
up; going online.
[2020-02-14 11:05:22.421669] I [rpc-clnt.c:1963:rpc_clnt_reconfig] 
0-ssd_storage-client-2: changing port to 49152 (from 0)
[2020-02-14 11:05:22.421686] I [socket.c:864:__socket_shutdown] 
0-ssd_storage-client-2: intentional socket shutdown(11)
[2020-02-14 11:05:22.422460] I [MSGID: 114057] 
[client-handshake.c:1376:select_server_supported_programs] 
0-ssd_storage-client-1: Using Program GlusterFS 4.x v1, Num (1298437), 
Version (400)
[2020-02-14 11:05:22.423377] I [MSGID: 114046] 
[client-handshake.c:1106:client_setvolume_cbk] 0-ssd_storage-client-1: 
Connected to ssd_storage-client-1, attached to remote volume 
'/gluster_bricks/node02.company.com/gluster'.
[2020-02-14 11:05:22.423391] I [MSGID: 108002] 
[afr-common.c:5647:afr_notify] 0-ssd_storage-replicate-0: Client-quorum 
is met
[2020-02-14 11:05:22.424586] I [MSGID: 114057] 
[client-handshake.c:1376:select_server_supported_programs] 
0-ssd_storage-client-2: Using Program GlusterFS 4.x v1, Num (1298437), 
Version (400)
[2020-02-14 11:05:22.425323] I [MSGID: 114046] 
[client-handshake.c:1106:client_setvolume_cbk] 0-ssd_storage-client-2: 
Connected to ssd_storage-client-2, attached to remote volume 
'/gluster_bricks/node03.company.com/gluster'.
[2020-02-14 11:05:22.426613] I [MSGID: 108031] 
[afr-common.c:2580:afr_local_discovery_cbk] 0-ssd_storage-replicate-0: 
selecting local read_child ssd_storage-client-0
[2020-02-14 11:05:22.426758] I [MSGID: 104041] 
[glfs-resolve.c:954:__glfs_active_subvol] 0-ssd_storage: switched to 
graph 6e6f6465-3031-2e64-632d-6475732e6461 (0)


Can you guys make any sense out of this? 5 unsynced entries remain.

-- 
with kind regards,
mit freundlichen Gruessen,

Christian Reiss



More information about the Gluster-users mailing list