From ezepsuarez at gmail.com Wed May 11 01:55:16 2022 From: ezepsuarez at gmail.com (Ezequiel Suarez) Date: Tue, 10 May 2022 22:55:16 -0300 Subject: [Gluster-infra] failed (No route to host) disconnecting socket Message-ID: I have openshift 3.11 with 4 gluster nodes (glusterfs 6.0). A few days ago users reported problems with pv (disconnection). I see the log in one of the gluster nodes and I find these messages. [2022-05-11 01:52:17.684622] I [rpc-clnt.c:2013:rpc_clnt_reconfig] 0-vol_f2581329a8e5ef03cac6c6fa92cf6396-client-1: changing port to 49254 (from 0) [2022-05-11 01:52:17.687888] I [rpc-clnt.c:2013:rpc_clnt_reconfig] 0-vol_f6324c9528f1761a26f49d4dbf3cb568-client-2: changing port to 49256 (from 0) [2022-05-11 01:52:17.694184] I [rpc-clnt.c:2013:rpc_clnt_reconfig] 0-vol_eaff3cce15e6c814172440c72692b915-client-5: changing port to 49253 (from 0) [2022-05-11 01:52:18.671410] E [socket.c:2376:socket_connect_finish] 0-vol_eaff3cce15e6c814172440c72692b915-client-2: connection to 10.254.7.214:49253 failed (No route to host); disconnecting socket [2022-05-11 01:52:20.699985] E [socket.c:2376:socket_connect_finish] 0-vol_f2581329a8e5ef03cac6c6fa92cf6396-client-1: connection to 10.254.7.214:49254 failed (No route to host); disconnecting socket [2022-05-11 01:52:20.701326] E [socket.c:2376:socket_connect_finish] 0-vol_f6324c9528f1761a26f49d4dbf3cb568-client-2: connection to 10.254.7.214:49256 failed (No route to host); disconnecting socket [2022-05-11 01:52:21.097323] I [rpc-clnt.c:2013:rpc_clnt_reconfig] 0-vol_eaff3cce15e6c814172440c72692b915-client-2: changing port to 49253 (from 0) [2022-05-11 01:52:21.103126] I [rpc-clnt.c:2013:rpc_clnt_reconfig] 0-vol_e8b901f78c0e84f3c422887068dfccd6-client-0: changing port to 49252 (from 0) [2022-05-11 01:52:21.104614] E [socket.c:2376:socket_connect_finish] 0-vol_eaff3cce15e6c814172440c72692b915-client-2: connection to 10.254.7.214:49253 failed (No route to host); disconnecting socket [2022-05-11 01:52:21.109317] I [rpc-clnt.c:2013:rpc_clnt_reconfig] 0-vol_f2581329a8e5ef03cac6c6fa92cf6396-client-1: changing port to 49254 (from 0) [2022-05-11 01:52:21.122744] I [rpc-clnt.c:2013:rpc_clnt_reconfig] 0-vol_f6324c9528f1761a26f49d4dbf3cb568-client-2: changing port to 49256 (from 0) [2022-05-11 01:52:22.123479] E [socket.c:2376:socket_connect_finish] 0-vol_f2581329a8e5ef03cac6c6fa92cf6396-client-1: connection to 10.254.7.214:49254 failed (No route to host); disconnecting socket We have tried all the gluster commands to check the status of the cluster and volumes and found no errors. Can someone help me or give me an idea of where to look? Thanks Ezequiel Suarez