<div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div>Hi Ravi and Joe,</div><div><br></div><div>The command "gluster volume status gvol0" shows all 3 nodes as being online, even on gfs3 as below. I've attached the glfsheal-gvol0.log, in which I can't see anything like a connection error. Would you have any further suggestions? Thank you.</div><div><br></div><div>[root@gfs3 glusterfs]# gluster volume status gvol0<br>Status of volume: gvol0<br>Gluster process TCP Port RDMA Port Online Pid<br>------------------------------------------------------------------------------<br>Brick gfs1:/nodirectwritedata/gluster/gvol0 49152 0 Y 7706 <br>Brick gfs2:/nodirectwritedata/gluster/gvol0 49152 0 Y 7625 <br>Brick gfs3:/nodirectwritedata/gluster/gvol0 49152 0 Y 7307 <br>Self-heal Daemon on localhost N/A N/A Y 7316 <br>Self-heal Daemon on gfs1 N/A N/A Y 40591<br>Self-heal Daemon on gfs2 N/A N/A Y 7634 <br> <br>Task Status of Volume gvol0<br>------------------------------------------------------------------------------<br>There are no active volume tasks<br><br></div></div></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, 29 May 2019 at 16:26, Ravishankar N <<a href="mailto:ravishankar@redhat.com">ravishankar@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p><br>
</p>
<div class="gmail-m_356646914622959683moz-cite-prefix">On 29/05/19 6:21 AM, David Cunningham
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div>Hello all,</div>
<div><br>
</div>
<div>We are seeing a strange issue where a new node
gfs3 shows another node gfs2 as not connected on the
"gluster volume heal" info:</div>
<div><br>
</div>
<div>[root@gfs3 bricks]# gluster volume heal gvol0
info<br>
Brick gfs1:/nodirectwritedata/gluster/gvol0<br>
Status: Connected<br>
Number of entries: 0<br>
<br>
Brick gfs2:/nodirectwritedata/gluster/gvol0<br>
Status: Transport endpoint is not connected<br>
Number of entries: -<br>
<br>
Brick gfs3:/nodirectwritedata/gluster/gvol0<br>
Status: Connected<br>
Number of entries: 0<br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>However it does show the same node connected on
"gluster peer status". Does anyone know why this
would be?</div>
<div><br>
</div>
<div>[root@gfs3 bricks]# gluster peer status<br>
Number of Peers: 2<br>
<br>
Hostname: gfs2<br>
Uuid: 91863102-23a8-43e1-b3d3-f0a1bd57f350<br>
State: Peer in Cluster (Connected)<br>
<br>
Hostname: gfs1<br>
Uuid: 32c99e7d-71f2-421c-86fc-b87c0f68ad1b<br>
State: Peer in Cluster (Connected)<br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>In nodirectwritedata-gluster-gvol0.log on gfs3 we
see this logged with regards to gfs2:</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<p>You need to check glfsheal-$volname.log on the node where you ran
the command and check for any connection related errors.</p>
<p>-Ravi<br>
</p>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div><br>
</div>
<div>[2019-05-29 00:17:50.646360] I [MSGID: 115029]
[server-handshake.c:537:server_setvolume]
0-gvol0-server: accepted client from
CTX_ID:30d74196-fece-4380-adc0-338760188b81-GRAPH_ID:0-PID:7718-HOST:gfs2.xxx.com-PC_NAME:gvol0-client-2-RECON_NO:-0
(version: 5.6)<br>
[2019-05-29 00:17:50.761120] I [MSGID: 115036]
[server.c:469:server_rpc_notify] 0-gvol0-server:
disconnecting connection from
CTX_ID:30d74196-fece-4380-adc0-338760188b81-GRAPH_ID:0-PID:7718-HOST:gfs2.xxx.com-PC_NAME:gvol0-client-2-RECON_NO:-0<br>
[2019-05-29 00:17:50.761352] I [MSGID: 101055]
[client_t.c:435:gf_client_unref] 0-gvol0-server:
Shutting down connection
CTX_ID:30d74196-fece-4380-adc0-338760188b81-GRAPH_ID:0-PID:7718-HOST:gfs2.xxx.com-PC_NAME:gvol0-client-2-RECON_NO:-0<br>
</div>
<div><br>
</div>
<div>Thanks in advance for any assistance.<br>
</div>
<div><br>
-- <br>
<div dir="ltr" class="gmail-m_356646914622959683gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>David Cunningham, Voisonics
Limited<br>
<a href="http://voisonics.com/" target="_blank">http://voisonics.com/</a><br>
USA: +1 213 221 1092<br>
New Zealand: +64 (0)28 2558
3782</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<fieldset class="gmail-m_356646914622959683mimeAttachmentHeader"></fieldset>
<pre class="gmail-m_356646914622959683moz-quote-pre">_______________________________________________
Gluster-users mailing list
<a class="gmail-m_356646914622959683moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a>
<a class="gmail-m_356646914622959683moz-txt-link-freetext" href="https://lists.gluster.org/mailman/listinfo/gluster-users" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
</div>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>David Cunningham, Voisonics Limited<br><a href="http://voisonics.com/" target="_blank">http://voisonics.com/</a><br>USA: +1 213 221 1092<br>New Zealand: +64 (0)28 2558 3782</div></div></div></div></div></div></div></div></div></div></div>