<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <p>I don't see a "Connected to gvol0-client-1" in the log.  Perhaps
      a firewall issue like the last time? Even in the earlier add-brick
      log from the other email thread, connection to the 2nd brick was
      not established.</p>
    <p>-Ravi<br>
    </p>
    <div class="moz-cite-prefix">On 29/05/19 2:26 PM, David Cunningham
      wrote:<br>
    </div>
    <blockquote type="cite"
cite="mid:CAHGbP-8vAC4+WLObZbXPtmMG+rt-WsqLr3gt=c2yPZAZMWKgMg@mail.gmail.com">
      <meta http-equiv="content-type" content="text/html; charset=UTF-8">
      <div dir="ltr">
        <div dir="ltr">
          <div dir="ltr">
            <div dir="ltr">
              <div>Hi Ravi and Joe,</div>
              <div><br>
              </div>
              <div>The command "gluster volume status gvol0" shows all 3
                nodes as being online, even on gfs3 as below. I've
                attached the glfsheal-gvol0.log, in which I can't see
                anything like a connection error. Would you have any
                further suggestions? Thank you.</div>
              <div><br>
              </div>
              <div>[root@gfs3 glusterfs]# gluster volume status gvol0<br>
                Status of volume: gvol0<br>
                Gluster process                             TCP Port 
                RDMA Port  Online  Pid<br>
------------------------------------------------------------------------------<br>
                Brick gfs1:/nodirectwritedata/gluster/gvol0 49152    
                0          Y       7706 <br>
                Brick gfs2:/nodirectwritedata/gluster/gvol0 49152    
                0          Y       7625 <br>
                Brick gfs3:/nodirectwritedata/gluster/gvol0 49152    
                0          Y       7307 <br>
                Self-heal Daemon on localhost               N/A      
                N/A        Y       7316 <br>
                Self-heal Daemon on gfs1                    N/A      
                N/A        Y       40591<br>
                Self-heal Daemon on gfs2                    N/A      
                N/A        Y       7634 <br>
                 <br>
                Task Status of Volume gvol0<br>
------------------------------------------------------------------------------<br>
                There are no active volume tasks<br>
                <br>
              </div>
            </div>
          </div>
        </div>
      </div>
      <br>
      <div class="gmail_quote">
        <div dir="ltr" class="gmail_attr">On Wed, 29 May 2019 at 16:26,
          Ravishankar N &lt;<a href="mailto:ravishankar@redhat.com"
            moz-do-not-send="true">ravishankar@redhat.com</a>&gt; wrote:<br>
        </div>
        <blockquote class="gmail_quote" style="margin:0px 0px 0px
          0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
          <div bgcolor="#FFFFFF">
            <p><br>
            </p>
            <div class="gmail-m_356646914622959683moz-cite-prefix">On
              29/05/19 6:21 AM, David Cunningham wrote:<br>
            </div>
            <blockquote type="cite">
              <div dir="ltr">
                <div dir="ltr">
                  <div dir="ltr">
                    <div dir="ltr">
                      <div dir="ltr">
                        <div dir="ltr">
                          <div>Hello all,</div>
                          <div><br>
                          </div>
                          <div>We are seeing a strange issue where a new
                            node gfs3 shows another node gfs2 as not
                            connected on the "gluster volume heal" info:</div>
                          <div><br>
                          </div>
                          <div>[root@gfs3 bricks]# gluster volume heal
                            gvol0 info<br>
                            Brick gfs1:/nodirectwritedata/gluster/gvol0<br>
                            Status: Connected<br>
                            Number of entries: 0<br>
                            <br>
                            Brick gfs2:/nodirectwritedata/gluster/gvol0<br>
                            Status: Transport endpoint is not connected<br>
                            Number of entries: -<br>
                            <br>
                            Brick gfs3:/nodirectwritedata/gluster/gvol0<br>
                            Status: Connected<br>
                            Number of entries: 0<br>
                          </div>
                          <div><br>
                          </div>
                          <div><br>
                          </div>
                          <div>However it does show the same node
                            connected on "gluster peer status". Does
                            anyone know why this would be?</div>
                          <div><br>
                          </div>
                          <div>[root@gfs3 bricks]# gluster peer status<br>
                            Number of Peers: 2<br>
                            <br>
                            Hostname: gfs2<br>
                            Uuid: 91863102-23a8-43e1-b3d3-f0a1bd57f350<br>
                            State: Peer in Cluster (Connected)<br>
                            <br>
                            Hostname: gfs1<br>
                            Uuid: 32c99e7d-71f2-421c-86fc-b87c0f68ad1b<br>
                            State: Peer in Cluster (Connected)<br>
                          </div>
                          <div><br>
                          </div>
                          <div><br>
                          </div>
                          <div>In nodirectwritedata-gluster-gvol0.log on
                            gfs3 we see this logged with regards to
                            gfs2:</div>
                        </div>
                      </div>
                    </div>
                  </div>
                </div>
              </div>
            </blockquote>
            <p>You need to check glfsheal-$volname.log on the node where
              you ran the command and check for any connection related
              errors.</p>
            <p>-Ravi<br>
            </p>
            <blockquote type="cite">
              <div dir="ltr">
                <div dir="ltr">
                  <div dir="ltr">
                    <div dir="ltr">
                      <div dir="ltr">
                        <div dir="ltr">
                          <div><br>
                          </div>
                          <div>[2019-05-29 00:17:50.646360] I [MSGID:
                            115029]
                            [server-handshake.c:537:server_setvolume]
                            0-gvol0-server: accepted client from
CTX_ID:30d74196-fece-4380-adc0-338760188b81-GRAPH_ID:0-PID:7718-HOST:gfs2.xxx.com-PC_NAME:gvol0-client-2-RECON_NO:-0
                            (version: 5.6)<br>
                            [2019-05-29 00:17:50.761120] I [MSGID:
                            115036] [server.c:469:server_rpc_notify]
                            0-gvol0-server: disconnecting connection
                            from
CTX_ID:30d74196-fece-4380-adc0-338760188b81-GRAPH_ID:0-PID:7718-HOST:gfs2.xxx.com-PC_NAME:gvol0-client-2-RECON_NO:-0<br>
                            [2019-05-29 00:17:50.761352] I [MSGID:
                            101055] [client_t.c:435:gf_client_unref]
                            0-gvol0-server: Shutting down connection
CTX_ID:30d74196-fece-4380-adc0-338760188b81-GRAPH_ID:0-PID:7718-HOST:gfs2.xxx.com-PC_NAME:gvol0-client-2-RECON_NO:-0<br>
                          </div>
                          <div><br>
                          </div>
                          <div>Thanks in advance for any assistance.<br>
                          </div>
                          <div><br>
                            -- <br>
                            <div dir="ltr"
                              class="gmail-m_356646914622959683gmail_signature">
                              <div dir="ltr">
                                <div>
                                  <div dir="ltr">
                                    <div>
                                      <div dir="ltr">
                                        <div>
                                          <div dir="ltr">
                                            <div>
                                              <div dir="ltr">
                                                <div>David Cunningham,
                                                  Voisonics Limited<br>
                                                  <a
                                                    href="http://voisonics.com/"
                                                    target="_blank"
                                                    moz-do-not-send="true">http://voisonics.com/</a><br>
                                                  USA: +1 213 221 1092<br>
                                                  New Zealand: +64 (0)28
                                                  2558 3782</div>
                                              </div>
                                            </div>
                                          </div>
                                        </div>
                                      </div>
                                    </div>
                                  </div>
                                </div>
                              </div>
                            </div>
                          </div>
                        </div>
                      </div>
                    </div>
                  </div>
                </div>
              </div>
              <br>
              <fieldset
                class="gmail-m_356646914622959683mimeAttachmentHeader"></fieldset>
              <pre class="gmail-m_356646914622959683moz-quote-pre">_______________________________________________
Gluster-users mailing list
<a class="gmail-m_356646914622959683moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org" target="_blank" moz-do-not-send="true">Gluster-users@gluster.org</a>
<a class="gmail-m_356646914622959683moz-txt-link-freetext" href="https://lists.gluster.org/mailman/listinfo/gluster-users" target="_blank" moz-do-not-send="true">https://lists.gluster.org/mailman/listinfo/gluster-users</a></pre>
            </blockquote>
          </div>
        </blockquote>
      </div>
      <br clear="all">
      <br>
      -- <br>
      <div dir="ltr" class="gmail_signature">
        <div dir="ltr">
          <div>
            <div dir="ltr">
              <div>
                <div dir="ltr">
                  <div>
                    <div dir="ltr">
                      <div>
                        <div dir="ltr">
                          <div>David Cunningham, Voisonics Limited<br>
                            <a href="http://voisonics.com/"
                              target="_blank" moz-do-not-send="true">http://voisonics.com/</a><br>
                            USA: +1 213 221 1092<br>
                            New Zealand: +64 (0)28 2558 3782</div>
                        </div>
                      </div>
                    </div>
                  </div>
                </div>
              </div>
            </div>
          </div>
        </div>
      </div>
    </blockquote>
  </body>
</html>