<html>
  <head>
    <meta content="text/html; charset=windows-1252"
      http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <p>I think the problem here is by default the quorum is playing
      around here, to get rid of this you can change quorum type as
      fixed and the value as 2 , or you can disable the quorum.</p>
    <p><br>
    </p>
    <p>Regards</p>
    <p>Rafi KC<br>
    </p>
    <br>
    <div class="moz-cite-prefix">On 11/08/2017 04:03 AM, Jon Cope wrote:<br>
    </div>
    <blockquote
      cite="mid:316637224.23675959.1510093990211.JavaMail.zimbra@redhat.com"
      type="cite">
      <div style="font-family: courier
        new,courier,monaco,monospace,sans-serif; font-size: 10pt; color:
        #000000">
        <div><span style="font-family: arial, helvetica, sans-serif;">Hi
            all,</span></div>
        <div><br>
        </div>
        <div><span style="font-family: arial, helvetica, sans-serif;">I'm
            taking a stab at deploying a storage cluster to explore the
            Halo AFR feature and running into some trouble.  In GCE, I
            have 4 instances, each with one 10gb brick.  2 instances are
            in the US and the other 2 are in Asia (with the hope that it
            will drive up latency sufficiently).  The bricks make up a
            Replica-4 volume.  Before I enable halo, I can mount to
            volume and r/w files.</span></div>
        <div><br>
        </div>
        <div><span style="font-family: arial, helvetica, sans-serif;">The
            issue is when I set the `cluster.halo-enabled yes`, I can no
            longer write to the volume:</span><br>
          <br>
          [root@jcope-rhs-g2fn vol]# touch /mnt/vol/test1<br>
        </div>
        <div>touch: setting times of ‘test1’: Read-only file system</div>
        <div><br>
        </div>
        <div><span style="font-family: arial, helvetica, sans-serif;">This
            can be fixed by turning halo off again.  While halo is
            enabled and writes return the above message, the mount still
            shows it to be r/w:<br>
          </span></div>
        <div><span style="font-family: arial, helvetica, sans-serif;"><br>
          </span></div>
        <div><span style="font-family: arial, helvetica, sans-serif;"><span
              style="font-size: 13.3333px; font-family: &quot;courier
              new&quot;, courier, monaco, monospace, sans-serif;">[root@jcope-rhs-g2fn
              vol]# mount</span><br>
            <span style="font-family: &quot;courier new&quot;, courier,
              monaco, monospace, sans-serif;">gce-node1:gv0 on /mnt/vol
              type fuse.glusterfs
(rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)</span><br>
          </span></div>
        <div><span style="font-family: arial, helvetica, sans-serif;"><span
              style="font-family: &quot;courier new&quot;, courier,
              monaco, monospace, sans-serif;"><br>
            </span></span></div>
        <div><br>
        </div>
        <div><span style="font-family: arial, helvetica, sans-serif;">Thanks
            in advace,</span><br>
        </div>
        <div><span style="font-family: arial, helvetica, sans-serif;">-Jon</span></div>
        <div><br>
        </div>
        <div><span style="font-family: arial, helvetica, sans-serif;"><span
              style="font-family: &quot;courier new&quot;, courier,
              monaco, monospace, sans-serif;"><br>
            </span></span></div>
        <div>
          <div style="font-size: 13.3333px;" data-mce-style="font-size:
            13.3333px;"><span face="arial, helvetica, sans-serif"
              style="font-family: arial, helvetica, sans-serif;"
              data-mce-style="font-family: arial, helvetica,
              sans-serif;">Setup info</span></div>
          <div style="font-size: 13.3333px;" data-mce-style="font-size:
            13.3333px;"><span face="arial, helvetica, sans-serif"
              style="font-family: &quot;courier new&quot;, courier,
              monaco, monospace, sans-serif;"
              data-mce-style="font-family: 'courier new', courier,
              monaco, monospace, sans-serif;">CentOS Linux release
              7.4.1708 (Core)</span></div>
          <div style="font-size: 13.3333px;" data-mce-style="font-size:
            13.3333px;">
            <div style="font-size: 13.3333px;"
              data-mce-style="font-size: 13.3333px;"><span face="arial,
                helvetica, sans-serif">4 GCE Instances (2 US, 2 Asia)</span></div>
            <div style="font-size: 13.3333px;"
              data-mce-style="font-size: 13.3333px;"><span face="arial,
                helvetica, sans-serif">1 10gb Brick/Instance</span></div>
            <div style="font-size: 13.3333px;"
              data-mce-style="font-size: 13.3333px;"><span face="arial,
                helvetica, sans-serif">replica 4 volume</span></div>
            <div style="font-size: 13.3333px;"
              data-mce-style="font-size: 13.3333px;"><span face="arial,
                helvetica, sans-serif"><br>
              </span></div>
            <div style="font-size: 13.3333px;"
              data-mce-style="font-size: 13.3333px;"><span face="arial,
                helvetica, sans-serif" style="font-family: arial,
                helvetica, sans-serif;">Packages:</span></div>
          </div>
          <div style="font-size: 13.3333px;" data-mce-style="font-size:
            13.3333px;"><span face="arial, helvetica, sans-serif"
              style="font-family: &quot;courier new&quot;, courier,
              monaco, monospace, sans-serif;"
              data-mce-style="font-family: 'courier new', courier,
              monaco, monospace, sans-serif;"><span face="arial,
                helvetica, sans-serif"></span></span>
            <p style="margin: 0px;" data-mce-style="margin: 0px;"><span
                style="font-family: &quot;courier new&quot;, courier,
                monaco, monospace, sans-serif;"
                data-mce-style="font-family: 'courier new', courier,
                monaco, monospace, sans-serif;">glusterfs-client-xlators-3.12.1-2.el7.x86_64</span><br>
              <span style="font-family: &quot;courier new&quot;,
                courier, monaco, monospace, sans-serif;"
                data-mce-style="font-family: 'courier new', courier,
                monaco, monospace, sans-serif;">glusterfs-cli-3.12.1-2.el7.x86_64</span><br>
              <span style="font-family: &quot;courier new&quot;,
                courier, monaco, monospace, sans-serif;"
                data-mce-style="font-family: 'courier new', courier,
                monaco, monospace, sans-serif;">python2-gluster-3.12.1-2.el7.x86_64</span><br>
              <span style="font-family: &quot;courier new&quot;,
                courier, monaco, monospace, sans-serif;"
                data-mce-style="font-family: 'courier new', courier,
                monaco, monospace, sans-serif;">glusterfs-3.12.1-2.el7.x86_64</span><br>
              <span style="font-family: &quot;courier new&quot;,
                courier, monaco, monospace, sans-serif;"
                data-mce-style="font-family: 'courier new', courier,
                monaco, monospace, sans-serif;">glusterfs-api-3.12.1-2.el7.x86_64</span><br>
              <span style="font-family: &quot;courier new&quot;,
                courier, monaco, monospace, sans-serif;"
                data-mce-style="font-family: 'courier new', courier,
                monaco, monospace, sans-serif;">glusterfs-fuse-3.12.1-2.el7.x86_64</span><br>
              <span style="font-family: &quot;courier new&quot;,
                courier, monaco, monospace, sans-serif;"
                data-mce-style="font-family: 'courier new', courier,
                monaco, monospace, sans-serif;">glusterfs-server-3.12.1-2.el7.x86_64</span><br>
              <span style="font-family: &quot;courier new&quot;,
                courier, monaco, monospace, sans-serif;"
                data-mce-style="font-family: 'courier new', courier,
                monaco, monospace, sans-serif;">glusterfs-libs-3.12.1-2.el7.x86_64</span><br>
              <span style="font-family: &quot;courier new&quot;,
                courier, monaco, monospace, sans-serif;"
                data-mce-style="font-family: 'courier new', courier,
                monaco, monospace, sans-serif;">glusterfs-geo-replication-3.12.1-2.el7.x86_64</span></p>
          </div>
          <div style="font-size: 13.3333px;" data-mce-style="font-size:
            13.3333px;"><br>
          </div>
          <div style="font-size: 13.3333px;" data-mce-style="font-size:
            13.3333px;"><span style="font-family: arial, helvetica,
              sans-serif; font-size: 10pt;"> </span></div>
        </div>
        <div><span style="font-family: &quot;times new roman&quot;,
            &quot;new york&quot;, times, serif;"
            data-mce-style="font-family: 'times new roman', 'new york',
            times, serif;"><span style="font-family: &quot;times new
              roman&quot;, &quot;new york&quot;, times, serif;"
              data-mce-style="font-family: 'times new roman', 'new
              york', times, serif;"><span style="font-family: arial,
                helvetica, sans-serif;">Logs, beginning when halo is
                enabled:</span><br>
            </span></span>
          <p style="margin: 0px;" data-mce-style="margin: 0px;">[2017-11-07
            22:20:15.029298] W [MSGID: 101095]
            [xlator.c:213:xlator_dynload] 0-xlator:
            /usr/lib64/glusterfs/3.12.1/xlator/nfs/server.so: cannot
            open shared object file: No such file or directory<br>
            [2017-11-07 22:20:15.204241] W [MSGID: 101095]
            [xlator.c:162:xlator_volopt_dynload] 0-xlator:
            /usr/lib64/glusterfs/3.12.1/xlator/nfs/server.so: cannot
            open shared object file: No such file or directory<br>
            [2017-11-07 22:20:15.232176] I [MSGID: 106600]
            [glusterd-nfs-svc.c:163:glusterd_nfssvc_reconfigure]
            0-management: nfs/server.so xlator is not installed<br>
            [2017-11-07 22:20:15.235481] I [MSGID: 106132]
            [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management:
            quotad already stopped<br>
            [2017-11-07 22:20:15.235512] I [MSGID: 106568]
            [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management:
            quotad service is stopped<br>
            [2017-11-07 22:20:15.235572] I [MSGID: 106132]
            [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management:
            bitd already stopped<br>
            [2017-11-07 22:20:15.235585] I [MSGID: 106568]
            [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management:
            bitd service is stopped<br>
            [2017-11-07 22:20:15.235638] I [MSGID: 106132]
            [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management:
            scrub already stopped<br>
            [2017-11-07 22:20:15.235650] I [MSGID: 106568]
            [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management:
            scrub service is stopped<br>
            [2017-11-07 22:20:15.250297] I [run.c:190:runner_log]
            (--&gt;/usr/lib64/glusterfs/3.12.1/xlator/mgmt/glusterd.so(+0xde17a)
            [0x7fc23442117a]
            --&gt;/usr/lib64/glusterfs/3.12.1/xlator/mgmt/glusterd.so(+0xddc3d)
            [0x7fc234420c3d]
            --&gt;/lib64/libglusterfs.so.0(runner_log+0x115)
            [0x7fc23f915da5] ) 0-management: Ran script: /var/lib<br>
            /glusterd/hooks/1/set/post/S30samba-set.sh --volname=gv0 -o
            cluster.halo-enabled=yes --gd-workdir=/var/lib/glusterd<br>
            [2017-11-07 22:20:15.255777] I [run.c:190:runner_log]
            (--&gt;/usr/lib64/glusterfs/3.12.1/xlator/mgmt/glusterd.so(+0xde17a)
            [0x7fc23442117a]
            --&gt;/usr/lib64/glusterfs/3.12.1/xlator/mgmt/glusterd.so(+0xddc3d)
            [0x7fc234420c3d]
            --&gt;/lib64/libglusterfs.so.0(runner_log+0x115)
            [0x7fc23f915da5] ) 0-management: Ran script: /var/lib<br>
/glusterd/hooks/1/set/post/S32gluster_enable_shared_storage.sh
            --volname=gv0 -o cluster.halo-enabled=yes
            --gd-workdir=/var/lib/glusterd<br>
            [2017-11-07 22:20:47.420098] W [MSGID: 101095]
            [xlator.c:213:xlator_dynload] 0-xlator:
            /usr/lib64/glusterfs/3.12.1/xlator/nfs/server.so: cannot
            open shared object file: No such file or directory<br>
            [2017-11-07 22:20:47.595960] W [MSGID: 101095]
            [xlator.c:162:xlator_volopt_dynload] 0-xlator:
            /usr/lib64/glusterfs/3.12.1/xlator/nfs/server.so: cannot
            open shared object file: No such file or directory<br>
            [2017-11-07 22:20:47.631833] I [MSGID: 106600]
            [glusterd-nfs-svc.c:163:glusterd_nfssvc_reconfigure]
            0-management: nfs/server.so xlator is not installed<br>
            [2017-11-07 22:20:47.635109] I [MSGID: 106132]
            [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management:
            quotad already stopped<br>
            [2017-11-07 22:20:47.635136] I [MSGID: 106568]
            [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management:
            quotad service is stopped<br>
            [2017-11-07 22:20:47.635201] I [MSGID: 106132]
            [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management:
            bitd already stopped<br>
            [2017-11-07 22:20:47.635216] I [MSGID: 106568]
            [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management:
            bitd service is stopped<br>
            [2017-11-07 22:20:47.635284] I [MSGID: 106132]
            [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management:
            scrub already stopped<br>
            [2017-11-07 22:20:47.635297] I [MSGID: 106568]
            [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management:
            scrub service is stopped<br>
            [2017-11-07 22:20:47.648524] I [run.c:190:runner_log]
            (--&gt;/usr/lib64/glusterfs/3.12.1/xlator/mgmt/glusterd.so(+0xde17a)
            [0x7fc23442117a]
            --&gt;/usr/lib64/glusterfs/3.12.1/xlator/mgmt/glusterd.so(+0xddc3d)
            [0x7fc234420c3d]
            --&gt;/lib64/libglusterfs.so.0(runner_log+0x115)
            [0x7fc23f915da5] ) 0-management: Ran script: /var/lib<br>
            /glusterd/hooks/1/set/post/S30samba-set.sh --volname=gv0 -o
            cluster.halo-max-latency=10 --gd-workdir=/var/lib/glusterd<br>
            [2017-11-07 22:20:47.654091] I [run.c:190:runner_log]
            (--&gt;/usr/lib64/glusterfs/3.12.1/xlator/mgmt/glusterd.so(+0xde17a)
            [0x7fc23442117a]
            --&gt;/usr/lib64/glusterfs/3.12.1/xlator/mgmt/glusterd.so(+0xddc3d)
            [0x7fc234420c3d]
            --&gt;/lib64/libglusterfs.so.0(runner_log+0x115)
            [0x7fc23f915da5] ) 0-management: Ran script: /var/lib<br>
/glusterd/hooks/1/set/post/S32gluster_enable_shared_storage.sh
            --volname=gv0 -o cluster.halo-max-latency=10
            --gd-workdir=/var/lib/glusterd</p>
          <br>
        </div>
      </div>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap="">_______________________________________________
Gluster-users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a class="moz-txt-link-freetext" href="http://lists.gluster.org/mailman/listinfo/gluster-users">http://lists.gluster.org/mailman/listinfo/gluster-users</a></pre>
    </blockquote>
    <br>
  </body>
</html>