<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <p>Hi all,</p>
    by downgrade of <br>
    ii  libc6:amd64 2.23-0ubuntu10<br>
    <br>
    to<br>
    <br>
    ii  libc6:amd64 2.23-0ubuntu3<br>
    <br>
    the problem was solved. at least in gfs testenvironment running gfs
    3.13.2 and 3.7.20 and on our productive environment with 3.7.18.<br>
    possibly it helps someone...<br>
    <br>
    best regards<br>
    Dietmar<br>
    <br>
    <br>
    <p><br>
    </p>
    <br>
    <div class="moz-cite-prefix">Am 25.01.2018 um 14:06 schrieb Dietmar
      Putz:<br>
    </div>
    <blockquote type="cite"
      cite="mid:2ff3b176-47bf-3072-5274-6c6b81884c8d@3qsdn.com">
      <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
      <p>Hi Kotresh,</p>
      <p>thanks for your response...<br>
      </p>
      <p>i have made further tests based on ubuntu 16.04.3 (latest
        upgrades) and gfs 3.12.5 with following rsync version :<br>
      </p>
      <tt>1. ii  rsync                              3.1.1-3ubuntu1<br>
        2. ii  rsync                              3.1.1-3ubuntu1.2<br>
        3. ii  rsync                              3.1.2-2ubuntu0.1</tt><br>
      <br>
      in each test all nodes had the same rsync version installed. all
      tests failed with the below shown error.<br>
      then i started a test with the same setup. this time upgrades for
      ubuntu 16.04.3 were disabled.<br>
      again i have tested the above shown rsync versions on all nodes.
      the geo replication works fine in any case.<br>
      then i have installed  3.1.1-3ubuntu1.2 on each node as it is
      installed with latest ubuntu upgrade.<br>
      afterwards i have just upgraded all slave nodes with the latest
      ubuntu upgrades, reboot...the geo-replication still works fine.<br>
      after upgrade and reboot of the master nodes with the latest
      ubuntu upgrades the below shown error appeared again... geo rep is
      broken.<br>
      <br>
      i believe one of the below shown packages caused the error because
      directly after upgrade of these packages the error appeared<br>
      on the corresponding master node....<br>
      anyone else who has experienced this issue...?<br>
      any help would be appreciated.<br>
      <br>
      best regards<br>
      Dietmar<br>
      <br>
      <br>
      <br>
      <br>
      <div class="moz-cite-prefix">Am 25.01.2018 um 04:14 schrieb
        Kotresh Hiremath Ravishankar:<br>
      </div>
      <blockquote type="cite"
cite="mid:CAPgWtC5myPHxoX5fAYabr24p_77fP33HqDrKtwf5XkW+dq1HHA@mail.gmail.com">
        <div dir="ltr">
          <div>
            <div>It is clear that rsync is failing. Are the rsync
              versions on all masters and slave nodes same?<br>
            </div>
            I have seen that has caused problems sometimes.<br>
            <br>
          </div>
          -Kotresh HR<br>
        </div>
        <div class="gmail_extra"><br>
          <div class="gmail_quote">On Wed, Jan 24, 2018 at 10:29 PM,
            Dietmar Putz <span dir="ltr">&lt;<a
                href="mailto:dietmar.putz@3qsdn.com" target="_blank"
                moz-do-not-send="true">dietmar.putz@3qsdn.com</a>&gt;</span>
            wrote:<br>
            <blockquote class="gmail_quote" style="margin:0 0 0
              .8ex;border-left:1px #ccc solid;padding-left:1ex">
              <div text="#000000" bgcolor="#FFFFFF">
                <p>Hi all,</p>
                i have made some tests on the latest Ubuntu 16.04.3
                server image. Upgrades were disabled...<br>
                the configuration was always the same...a distributed
                replicated volume on 4 VM's with geo-replication to a
                dist. repl .volume on 4 VM's.<br>
                i started with 3.7.20, upgrade to 3.8.15, to 3.10.9 to
                3.12.5. After each upgrade i have tested the
                geo-replication which worked well anytime.<br>
                then i have made an update / upgrade on the first master
                node. directly after upgrade the below shown error
                appeared on that node.<br>
                after upgrade on the second master node the error
                appeared there also... geo replication is faulty.  <br>
                <br>
                this error affects gfs 3.7.20, 3.8.15, 3.10.9 and 3.12.5
                on Ubuntu 16.04.3<br>
                in one test i have updated rsync from 3.1.1 to 3.1.2 but
                with no effect.<br>
                <br>
                does anyone else experienced this behavior...any idea ?<br>
                <br>
                best regards<br>
                Dietmar<br>
                <br>
                <p><br>
                </p>
                <p><tt>gfs 3.12.5 geo-rep log on master :<br>
                  </tt></p>
                <p><tt>[2018-01-24 15:50:35.347959] I
                    [master(/brick1/mvol1):1385:<wbr>crawl] _GMaster:
                    slave's time    stime=(1516808792, 0) </tt><tt><br>
                  </tt><tt>[2018-01-24 15:50:35.604094] I
                    [master(/brick1/mvol1):1863:<wbr>syncjob] Syncer:
                    Sync Time Taken    duration=0.0294    num_files=1   
                    job=2    return_code=3 </tt><tt><br>
                  </tt><tt>[2018-01-24 15:50:35.605490] E
                    [resource(/brick1/mvol1):210:<wbr>errlog] Popen:
                    command returned error    cmd=rsync -aR0 --inplace
                    --files-from=- --super --stats --numeric-ids
                    --no-implied-dirs --existing --xattrs --acls
                    --ignore-missing-args . -e ssh
                    -oPasswordAuthentication=no
                    -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-<wbr>replication/secret.pem
                    -p 22 -oControlMaster=auto -S
                    /tmp/gsyncd-aux-ssh-MZwEp2/<wbr>cbad1c5f88978ecd713bdb1478fbab<wbr>be.sock
                    --compress root@gl-node5-int:/proc/2013/<wbr>cwd   
                    error=3 </tt><tt><br>
                  </tt><tt>[2018-01-24 15:50:35.628978] I
                    [syncdutils(/brick1/mvol1):<wbr>271:finalize]
                    &lt;top&gt;: exiting.</tt></p>
                <p><tt><br>
                  </tt></p>
                <p><br>
                </p>
                <p>after this upgrade one server fails :<br>
                  <tt>Start-Date: 2018-01-18  04:33:52 <br>
                    Commandline: /usr/bin/unattended-upgrade <br>
                    Upgrade: <br>
                    libdns-export162:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8,
                    1:9.10.3.dfsg.P4-8ubuntu1.10), <br>
                    libisccfg140:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8,
                    1:9.10.3.dfsg.P4-8ubuntu1.10), <br>
                    bind9-host:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8,
                    1:9.10.3.dfsg.P4-8ubuntu1.10), <br>
                    dnsutils:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8,
                    1:9.10.3.dfsg.P4-8ubuntu1.10), <br>
                    libc6:amd64 (2.23-0ubuntu9, 2.23-0ubuntu10), <br>
                    libisc160:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8,
                    1:9.10.3.dfsg.P4-8ubuntu1.10), <br>
                    locales:amd64 (2.23-0ubuntu9, 2.23-0ubuntu10), <br>
                    libisc-export160:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8,
                    1:9.10.3.dfsg.P4-8ubuntu1.10), <br>
                    libc-bin:amd64 (2.23-0ubuntu9, 2.23-0ubuntu10), <br>
                    liblwres141:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8,
                    1:9.10.3.dfsg.P4-8ubuntu1.10), <br>
                    libdns162:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8,
                    1:9.10.3.dfsg.P4-8ubuntu1.10), <br>
                    multiarch-support:amd64 (2.23-0ubuntu9,
                    2.23-0ubuntu10), <br>
                    libisccc140:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8,
                    1:9.10.3.dfsg.P4-8ubuntu1.10), <br>
                    libbind9-140:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8,
                    1:9.10.3.dfsg.P4-8ubuntu1.10) <br>
                    End-Date: 2018-01-18  04:34:32 </tt></p>
                <p><tt><br>
                  </tt></p>
                <p><br>
                  <tt><tt>strace rsync :<br>
                    </tt></tt></p>
                <p><tt><tt>30743 23:34:47 newfstatat(3, "6737",
                      {st_mode=S_IFDIR|0755, st_size=4096, ...},
                      AT_SYMLINK_NOFOLLOW) = 0<br>
                      30743 23:34:47 newfstatat(3, "6741",
                      {st_mode=S_IFDIR|0755, st_size=4096, ...},
                      AT_SYMLINK_NOFOLLOW) = 0<br>
                      30743 23:34:47 getdents(3, /* 0 entries */,
                      131072) = 0<br>
                      30743 23:34:47 munmap(0x7fa4feae7000, 135168) = 0<br>
                      30743 23:34:47 close(3)                 = 0<br>
                      30743 23:34:47 write(2, "rsync: getcwd(): No such
                      file or directory (2)", 46) = 46<br>
                      30743 23:34:47 write(2, "\n", 1)        = 1<br>
                      30743 23:34:47 rt_sigaction(SIGUSR1, {SIG_IGN, [],
                      SA_RESTORER, 0x7fa4fdf404b0}, NULL, 8) = 0<br>
                      30743 23:34:47 rt_sigaction(SIGUSR2, {SIG_IGN, [],
                      SA_RESTORER, 0x7fa4fdf404b0}, NULL, 8) = 0<br>
                      30743 23:34:47 write(2, "rsync error: errors
                      selecting input/output files, dirs (code 3) at
                      util.c(1056) [Receiver=3.1.1]", 96) = 96<br>
                      30743 23:34:47 write(2, "\n", 1)        = 1<br>
                      30743 23:34:47 exit_group(3)            = ?<br>
                      30743 23:34:47 +++ exited with 3 +++</tt> </tt></p>
                <p><br>
                </p>
                <p><br>
                </p>
                <br>
                <div class="m_3318466771008572220moz-cite-prefix">Am
                  19.01.2018 um 17:27 schrieb Joe Julian:<br>
                </div>
                <blockquote type="cite">
                  <pre class="m_3318466771008572220k9mail">ubuntu 16.04</pre>
                  <span class="HOEnZb"><font color="#888888"> </font></span></blockquote>
                <span class="HOEnZb"></span> </div>
              <br>
              ______________________________<wbr>_________________<br>
              Gluster-users mailing list<br>
              <a href="mailto:Gluster-users@gluster.org"
                moz-do-not-send="true">Gluster-users@gluster.org</a><br>
              <a
                href="http://lists.gluster.org/mailman/listinfo/gluster-users"
                rel="noreferrer" target="_blank" moz-do-not-send="true">http://lists.gluster.org/<wbr>mailman/listinfo/gluster-users</a><br>
            </blockquote>
          </div>
          <br>
          <br clear="all">
          <br>
          -- <br>
          <div class="gmail_signature" data-smartmail="gmail_signature">
            <div dir="ltr">
              <div>Thanks and Regards,<br>
              </div>
              Kotresh H R<br>
            </div>
          </div>
        </div>
      </blockquote>
      <br>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap="">_______________________________________________
Gluster-users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a class="moz-txt-link-freetext" href="http://lists.gluster.org/mailman/listinfo/gluster-users">http://lists.gluster.org/mailman/listinfo/gluster-users</a></pre>
    </blockquote>
    <br>
    <pre class="moz-signature" cols="72">-- 
Dietmar Putz
3Q GmbH
Kurfürstendamm 102
D-10711 Berlin
 
Mobile:   +49 171 / 90 160 39
Mail:     <a class="moz-txt-link-abbreviated" href="mailto:dietmar.putz@3qsdn.com">dietmar.putz@3qsdn.com</a></pre>
  </body>
</html>