<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <p>Hi Kotresh,</p>
    <p>thanks for your response...<br>
    </p>
    <p>i have made further tests based on ubuntu 16.04.3 (latest
      upgrades) and gfs 3.12.5 with following rsync version :<br>
    </p>
    <tt>1. ii  rsync                              3.1.1-3ubuntu1<br>
      2. ii  rsync                              3.1.1-3ubuntu1.2<br>
      3. ii  rsync                              3.1.2-2ubuntu0.1</tt><br>
    <br>
    in each test all nodes had the same rsync version installed. all
    tests failed with the below shown error.<br>
    then i started a test with the same setup. this time upgrades for
    ubuntu 16.04.3 were disabled.<br>
    again i have tested the above shown rsync versions on all nodes. the
    geo replication works fine in any case.<br>
    then i have installed  3.1.1-3ubuntu1.2 on each node as it is
    installed with latest ubuntu upgrade.<br>
    afterwards i have just upgraded all slave nodes with the latest
    ubuntu upgrades, reboot...the geo-replication still works fine.<br>
    after upgrade and reboot of the master nodes with the latest ubuntu
    upgrades the below shown error appeared again... geo rep is broken.<br>
    <br>
    i believe one of the below shown packages caused the error because
    directly after upgrade of these packages the error appeared<br>
    on the corresponding master node....<br>
    anyone else who has experienced this issue...?<br>
    any help would be appreciated.<br>
    <br>
    best regards<br>
    Dietmar<br>
    <br>
    <br>
    <br>
    <br>
    <div class="moz-cite-prefix">Am 25.01.2018 um 04:14 schrieb Kotresh
      Hiremath Ravishankar:<br>
    </div>
    <blockquote type="cite"
cite="mid:CAPgWtC5myPHxoX5fAYabr24p_77fP33HqDrKtwf5XkW+dq1HHA@mail.gmail.com">
      <div dir="ltr">
        <div>
          <div>It is clear that rsync is failing. Are the rsync versions
            on all masters and slave nodes same?<br>
          </div>
          I have seen that has caused problems sometimes.<br>
          <br>
        </div>
        -Kotresh HR<br>
      </div>
      <div class="gmail_extra"><br>
        <div class="gmail_quote">On Wed, Jan 24, 2018 at 10:29 PM,
          Dietmar Putz <span dir="ltr">&lt;<a
              href="mailto:dietmar.putz@3qsdn.com" target="_blank"
              moz-do-not-send="true">dietmar.putz@3qsdn.com</a>&gt;</span>
          wrote:<br>
          <blockquote class="gmail_quote" style="margin:0 0 0
            .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div text="#000000" bgcolor="#FFFFFF">
              <p>Hi all,</p>
              i have made some tests on the latest Ubuntu 16.04.3 server
              image. Upgrades were disabled...<br>
              the configuration was always the same...a distributed
              replicated volume on 4 VM's with geo-replication to a
              dist. repl .volume on 4 VM's.<br>
              i started with 3.7.20, upgrade to 3.8.15, to 3.10.9 to
              3.12.5. After each upgrade i have tested the
              geo-replication which worked well anytime.<br>
              then i have made an update / upgrade on the first master
              node. directly after upgrade the below shown error
              appeared on that node.<br>
              after upgrade on the second master node the error appeared
              there also... geo replication is faulty.  <br>
              <br>
              this error affects gfs 3.7.20, 3.8.15, 3.10.9 and 3.12.5
              on Ubuntu 16.04.3<br>
              in one test i have updated rsync from 3.1.1 to 3.1.2 but
              with no effect.<br>
              <br>
              does anyone else experienced this behavior...any idea ?<br>
              <br>
              best regards<br>
              Dietmar<br>
              <br>
              <p><br>
              </p>
              <p><tt>gfs 3.12.5 geo-rep log on master :<br>
                </tt></p>
              <p><tt>[2018-01-24 15:50:35.347959] I
                  [master(/brick1/mvol1):1385:<wbr>crawl] _GMaster:
                  slave's time    stime=(1516808792, 0) </tt><tt><br>
                </tt><tt>[2018-01-24 15:50:35.604094] I
                  [master(/brick1/mvol1):1863:<wbr>syncjob] Syncer: Sync
                  Time Taken    duration=0.0294    num_files=1   
                  job=2    return_code=3 </tt><tt><br>
                </tt><tt>[2018-01-24 15:50:35.605490] E
                  [resource(/brick1/mvol1):210:<wbr>errlog] Popen:
                  command returned error    cmd=rsync -aR0 --inplace
                  --files-from=- --super --stats --numeric-ids
                  --no-implied-dirs --existing --xattrs --acls
                  --ignore-missing-args . -e ssh
                  -oPasswordAuthentication=no -oStrictHostKeyChecking=no
                  -i /var/lib/glusterd/geo-<wbr>replication/secret.pem
                  -p 22 -oControlMaster=auto -S
                  /tmp/gsyncd-aux-ssh-MZwEp2/<wbr>cbad1c5f88978ecd713bdb1478fbab<wbr>be.sock
                  --compress root@gl-node5-int:/proc/2013/<wbr>cwd   
                  error=3 </tt><tt><br>
                </tt><tt>[2018-01-24 15:50:35.628978] I
                  [syncdutils(/brick1/mvol1):<wbr>271:finalize]
                  &lt;top&gt;: exiting.</tt></p>
              <p><tt><br>
                </tt></p>
              <p><br>
              </p>
              <p>after this upgrade one server fails :<br>
                <tt>Start-Date: 2018-01-18  04:33:52 <br>
                  Commandline: /usr/bin/unattended-upgrade <br>
                  Upgrade: <br>
                  libdns-export162:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8,
                  1:9.10.3.dfsg.P4-8ubuntu1.10), <br>
                  libisccfg140:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8,
                  1:9.10.3.dfsg.P4-8ubuntu1.10), <br>
                  bind9-host:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8,
                  1:9.10.3.dfsg.P4-8ubuntu1.10), <br>
                  dnsutils:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8,
                  1:9.10.3.dfsg.P4-8ubuntu1.10), <br>
                  libc6:amd64 (2.23-0ubuntu9, 2.23-0ubuntu10), <br>
                  libisc160:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8,
                  1:9.10.3.dfsg.P4-8ubuntu1.10), <br>
                  locales:amd64 (2.23-0ubuntu9, 2.23-0ubuntu10), <br>
                  libisc-export160:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8,
                  1:9.10.3.dfsg.P4-8ubuntu1.10), <br>
                  libc-bin:amd64 (2.23-0ubuntu9, 2.23-0ubuntu10), <br>
                  liblwres141:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8,
                  1:9.10.3.dfsg.P4-8ubuntu1.10), <br>
                  libdns162:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8,
                  1:9.10.3.dfsg.P4-8ubuntu1.10), <br>
                  multiarch-support:amd64 (2.23-0ubuntu9,
                  2.23-0ubuntu10), <br>
                  libisccc140:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8,
                  1:9.10.3.dfsg.P4-8ubuntu1.10), <br>
                  libbind9-140:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8,
                  1:9.10.3.dfsg.P4-8ubuntu1.10) <br>
                  End-Date: 2018-01-18  04:34:32 </tt></p>
              <p><tt><br>
                </tt></p>
              <p><br>
                <tt><tt>strace rsync :<br>
                  </tt></tt></p>
              <p><tt><tt>30743 23:34:47 newfstatat(3, "6737",
                    {st_mode=S_IFDIR|0755, st_size=4096, ...},
                    AT_SYMLINK_NOFOLLOW) = 0<br>
                    30743 23:34:47 newfstatat(3, "6741",
                    {st_mode=S_IFDIR|0755, st_size=4096, ...},
                    AT_SYMLINK_NOFOLLOW) = 0<br>
                    30743 23:34:47 getdents(3, /* 0 entries */, 131072)
                    = 0<br>
                    30743 23:34:47 munmap(0x7fa4feae7000, 135168) = 0<br>
                    30743 23:34:47 close(3)                 = 0<br>
                    30743 23:34:47 write(2, "rsync: getcwd(): No such
                    file or directory (2)", 46) = 46<br>
                    30743 23:34:47 write(2, "\n", 1)        = 1<br>
                    30743 23:34:47 rt_sigaction(SIGUSR1, {SIG_IGN, [],
                    SA_RESTORER, 0x7fa4fdf404b0}, NULL, 8) = 0<br>
                    30743 23:34:47 rt_sigaction(SIGUSR2, {SIG_IGN, [],
                    SA_RESTORER, 0x7fa4fdf404b0}, NULL, 8) = 0<br>
                    30743 23:34:47 write(2, "rsync error: errors
                    selecting input/output files, dirs (code 3) at
                    util.c(1056) [Receiver=3.1.1]", 96) = 96<br>
                    30743 23:34:47 write(2, "\n", 1)        = 1<br>
                    30743 23:34:47 exit_group(3)            = ?<br>
                    30743 23:34:47 +++ exited with 3 +++</tt> </tt></p>
              <p><br>
              </p>
              <p><br>
              </p>
              <br>
              <div class="m_3318466771008572220moz-cite-prefix">Am
                19.01.2018 um 17:27 schrieb Joe Julian:<br>
              </div>
              <blockquote type="cite">
                <pre class="m_3318466771008572220k9mail">ubuntu 16.04</pre>
                <span class="HOEnZb"><font color="#888888"> </font></span></blockquote>
              <span class="HOEnZb"><font color="#888888"></font></span>
            </div>
            <br>
            ______________________________<wbr>_________________<br>
            Gluster-users mailing list<br>
            <a href="mailto:Gluster-users@gluster.org"
              moz-do-not-send="true">Gluster-users@gluster.org</a><br>
            <a
              href="http://lists.gluster.org/mailman/listinfo/gluster-users"
              rel="noreferrer" target="_blank" moz-do-not-send="true">http://lists.gluster.org/<wbr>mailman/listinfo/gluster-users</a><br>
          </blockquote>
        </div>
        <br>
        <br clear="all">
        <br>
        -- <br>
        <div class="gmail_signature" data-smartmail="gmail_signature">
          <div dir="ltr">
            <div>Thanks and Regards,<br>
            </div>
            Kotresh H R<br>
          </div>
        </div>
      </div>
    </blockquote>
    <br>
  </body>
</html>