[Gluster-users] geo-replication command rsync returned with 3

[Dietmar Putz]

Hi all,

i have made some tests on the latest Ubuntu 16.04.3 server image. 
Upgrades were disabled...
the configuration was always the same...a distributed replicated volume 
on 4 VM's with geo-replication to a dist. repl .volume on 4 VM's.
i started with 3.7.20, upgrade to 3.8.15, to 3.10.9 to 3.12.5. After 
each upgrade i have tested the geo-replication which worked well anytime.
then i have made an update / upgrade on the first master node. directly 
after upgrade the below shown error appeared on that node.
after upgrade on the second master node the error appeared there also... 
geo replication is faulty.

this error affects gfs 3.7.20, 3.8.15, 3.10.9 and 3.12.5 on Ubuntu 16.04.3
in one test i have updated rsync from 3.1.1 to 3.1.2 but with no effect.

does anyone else experienced this behavior...any idea ?

best regards
Dietmar


gfs 3.12.5 geo-rep log on master :

[2018-01-24 15:50:35.347959] I [master(/brick1/mvol1):1385:crawl] 
_GMaster: slave's time stime=(1516808792, 0)
[2018-01-24 15:50:35.604094] I [master(/brick1/mvol1):1863:syncjob] 
Syncer: Sync Time Taken duration=0.0294    num_files=1    job=2    
return_code=3
[2018-01-24 15:50:35.605490] E [resource(/brick1/mvol1):210:errlog] 
Popen: command returned error    cmd=rsync -aR0 --inplace --files-from=- 
--super --stats --numeric-ids --no-implied-dirs --existing --xattrs 
--acls --ignore-missing-args . -e ssh -oPasswordAuthentication=no 
-oStrictHostKeyChecking=no -i 
/var/lib/glusterd/geo-replication/secret.pem -p 22 -oControlMaster=auto 
-S /tmp/gsyncd-aux-ssh-MZwEp2/cbad1c5f88978ecd713bdb1478fbabbe.sock 
--compress root at gl-node5-int:/proc/2013/cwd    error=3
[2018-01-24 15:50:35.628978] I [syncdutils(/brick1/mvol1):271:finalize] 
<top>: exiting.



after this upgrade one server fails :
Start-Date: 2018-01-18  04:33:52
Commandline: /usr/bin/unattended-upgrade
Upgrade:
libdns-export162:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8, 
1:9.10.3.dfsg.P4-8ubuntu1.10),
libisccfg140:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8, 
1:9.10.3.dfsg.P4-8ubuntu1.10),
bind9-host:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8, 
1:9.10.3.dfsg.P4-8ubuntu1.10),
dnsutils:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8, 1:9.10.3.dfsg.P4-8ubuntu1.10),
libc6:amd64 (2.23-0ubuntu9, 2.23-0ubuntu10),
libisc160:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8, 
1:9.10.3.dfsg.P4-8ubuntu1.10),
locales:amd64 (2.23-0ubuntu9, 2.23-0ubuntu10),
libisc-export160:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8, 
1:9.10.3.dfsg.P4-8ubuntu1.10),
libc-bin:amd64 (2.23-0ubuntu9, 2.23-0ubuntu10),
liblwres141:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8, 
1:9.10.3.dfsg.P4-8ubuntu1.10),
libdns162:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8, 
1:9.10.3.dfsg.P4-8ubuntu1.10),
multiarch-support:amd64 (2.23-0ubuntu9, 2.23-0ubuntu10),
libisccc140:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8, 
1:9.10.3.dfsg.P4-8ubuntu1.10),
libbind9-140:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8, 
1:9.10.3.dfsg.P4-8ubuntu1.10)
End-Date: 2018-01-18  04:34:32



strace rsync :

30743 23:34:47 newfstatat(3, "6737", {st_mode=S_IFDIR|0755, 
st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0
30743 23:34:47 newfstatat(3, "6741", {st_mode=S_IFDIR|0755, 
st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0
30743 23:34:47 getdents(3, /* 0 entries */, 131072) = 0
30743 23:34:47 munmap(0x7fa4feae7000, 135168) = 0
30743 23:34:47 close(3)                 = 0
30743 23:34:47 write(2, "rsync: getcwd(): No such file or directory 
(2)", 46) = 46
30743 23:34:47 write(2, "\n", 1)        = 1
30743 23:34:47 rt_sigaction(SIGUSR1, {SIG_IGN, [], SA_RESTORER, 
0x7fa4fdf404b0}, NULL, 8) = 0
30743 23:34:47 rt_sigaction(SIGUSR2, {SIG_IGN, [], SA_RESTORER, 
0x7fa4fdf404b0}, NULL, 8) = 0
30743 23:34:47 write(2, "rsync error: errors selecting input/output 
files, dirs (code 3) at util.c(1056) [Receiver=3.1.1]", 96) = 96
30743 23:34:47 write(2, "\n", 1)        = 1
30743 23:34:47 exit_group(3)            = ?
30743 23:34:47 +++ exited with 3 +++




Am 19.01.2018 um 17:27 schrieb Joe Julian:
> ubuntu 16.04

-- 
Dietmar Putz
3Q GmbH
Kurfürstendamm 102
D-10711 Berlin
  
Mobile:   +49 171 / 90 160 39
Mail:     dietmar.putz at 3qsdn.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180124/76264a75/attachment.html>