Looks good.<div>I will check in the history of the VMs if I did something else after the push-pem.</div><div>If you want, I can share the 2 KVM VMs and their definition, so you can try to figure out the differences ?<br><br><div id="ymail_android_signature">Best Regards,<br>Strahil Nikolov</div> <br> <blockquote style="margin: 0 0 20px 0;"> <div style="font-family:Roboto, sans-serif; color:#6D00F6;"> <div>On Sun, Sep 1, 2024 at 13:56, Karl Kleinpaste</div><div><karl@kleinpaste.org> wrote:</div> </div> <div style="padding: 10px 0 0 20px; margin: 10px 0 0 0; border-left: 1px solid #6D00F6;"> <div id="yiv7070595937"><div>
<div class="yiv7070595937moz-cite-prefix"><font face="FreeSerif">FYI, I will be
traveling for the next week, and may not see email much until
then.<br clear="none">
<br clear="none">
Your questions...<br clear="none">
</font><br clear="none">
On 8/31/24 04:59, Strahil Nikolov wrote:<br clear="none">
</div>
<blockquote type="cite">One
silly question: Did you try adding some files on the source volume
after the georep was created ?</blockquote>
<br clear="none">
<font face="FreeSerif">Yes. I wondered that, too, whether geo-rep
would not start simply because there was nothing to do. But yes,
there are a few files created after the geo-rep session was
created. And status remains just "Created."</font><br clear="none">
<br clear="none">
<blockquote type="cite">
<div>Can you share the output of your version of '<span style="">sh -x
/usr/libexec/glusterfs/gverify.sh masterVol slaveUser
slaveHost slaveVol sshPort logFileName' check ?</span></div>
</blockquote>
<br clear="none">
<font face="monospace">sh -x /usr/libexec/glusterfs/gverify.sh j
geoacct pms n 7887 /tmp/logger<br clear="none">
+ BUFFER_SIZE=104857600<br clear="none">
+ SSH_PORT=7887<br clear="none">
++ gluster --print-logdir<br clear="none">
+
primary_log_file=/var/log/glusterfs/geo-replication/gverify-primarymnt.log<br clear="none">
++ gluster --print-logdir<br clear="none">
+
secondary_log_file=/var/log/glusterfs/geo-replication/gverify-secondarymnt.log<br clear="none">
+ main j geoacct pms n 7887 /tmp/logger<br clear="none">
+ log_file=/tmp/logger<br clear="none">
+ inet6=<br clear="none">
+ local cmd_line<br clear="none">
+ local ver<br clear="none">
+ ping_host pms 7887<br clear="none">
+ '[' 0 -ne 0 ']'<br clear="none">
+ [[ -z '' ]]<br clear="none">
+ ssh -p 7887 -oNumberOfPasswordPrompts=0
-oStrictHostKeyChecking=no geoacct@pms 'echo
Testing_Passwordless_SSH'<br clear="none">
Testing_Passwordless_SSH<br clear="none">
+ '[' 0 -ne 0 ']'<br clear="none">
++ cmd_secondary<br clear="none">
++ local cmd_line<br clear="none">
+++ cat<br clear="none">
++ cmd_line='function do_verify() {<br clear="none">
ver=$(gluster --version | head -1 | cut -f2 -d " ");<br clear="none">
echo $ver;<br clear="none">
};<br clear="none">
source /etc/profile && do_verify;'<br clear="none">
++ echo function 'do_verify()' '{' 'ver=$(gluster' --version '|'
head -1 '|' cut -f2 -d '"' '");' echo '$ver;' '};' source
/etc/profile '&&' 'do_verify;'<br clear="none">
+ cmd_line='function do_verify() { ver=$(gluster --version | head
-1 | cut -f2 -d " "); echo $ver; }; source /etc/profile &&
do_verify;'<br clear="none">
+ [[ -z '' ]]<br clear="none">
++ ssh -p 7887 -oNumberOfPasswordPrompts=0
-oStrictHostKeyChecking=no geoacct@pms bash -c ''\''function
do_verify() { ver=$(gluster --version | head -1 | cut -f2 -d " ");
echo $ver; }; source /etc/profile && do_verify;'\'''<br clear="none">
+ ver=11.1<br clear="none">
+ '[' -z 11.1 ']'<br clear="none">
+ ERRORS=0<br clear="none">
++ primary_stats j<br clear="none">
++ PRIMARYVOL=j<br clear="none">
++ local inet6=<br clear="none">
++ local d<br clear="none">
++ local i<br clear="none">
++ local disk_size<br clear="none">
++ local used_size<br clear="none">
++ local ver<br clear="none">
++ local m_status<br clear="none">
+++ mktemp -d -t gverify.sh.XXXXXX<br clear="none">
++ d=/tmp/gverify.sh.7QpBxa<br clear="none">
++ '[' '' = inet6 ']'<br clear="none">
++ glusterfs -s localhost
'--xlator-option=*dht.lookup-unhashed=off' --volfile-id j -l
/var/log/glusterfs/geo-replication/gverify-primarymnt.log
/tmp/gverify.sh.7QpBxa<br clear="none">
+++ get_inode_num /tmp/gverify.sh.7QpBxa<br clear="none">
+++ local os<br clear="none">
+++ case `uname -s` in<br clear="none">
++++ uname -s<br clear="none">
+++ os=Linux<br clear="none">
+++ [[ XLinux = \X\N\e\t\B\S\D ]]<br clear="none">
++++ stat -c %i /tmp/gverify.sh.7QpBxa<br clear="none">
+++ echo 1<br clear="none">
++ i=1<br clear="none">
++ [[ 1 -ne 1 ]]<br clear="none">
++ cd /tmp/gverify.sh.7QpBxa<br clear="none">
+++ disk_usage /tmp/gverify.sh.7QpBxa<br clear="none">
+++ local os<br clear="none">
+++ case `uname -s` in<br clear="none">
+++ awk '{print $2}'<br clear="none">
++++ uname -s<br clear="none">
+++ os=Linux<br clear="none">
+++ [[ XLinux = \X\N\e\t\B\S\D ]]<br clear="none">
++++ df -P -B1 /tmp/gverify.sh.7QpBxa<br clear="none">
++++ tail -1<br clear="none">
+++ echo localhost:j 286755311616 2869571584 283885740032 2%
/tmp/gverify.sh.7QpBxa<br clear="none">
++ disk_size=286755311616<br clear="none">
+++ disk_usage /tmp/gverify.sh.7QpBxa<br clear="none">
+++ local os<br clear="none">
+++ case `uname -s` in<br clear="none">
+++ awk '{print $3}'<br clear="none">
++++ uname -s<br clear="none">
+++ os=Linux<br clear="none">
+++ [[ XLinux = \X\N\e\t\B\S\D ]]<br clear="none">
++++ df -P -B1 /tmp/gverify.sh.7QpBxa<br clear="none">
++++ tail -1<br clear="none">
+++ echo localhost:j 286755311616 2869571584 283885740032 2%
/tmp/gverify.sh.7QpBxa<br clear="none">
++ used_size=2869571584<br clear="none">
++ umount_lazy /tmp/gverify.sh.7QpBxa<br clear="none">
++ local os<br clear="none">
++ case `uname -s` in<br clear="none">
+++ uname -s<br clear="none">
++ os=Linux<br clear="none">
++ [[ XLinux = \X\N\e\t\B\S\D ]]<br clear="none">
++ umount -l /tmp/gverify.sh.7QpBxa<br clear="none">
++ rmdir /tmp/gverify.sh.7QpBxa<br clear="none">
+++ gluster --version<br clear="none">
+++ head -1<br clear="none">
+++ cut -f2 -d ' '<br clear="none">
++ ver=11.1<br clear="none">
+++ echo 286755311616:2869571584:11.1<br clear="none">
++ m_status=286755311616:2869571584:11.1<br clear="none">
++ echo 286755311616:2869571584:11.1<br clear="none">
+ primary_data=286755311616:2869571584:11.1<br clear="none">
++ secondary_stats geoacct pms n<br clear="none">
++ set -x<br clear="none">
++ SECONDARYUSER=geoacct<br clear="none">
++ SECONDARYHOST=pms<br clear="none">
++ SECONDARYVOL=n<br clear="none">
++ local inet6=<br clear="none">
++ local cmd_line<br clear="none">
++ local ver<br clear="none">
++ local status<br clear="none">
+++ mktemp -d -t gverify.sh.XXXXXX<br clear="none">
++ d=/tmp/gverify.sh.WqWlni<br clear="none">
++ '[' '' = inet6 ']'<br clear="none">
++ glusterfs '--xlator-option=*dht.lookup-unhashed=off'
--volfile-server pms --volfile-id n -l
/var/log/glusterfs/geo-replication/gverify-secondarymnt.log
/tmp/gverify.sh.WqWlni<br clear="none">
+++ get_inode_num /tmp/gverify.sh.WqWlni<br clear="none">
+++ local os<br clear="none">
+++ case `uname -s` in<br clear="none">
++++ uname -s<br clear="none">
+++ os=Linux<br clear="none">
+++ [[ XLinux = \X\N\e\t\B\S\D ]]<br clear="none">
++++ stat -c %i /tmp/gverify.sh.WqWlni<br clear="none">
+++ echo 1<br clear="none">
++ i=1<br clear="none">
++ [[ 1 -ne 1 ]]<br clear="none">
++ cd /tmp/gverify.sh.WqWlni<br clear="none">
+++ disk_usage /tmp/gverify.sh.WqWlni<br clear="none">
+++ local os<br clear="none">
+++ case `uname -s` in<br clear="none">
+++ awk '{print $2}'<br clear="none">
++++ uname -s<br clear="none">
+++ os=Linux<br clear="none">
+++ [[ XLinux = \X\N\e\t\B\S\D ]]<br clear="none">
++++ df -P -B1 /tmp/gverify.sh.WqWlni<br clear="none">
++++ tail -1<br clear="none">
+++ echo pms:n 286755311616 2868690944 283886620672 2%
/tmp/gverify.sh.WqWlni<br clear="none">
++ disk_size=286755311616<br clear="none">
+++ disk_usage /tmp/gverify.sh.WqWlni<br clear="none">
+++ local os<br clear="none">
+++ awk '{print $3}'<br clear="none">
+++ case `uname -s` in<br clear="none">
++++ uname -s<br clear="none">
+++ os=Linux<br clear="none">
+++ [[ XLinux = \X\N\e\t\B\S\D ]]<br clear="none">
++++ df -P -B1 /tmp/gverify.sh.WqWlni<br clear="none">
++++ tail -1<br clear="none">
+++ echo pms:n 286755311616 2868690944 283886620672 2%
/tmp/gverify.sh.WqWlni<br clear="none">
++ used_size=2868690944<br clear="none">
+++ find /tmp/gverify.sh.WqWlni -maxdepth 1 -path
/tmp/gverify.sh.WqWlni/.trashcan -prune -o -path
/tmp/gverify.sh.WqWlni -o -print0 -quit<br clear="none">
++ no_of_files=<br clear="none">
++ umount_lazy /tmp/gverify.sh.WqWlni<br clear="none">
++ local os<br clear="none">
++ case `uname -s` in<br clear="none">
+++ uname -s<br clear="none">
++ os=Linux<br clear="none">
++ [[ XLinux = \X\N\e\t\B\S\D ]]<br clear="none">
++ umount -l /tmp/gverify.sh.WqWlni<br clear="none">
++ rmdir /tmp/gverify.sh.WqWlni<br clear="none">
+++ cmd_secondary<br clear="none">
+++ local cmd_line<br clear="none">
++++ cat<br clear="none">
+++ cmd_line='function do_verify() {<br clear="none">
ver=$(gluster --version | head -1 | cut -f2 -d " ");<br clear="none">
echo $ver;<br clear="none">
};<br clear="none">
source /etc/profile && do_verify;'<br clear="none">
+++ echo function 'do_verify()' '{' 'ver=$(gluster' --version '|'
head -1 '|' cut -f2 -d '"' '");' echo '$ver;' '};' source
/etc/profile '&&' 'do_verify;'<br clear="none">
++ cmd_line='function do_verify() { ver=$(gluster --version | head
-1 | cut -f2 -d " "); echo $ver; }; source /etc/profile &&
do_verify;'<br clear="none">
+++ SSHM geoacct@pms bash -c ''\''function do_verify() {
ver=$(gluster --version | head -1 | cut -f2 -d " "); echo $ver; };
source /etc/profile && do_verify;'\'''<br clear="none">
+++ [[ -z '' ]]<br clear="none">
+++ ssh -p 7887 -q -oPasswordAuthentication=no
-oStrictHostKeyChecking=no -oControlMaster=yes geoacct@pms bash -c
''\''function do_verify() { ver=$(gluster --version | head -1 |
cut -f2 -d " "); echo $ver; }; source /etc/profile &&
do_verify;'\'''<div id="yiv7070595937yqtfd69700" class="yiv7070595937yqt5155495171"><br clear="none">
shell-init: error retrieving current directory: getcwd: cannot
access parent directories: No such file or directory</div><br clear="none">
++ ver=11.1<br clear="none">
++ status=286755311616:2868690944:11.1:<br clear="none">
++ echo 286755311616:2868690944:11.1:<br clear="none">
++ set +x<br clear="none">
+ secondary_data=286755311616:2868690944:11.1:<br clear="none">
++ echo 286755311616:2869571584:11.1<br clear="none">
++ cut -f1 -d:<br clear="none">
+ primary_disk_size=286755311616<br clear="none">
++ echo 286755311616:2868690944:11.1:<br clear="none">
++ cut -f1 -d:<br clear="none">
+ secondary_disk_size=286755311616<br clear="none">
++ echo 286755311616:2869571584:11.1<br clear="none">
++ cut -f2 -d:<br clear="none">
+ primary_used_size=2869571584<br clear="none">
++ echo 286755311616:2868690944:11.1:<br clear="none">
++ cut -f2 -d:<br clear="none">
+ secondary_used_size=2868690944<br clear="none">
++ echo 286755311616:2869571584:11.1<br clear="none">
++ cut -f3 -d:<br clear="none">
+ primary_version=11.1<br clear="none">
++ echo 286755311616:2868690944:11.1:<br clear="none">
++ cut -f3 -d:<br clear="none">
+ secondary_version=11.1<br clear="none">
++ echo 286755311616:2868690944:11.1:<br clear="none">
++ cut -f4 -d:<br clear="none">
+ secondary_no_of_files=<br clear="none">
+ [[ x286755311616 = \x ]]<br clear="none">
+ [[ x11.1 = \x ]]<br clear="none">
+ [[ 286755311616 -eq 0 ]]<br clear="none">
+ [[ x286755311616 = \x ]]<br clear="none">
+ [[ x11.1 = \x ]]<br clear="none">
+ [[ 286755311616 -eq 0 ]]<br clear="none">
+ '[' 286755311616 -lt 286755311616 ']'<br clear="none">
+ effective_primary_used_size=2974429184<br clear="none">
+ secondary_available_size=283886620672<br clear="none">
+ primary_available_size=283780882432<br clear="none">
+ '[' 283886620672 -lt 283780882432 ']'<br clear="none">
+ '[' '!' -z ']'<br clear="none">
+ [[ 11.1 != 11.1 ]]<br clear="none">
+ exit 0</font><div id="yiv7070595937yqtfd68262" class="yiv7070595937yqt5155495171"><br clear="none">
</div></div></div> </div> </blockquote></div>