<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div class="elementToProof"><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">Hi
<a href="mailto:aravinda@kadalu.tech" id="OWAAM737709" class="tWKOu mention ms-bgc-nlr ms-fcl-b" data-loopstyle="linkonly">
@Aravinda</a>,</span></div>
<div class="elementToProof"><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"><br>
</span></div>
<p class="elementToProof" style="margin-top: 0px; margin-bottom: 0px;"><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">I have checked the rsync version, and it's
the same on primary and secondary nodes. We have rsync version 3.1.3, protocol version 31, on all servers. It's very strange that we have not made any changes, that we are aware of, and this geo-replication was working fine for the last 5 years, and suddenly
it has stopped, and we are unable to understand the root cause of it.</span></p>
<p><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"><br>
</span></p>
<p><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">I have checked the tcpdump and I can see that the master node is sending RST to the secondary node when geo-replication
connects, but we are not seeing any RST when we do the ssh using the root user from master to secondary node ourselves, which makes me think that geo-replication is able to connect to the secondary node but after that, it's not liking something and tries to
reset the connection, and this is repeating in a loop.</span></p>
<p><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"><br>
</span></p>
<p><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">I have also enabled geo-replication debug logs and I am getting this error in the master node gsyncd logs.</span></p>
<p><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"><br>
</span></p>
<p><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">[2024-02-07 22:37:36.820978] D [repce(worker /opt/tier1data2019/brick):195:push] RepceClient: call 2563661:140414778891136:1707345456.8209238
entry_ops([{'op': 'CREATE', 'skip_entry': False, 'gfid': '3d57e1e4-7bd2-44f6-a6d1-d628208b3697', 'entry': '.gfid/9a39167c-6c28-470a-b699-11eeaaff8edd/app_docmerge8795785720233840105.docx', 'mode': 33188, 'uid': 0, 'gid': 0}, {'op': 'UNLINK', 'skip_entry':
False, 'gfid': '3d57e1e4-7bd2-44f6-a6d1-d628208b3697', 'entry': '.gfid/9a39167c-6c28-470a-b699-11eeaaff8edd/app_docmerge8795785720233840105.docx'}, {'op': 'CREATE', 'skip_entry': False, 'gfid': '7bd35f91-1408-476d-869a-9936f2d94afc', 'entry': '.gfid/9a39167c-6c28-470a-b699-11eeaaff8edd/0c3fb22f-0fbe-4445-845b-9d94d84a9888',
'mode': 33188, 'uid': 0, 'gid': 0}, {'op': 'CREATE', 'skip_entry': False, 'gfid': '3837018c-2f5e-43d4-ab58-0ed8b7456e73', 'entry': '.gfid/861afb81-386a-4b5b-af37-cef63a55a436/26fcd7e7-2c8c-4dcb-96f2-2c8a0d79f3d4', 'mode': 33188, 'uid': 0, 'gid': 0}, {'op':
'CREATE', 'skip_entry': False, 'gfid': 'db311b10-b1e2-4b84-adea-a6746214aeda', 'entry': '.gfid/861afb81-386a-4b5b-af37-cef63a55a436/0526d0da-1f36-4203-8563-7e23aacf6237', 'mode': 33188, 'uid': 0, 'gid': 0}, {'op': 'CREATE', 'skip_entry': False, 'gfid': '9bbb253a-226a-44b1-a968-7cfa76cf9463',
'entry': '.gfid/e861ff10-696a-4b03-9716-39d9e7dd08d7/app_docmergeLLRenewalLetterDocusign_1_22_15_1_18_153.doc', 'mode': 33188, 'uid': 0, 'gid': 0}, {'op': 'UNLINK', 'skip_entry': False, 'gfid': '9bbb253a-226a-44b1-a968-7cfa76cf9463', 'entry': '.gfid/e861ff10-696a-4b03-9716-39d9e7dd08d7/app_docmergeLLRenewalLetterDocusign_1_22_15_1_18_153.doc'},
{'op': 'CREATE', 'skip_entry': False, 'gfid': 'f62d0c65-6ede-48ff-b9bf-c44a33e5e023', 'entry': '.gfid/e861ff10-696a-4b03-9716-39d9e7dd08d7/85530794-c15f-44d4-8660-87a14c2c9c8c', 'mode': 33188, 'uid': 0, 'gid': 0}, {'op': 'CREATE', 'skip_entry': False, 'gfid':
'fd3d0af6-8ef5-4b76-bb47-0bc508df0ed0', 'entry': '.gfid/e861ff10-696a-4b03-9716-39d9e7dd08d7/app_docmergeMOA_1_22_15_1_20_501.doc', 'mode': 33188, 'uid': 0, 'gid': 0}, {'op': 'UNLINK', 'skip_entry': False, 'gfid': 'fd3d0af6-8ef5-4b76-bb47-0bc508df0ed0', 'entry':
'.gfid/e861ff10-696a-4b03-9716-39d9e7dd08d7/app_docmergeMOA_1_22_15_1_20_501.doc'}, {'op': 'CREATE', 'skip_entry': False, 'gfid': 'e93c5771-9676-40d4-90cd-f0586ec05dd9', 'entry': '.gfid/e861ff10-696a-4b03-9716-39d9e7dd08d7/cc372667-3b77-468f-bac6-671d4eb069e9',
'mode': 33188, 'uid': 0, 'gid': 0}, {'op': 'CREATE', 'skip_entry': False, 'gfid': '02045f44-68ff-4a35-a843-08939afc46a4', 'entry': '.gfid/e861ff10-696a-4b03-9716-39d9e7dd08d7/app_docmergeTTRenewalLetterASTNoFee-2022_1_22_15_1_19_530.doc', 'mode': 33188, 'uid':
0, 'gid': 0}, {'op': 'UNLINK', 'skip_entry': False, 'gfid': '02045f44-68ff-4a35-a843-08939afc46a4', 'entry': '.gfid/e861ff10-696a-4b03-9716-39d9e7dd08d7/app_docmergeTTRenewalLetterASTNoFee-2022_1_22_15_1_19_530.doc'}, {'op': 'CREATE', 'skip_entry': False,
'gfid': '6f5766c9-2dc3-4636-9041-9cf4ac64d26b', 'entry': '.gfid/e861ff10-696a-4b03-9716-39d9e7dd08d7/556a0e3c-510d-4396-8f32-335aafec1314', 'mode': 33188, 'uid': 0, 'gid': 0}, {'op': 'UNLINK', 'skip_entry': False, 'gfid': 'f78561f0-c9f2-4192-a82a-8368e0ad8b2b',
'entry': '.gfid/ec161c2e-bb32-4639-a7b2-9be961221d86/app_1705935977525.tmp'}, {'op': 'CREATE', 'skip_entry': False, 'gfid': 'd1e33edb-523e-41c1-a021-8bd3a5a2c7c0', 'entry': '.gfid/e861ff10-696a-4b03-9716-39d9e7dd08d7/c655e3e5-9d4c-43d7-9171-949f01612e6d',
'mode': 33188, 'uid': 0, 'gid': 0}, {'op': 'CREATE', 'skip_entry': False, 'gfid': 'b6f44b28-c2bf-4e70-b953-1c559ded7835', 'entry': '.gfid/9a39167c-6c28-470a-b699-11eeaaff8edd/app_docmerge7370453767656401681.docx', 'mode': 33188, 'uid': 0, 'gid': 0}, {'op':
'UNLINK', 'skip_entry': False, 'gfid': 'b6f44b28-c2bf-4e70-b953-1c559ded7835', 'entry': '.gfid/9a39167c-6c28-470a-b699-11eeaaff8edd/app_docmerge7370453767656401681.docx'}, {'op': 'CREATE', 'skip_entry': False, 'gfid': '2d845d9e-7a49-4200-a100-759fe831ba0e',
'entry': '.gfid/9a39167c-6c28-470a-b699-11eeaaff8edd/84d47d84-5749-4a19-8f73-293078d17c63', 'mode': 33188, 'uid': 0, 'gid': 0}, {'op': 'CREATE', 'skip_entry': False, 'gfid': '44554c17-21aa-427a-b796-7ecec6af2570', 'entry': '.gfid/9a39167c-6c28-470a-b699-11eeaaff8edd/app_docmerge8634804987715893755.docx',
'mode': 33188, 'uid': 0, 'gid': 0}, {'op': 'CREATE', 'skip_entry': False, 'gfid': '652bf5d7-3b7a-41d8-aa4f-e52296034821', 'entry': '.gfid/9a39167c-6c28-470a-b699-11eeaaff8edd/91a25682-69ea-4edc-9250-d6c7aac56853', 'mode': 33188, 'uid': 0, 'gid': 0}, {'op':
'UNLINK', 'skip_entry': False, 'gfid': '44554c17-21aa-427a-b796-7ecec6af2570', 'entry': '.gfid/9a39167c-6c28-470a-b699-11eeaaff8edd/app_docmerge8634804987715893755.docx'}, {'op': 'CREATE', 'skip_entry': False, 'gfid': '04720811-b90e-42b7-a5d1-656afd92e245',
'entry': '.gfid/9a39167c-6c28-470a-b699-11eeaaff8edd/a66cbc42-61dc-4896-bb69-c715f1a820db', 'mode': 33188, 'uid': 0, 'gid': 0}],) ...</span></p>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">[2024-02-07 22:37:36.909606] D [repce(worker /opt/tier1data2019/brick):215:__call__] RepceClient: call 2563661:140414778891136:1707345456.8209238
entry_ops -> []</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">[2024-02-07 22:37:36.911032] D [master(worker /opt/tier1data2019/brick):317:a_syncdata] _GMaster: files [{files={'.gfid/652bf5d7-3b7a-41d8-aa4f-e52296034821',
'.gfid/2d845d9e-7a49-4200-a100-759fe831ba0e', '.gfid/3837018c-2f5e-43d4-ab58-0ed8b7456e73', '.gfid/e93c5771-9676-40d4-90cd-f0586ec05dd9', '.gfid/f62d0c65-6ede-48ff-b9bf-c44a33e5e023', '.gfid/7bd35f91-1408-476d-869a-9936f2d94afc', '.gfid/04720811-b90e-42b7-a5d1-656afd92e245',
'.gfid/6f5766c9-2dc3-4636-9041-9cf4ac64d26b', '.gfid/db311b10-b1e2-4b84-adea-a6746214aeda', '.gfid/d1e33edb-523e-41c1-a021-8bd3a5a2c7c0'}}]</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">[2024-02-07 22:37:36.911089] D [master(worker /opt/tier1data2019/brick):320:a_syncdata] _GMaster: candidate
for syncing [{file=.gfid/652bf5d7-3b7a-41d8-aa4f-e52296034821}]</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">[2024-02-07 22:37:36.911133] D [master(worker /opt/tier1data2019/brick):320:a_syncdata] _GMaster: candidate
for syncing [{file=.gfid/2d845d9e-7a49-4200-a100-759fe831ba0e}]</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">[2024-02-07 22:37:36.911169] D [master(worker /opt/tier1data2019/brick):320:a_syncdata] _GMaster: candidate
for syncing [{file=.gfid/3837018c-2f5e-43d4-ab58-0ed8b7456e73}]</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">[2024-02-07 22:37:36.911202] D [master(worker /opt/tier1data2019/brick):320:a_syncdata] _GMaster: candidate
for syncing [{file=.gfid/e93c5771-9676-40d4-90cd-f0586ec05dd9}]</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">[2024-02-07 22:37:36.911235] D [master(worker /opt/tier1data2019/brick):320:a_syncdata] _GMaster: candidate
for syncing [{file=.gfid/f62d0c65-6ede-48ff-b9bf-c44a33e5e023}]</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">[2024-02-07 22:37:36.911268] D [master(worker /opt/tier1data2019/brick):320:a_syncdata] _GMaster: candidate
for syncing [{file=.gfid/7bd35f91-1408-476d-869a-9936f2d94afc}]</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">[2024-02-07 22:37:36.911301] D [master(worker /opt/tier1data2019/brick):320:a_syncdata] _GMaster: candidate
for syncing [{file=.gfid/04720811-b90e-42b7-a5d1-656afd92e245}]</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">[2024-02-07 22:37:36.911333] D [master(worker /opt/tier1data2019/brick):320:a_syncdata] _GMaster: candidate
for syncing [{file=.gfid/6f5766c9-2dc3-4636-9041-9cf4ac64d26b}]</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">[2024-02-07 22:37:36.911366] D [master(worker /opt/tier1data2019/brick):320:a_syncdata] _GMaster: candidate
for syncing [{file=.gfid/db311b10-b1e2-4b84-adea-a6746214aeda}]</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">[2024-02-07 22:37:36.911398] D [master(worker /opt/tier1data2019/brick):320:a_syncdata] _GMaster: candidate
for syncing [{file=.gfid/d1e33edb-523e-41c1-a021-8bd3a5a2c7c0}]</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">[2024-02-07 22:37:36.911439] D [master(worker /opt/tier1data2019/brick):1344:process] _GMaster: processing
change [{changelog=/var/lib/misc/gluster/gsyncd/tier1data_drtier1data_drtier1data/opt-tier1data2019-brick/.history/.processing/CHANGELOG.1705936007}]</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">[2024-02-07 22:37:36.915193] E [syncdutils(worker /opt/tier1data2019/brick):346:log_raise_exception] <top>:
Gluster Mount process exited [{error=ENOTCONN}]</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">[2024-02-07 22:37:36.915252] E [syncdutils(worker /opt/tier1data2019/brick):363:log_raise_exception] <top>:
FULL EXCEPTION TRACE:</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">Traceback (most recent call last):</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line 317, in main</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> func(args)</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> File "/usr/libexec/glusterfs/python/syncdaemon/subcmds.py", line 86, in subcmd_worker</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> local.service_loop(remote)</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> File "/usr/libexec/glusterfs/python/syncdaemon/resource.py", line 1298, in service_loop</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> g3.crawlwrap(oneshot=True)</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> File "/usr/libexec/glusterfs/python/syncdaemon/master.py", line 604, in crawlwrap</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> self.crawl()</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> File "/usr/libexec/glusterfs/python/syncdaemon/master.py", line 1614, in crawl</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> self.changelogs_batch_process(changes)</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> File "/usr/libexec/glusterfs/python/syncdaemon/master.py", line 1510, in changelogs_batch_process</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> self.process(batch)</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> File "/usr/libexec/glusterfs/python/syncdaemon/master.py", line 1345, in process</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> self.process_change(change, done, retry)</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> File "/usr/libexec/glusterfs/python/syncdaemon/master.py", line 1071, in process_change</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> st = lstat(pt)</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py", line 589, in lstat</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> return errno_wrap(os.lstat, [e], [ENOENT], [ESTALE, EBUSY])</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py", line 571, in errno_wrap</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> return call(*arg)</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">OSError: [Errno 107] Transport endpoint is not connected: '.gfid/d53fad8f-84e9-4b24-9eb0-ccbcbdc4baa8'</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">[2024-02-07 22:37:37.344426] I [monitor(monitor):228:monitor] Monitor: worker died in startup phase [{brick=/opt/tier1data2019/brick}]</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">[2024-02-07 22:37:37.346601] I [gsyncdstatus(monitor):248:set_worker_status] GeorepStatus: Worker Status Change
[{status=Faulty}]</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"><br>
</span></div>
<div id="appendonsend"></div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Thanks,</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Anant</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<hr style="display: inline-block; width: 98%;">
<div dir="ltr" id="divRplyFwdMsg"><span style="font-family: Calibri, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);"><b>From:</b> Aravinda <aravinda@kadalu.tech><br>
<b>Sent:</b> 07 February 2024 2:54 PM<br>
<b>To:</b> Anant Saraswat <anant.saraswat@techblue.co.uk><br>
<b>Cc:</b> Strahil Nikolov <hunter86_bg@yahoo.com>; gluster-users@gluster.org <gluster-users@gluster.org><br>
<b>Subject:</b> Re: [Gluster-users] __Geo-replication status is getting Faulty after few seconds</span>
<div> </div>
</div>
<p><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(250, 247, 190);"><b>EXTERNAL: Do not click links or open attachments if you do not recognize
the sender.</b></span></p>
<div><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt; color: rgb(0, 0, 0);">It will keep track of last sync time if you change to non-root user. But I don't think the issue is related to root vs non-root user. </span></div>
<div><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt; color: rgb(0, 0, 0);"><br>
</span></div>
<div><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt; color: rgb(0, 0, 0);">Even in non-root user based Geo-rep, Primary volume is mounted using root user only. Only in the secondary node, it will use Glusterd mountbroker to
allow mounting the Secondary volume as non-priviliaged user.</span></div>
<div><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt; color: rgb(0, 0, 0);"><br>
</span></div>
<div><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt; color: rgb(0, 0, 0);">Check the rsync version in Primary and secondary nodes. Please fix the versions if not matching. </span></div>
<div><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt; color: rgb(0, 0, 0);"><br>
</span></div>
<div><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt; color: rgb(0, 0, 0);">--</span></div>
<div data-zbluepencil-ignore="true" data-sigid="3848334000000010003" id="x_Zm-_Id_-Sgn">
<div>Aravinda</div>
<div>Kadalu Technologies</div>
</div>
<div><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt; color: rgb(0, 0, 0);"><br>
</span></div>
<div style="line-height: 0px; margin-top: 10px; margin-bottom: 10px; border-top: 1px solid rgb(204, 204, 204); height: 0px;">
<span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt; color: rgb(0, 0, 0);"><br>
</span></div>
<div><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt; color: rgb(0, 0, 0);"><br>
</span></div>
<div id="x_Zm-_Id_-Sgn1">---- On Wed, 07 Feb 2024 20:11:47 +0530 <b>Anant Saraswat <anant.saraswat@techblue.co.uk></b> wrote ---<br>
</div>
<div><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt; color: rgb(0, 0, 0);"><br>
</span></div>
<div style="direction: ltr; margin-right: 0px; margin-left: 0px;"><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">No, It was setup and running using the root user
only. </span></div>
<div style="direction: ltr; margin: 0px;"><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"><br>
</span></div>
<div style="direction: ltr; margin: 0px;"><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">Do you think I should setup using
</span><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);"> a dedicated non-root user</span><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">?
will it keep the track of old files or will it consider it as a new geo-replication and copy all the files from the scratch?</span></div>
<div style="direction: ltr; margin: 0px;"><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"><br>
</span></div>
<hr style="direction: ltr; margin-right: 0px; margin-left: 0px; display: inline-block; width: 98%;">
<div id="x_x_1199982551divRplyFwdMsg" dir="ltr"><span style="font-family: Calibri, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);"><b>From:</b> Strahil Nikolov <<a href="mailto:hunter86_bg@yahoo.com" id="OWA4756b457-afd9-b67f-8230-3f7aa299dd52" class="OWAAutoLink" data-loopstyle="linkonly">hunter86_bg@yahoo.com</a>><br>
<b>Sent:</b> 07 February 2024 2:36 PM<br>
<b>To:</b> Anant Saraswat <<a href="mailto:anant.saraswat@techblue.co.uk" id="OWA49c32ff5-1f8e-6ae2-a28b-8ea002671071" class="OWAAutoLink" data-loopstyle="linkonly">anant.saraswat@techblue.co.uk</a>>; Aravinda <<a href="mailto:aravinda@kadalu.tech" id="OWAc39081d1-0b2e-e0ed-8cc0-a3e723191767" class="OWAAutoLink" data-loopstyle="linkonly">aravinda@kadalu.tech</a>><br>
<b>Cc:</b> <a href="mailto:gluster-users@gluster.org" id="OWAecefab6c-354a-76cc-385b-c8aa07eb54ba" class="OWAAutoLink" data-loopstyle="linkonly">gluster-users@gluster.org</a> <<a href="mailto:gluster-users@gluster.org" id="OWA3daff331-077d-27ed-9ce1-7ac3c58af7c5" class="OWAAutoLink" data-loopstyle="linkonly">gluster-users@gluster.org</a>><br>
<b>Subject:</b> Re: [Gluster-users] __Geo-replication status is getting Faulty after few seconds</span>
<div> </div>
</div>
<p style="direction: ltr; margin: 0px;"><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt; color: rgb(0, 0, 0); background-color: rgb(250, 247, 190);"><b>EXTERNAL: Do not click links or open attachments if you do not recognize
the sender.</b></span></p>
<div style="direction: ltr; margin-right: 0px; margin-left: 0px;"><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt; color: rgb(0, 0, 0);">Have you tried setting up gluster georep with a dedicated non-root user ?</span></div>
<div style="direction: ltr; margin-right: 0px; margin-left: 0px;"><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt; color: rgb(0, 0, 0);"><br>
</span></div>
<div style="direction: ltr; margin-right: 0px; margin-left: 0px;"><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt; color: rgb(0, 0, 0);">Best Regards,</span></div>
<div style="direction: ltr; margin-right: 0px; margin-left: 0px;"><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt; color: rgb(0, 0, 0);">Strahil Nikolov<br>
<br>
</span></div>
<blockquote style="margin: 0px 0px 20px;">
<div style="direction: ltr;"><span style="font-family: Roboto, sans-serif; font-size: 10pt; color: rgb(109, 0, 246);">On Tue, Feb 6, 2024 at 16:38, Anant Saraswat</span></div>
<div style="direction: ltr;"><span style="font-family: Roboto, sans-serif; font-size: 10pt; color: rgb(109, 0, 246);"><<a href="mailto:anant.saraswat@techblue.co.uk" id="OWA00253d86-b7a4-5520-7bef-b4cc63cf9bdd" class="OWAAutoLink" data-loopstyle="linkonly">anant.saraswat@techblue.co.uk</a>>
wrote:</span></div>
<div style="direction: ltr; margin: 10px 0px 0px; padding-top: 10px; padding-left: 20px; border-left: 1px solid rgb(109, 0, 246);">
<span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt; color: rgb(0, 0, 0);">________<br>
<br>
<br>
<br>
Community Meeting Calendar:<br>
<br>
Schedule -<br>
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC<br>
Bridge: <a href="https://urldefense.com/v3/__https://meet.google.com/cpu-eiue-hvk__;!!I_DbfM1H!Dm8_fHcUmz5wnOfTdrkMSb6PXqGdC_3VpklsIdfjPuKgee_Ds7JD__1KjwR4F62a67f5292of5PyQVk9y3-TRe_00eSiJw$" id="OWA329d8c5a-4869-7ebd-3082-4bb58e747808" class="x_x_1199982551OWAAutoLink" data-auth="NotApplicable" data-loopstyle="linkonly">
https://meet.google.com/cpu-eiue-hvk</a><br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" id="OWA44a88427-98dd-1302-caab-8db590c7e511" class="x_x_1199982551OWAAutoLink" data-loopstyle="linkonly">Gluster-users@gluster.org</a><br>
<a href="https://urldefense.com/v3/__https://lists.gluster.org/mailman/listinfo/gluster-users__;!!I_DbfM1H!Dm8_fHcUmz5wnOfTdrkMSb6PXqGdC_3VpklsIdfjPuKgee_Ds7JD__1KjwR4F62a67f5292of5PyQVk9y3-TRe-GwoljEQ$" id="OWA0ef3fb17-e8d8-4524-0495-d1df8e72d974" class="x_x_1199982551OWAAutoLink" data-auth="NotApplicable" data-loopstyle="linkonly">https://lists.gluster.org/mailman/listinfo/gluster-users</a></span></div>
</blockquote>
<p style="direction: ltr; margin: 0px;"><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt; color: rgb(0, 0, 0);"><br>
</span></p>
<p style="direction: ltr; margin: 0px;"><span style="font-family: ARIAL; font-size: 10pt; color: rgb(0, 0, 0);">DISCLAIMER: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error, please notify the sender. This message contains confidential information and is intended only for the individual named. If you are not the named addressee, you should not disseminate, distribute or copy
this email. Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system.<br>
<br>
If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. Thanks for your cooperation.</span></p>
<div><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt; color: rgb(0, 0, 0);"><br>
</span></div>
<span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"><br>
</span>
<p style="FONT-SIZE: 10pt; FONT-FAMILY: ARIAL"><span style="FONT-FAMILY: Calibri Light"></p>
<p style="FONT-SIZE: 10pt; FONT-FAMILY: ARIAL">DISCLAIMER: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please
notify the sender. This message contains confidential information and is intended only for the individual named. If you are not the named addressee, you should not disseminate, distribute or copy this email. Please notify the sender immediately by email if
you have received this email by mistake and delete this email from your system. <br>
<br>
If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. Thanks for your cooperation.
</span></p>
</body>
</html>