[Gluster-users] If i will upload file on one peer

[Alexey Shalin]

Hello
and again.. after some time .. rsync stop working
in storage.log I see a lot of message like:


[2013-08-16 02:56:08.025068] I [afr-self-heal-common.c:1213:afr_sh_missing_entry_call_impunge_recreate] 0-storage-replicate-0: no missing files - /home/bizexperts/data/www/bizexpert.kg/libraries/joomla/database/databaseexception.php. proceeding to metadata check
[2013-08-16 02:56:08.033602] I [afr-self-heal-common.c:1213:afr_sh_missing_entry_call_impunge_recreate] 0-storage-replicate-0: no missing files - /home/bizexperts/data/www/bizexpert.kg/libraries/joomla/database/exception.php. proceeding to metadata check

I checked this file on both nodes (peers)
on first:
root at mirror:~# ls -al /home/bizon1/data/www/bmx.on.kg/engine/editor/static.php
-rw-r--r-- 2 bizon1 bizon1 12331 Sep 25  2012 /home/bizon1/data/www/bmx.on.kg/engine/editor/static.php
root at mirror:~#

on second:
root at nas2-backup:/# ls -al /home/bizon1/data/www/bmx.on.kg/engine/editor/static.php
-rw-r--r-- 1 bizon1 bizon1 12331 Sep 25  2012 /home/bizon1/data/www/bmx.on.kg/engine/editor/static.php
root at nas2-backup:/#

and on client:

root at ispcp:/var/log/glusterfs# ls -al /home/bizon1/data/www/bmx.on.kg/engine/editor/static.php
-rw-r--r-- 1 bizon1 bizon1 12331 Sep 25  2012 /home/bizon1/data/www/bmx.on.kg/engine/editor/static.php
root at ispcp:/var/log/glusterfs#




when i started rsync on client:
rsync --log-file=/root/home_sync -altusgopr --inplace  --exclude-from=/root/exclude.txt -e ssh  --delete --delete-excluded root at 192.168.15.71:/storage/home/ /storage/home/

i checked logs storage.log on client and here is some lines:

[2013-08-16 03:01:05.103442] W [client-rpc-fops.c:2624:client3_3_lookup_cbk] 0-storage-client-0: remote operation failed: No data available. Path: /home/tmp/.test.sh (00000000-0000-0000-0000-000000000000)
[2013-08-16 03:01:05.103511] I [afr-self-heal-entry.c:607:afr_sh_entry_expunge_entry_cbk] 0-storage-replicate-0: looking up /home/tmp/.test.sh under storage-client-0 failed (No data available)
[2013-08-16 03:01:05.103558] W [client-rpc-fops.c:2624:client3_3_lookup_cbk] 0-storage-client-0: remote operation failed: No data available. Path: /home/tmp/mc-1181 (00000000-0000-0000-0000-000000000000)
[2013-08-16 03:01:05.103571] I [afr-self-heal-entry.c:607:afr_sh_entry_expunge_entry_cbk] 0-storage-replicate-0: looking up /home/tmp/mc-1181 under storage-client-0 failed (No data available)
[2013-08-16 03:01:05.103593] W [client-rpc-fops.c:2624:client3_3_lookup_cbk] 0-storage-client-0: remote operation failed: No data available. Path: /home/tmp/mc-3031 (00000000-0000-0000-0000-000000000000)
[2013-08-16 03:01:05.103607] I [afr-self-heal-entry.c:607:afr_sh_entry_expunge_entry_cbk] 0-storage-replicate-0: looking up /home/tmp/mc-3031 under storage-client-0 failed (No data available)
[2013-08-16 03:01:05.103710] W [client-rpc-fops.c:2624:client3_3_lookup_cbk] 0-storage-client-0: remote operation failed: No data available. Path: /home/tmp/doctrine_schema_22469.yml (00000000-0000-0000-0000-000000000000)
[2013-08-16 03:01:05.103782] I [afr-self-heal-entry.c:607:afr_sh_entry_expunge_entry_cbk] 0-storage-replicate-0: looking up /home/tmp/doctrine_schema_22469.yml under storage-client-0 failed (No data available)
[2013-08-16 03:01:05.106094] W [client-rpc-fops.c:2624:client3_3_lookup_cbk] 0-storage-client-0: remote operation failed: No data available. Path: /home/tmp/mc-2290 (00000000-0000-0000-0000-000000000000)
[2013-08-16 03:01:05.106163] I [afr-self-heal-entry.c:607:afr_sh_entry_expunge_entry_cbk] 0-storage-replicate-0: looking up /home/tmp/mc-2290 under storage-client-0 failed (No data available)
[2013-08-16 03:01:05.106205] W [client-rpc-fops.c:2624:client3_3_lookup_cbk] 0-storage-client-0: remote operation failed: No data available. Path: /home/tmp/mc-mad (00000000-0000-0000-0000-000000000000)
[2013-08-16 03:01:05.106241] I [afr-self-heal-entry.c:607:afr_sh_entry_expunge_entry_cbk] 0-storage-replicate-0: looking up /home/tmp/mc-mad under storage-client-0 failed (No data available)
[2013-08-16 03:01:05.110690] I [afr-self-heal-entry.c:1837:afr_sh_entry_common_lookup_done] 0-storage-replicate-0: /home/tmp/mc-1181: Skipping entry self-heal because of gfid absence
[2013-08-16 03:01:05.110741] I [afr-self-heal-entry.c:1837:afr_sh_entry_common_lookup_done] 0-storage-replicate-0: /home/tmp/.test.sh: Skipping entry self-heal because of gfid absence
[2013-08-16 03:01:05.110777] I [afr-self-heal-entry.c:1837:afr_sh_entry_common_lookup_done] 0-storage-replicate-0: /home/tmp/mc-2290: Skipping entry self-heal because of gfid absence
[2013-08-16 03:01:05.111228] I [afr-self-heal-entry.c:1837:afr_sh_entry_common_lookup_done] 0-storage-replicate-0: /home/tmp/mc-3031: Skipping entry self-heal because of gfid absence
[2013-08-16 03:01:05.114187] I [afr-self-heal-entry.c:1837:afr_sh_entry_common_lookup_done] 0-storage-replicate-0: /home/tmp/mc-mad: Skipping entry self-heal because of gfid absence
[2013-08-16 03:01:05.114278] I [afr-self-heal-entry.c:1837:afr_sh_entry_common_lookup_done] 0-storage-replicate-0: /home/tmp/doctrine_schema_22469.yml: Skipping entry self-heal because of gfid absence
[2013-08-16 03:01:05.116824] E [afr-self-heal-common.c:2212:afr_self_heal_completion_cbk] 0-storage-replicate-0: background  entry self-heal failed on /home/tmp
[2013-08-16 03:01:06.807915] W [client-rpc-fops.c:2624:client3_3_lookup_cbk] 0-storage-client-0: remote operation failed: No data available. Path: /home/tmp/.test.sh (00000000-0000-0000-0000-000000000000)
[2013-08-16 03:01:06.807976] I [afr-self-heal-entry.c:607:afr_sh_entry_expunge_entry_cbk] 0-storage-replicate-0: looking up /home/tmp/.test.sh under storage-client-0 failed (No data available)
[2013-08-16 03:01:06.808030] W [client-rpc-fops.c:2624:client3_3_lookup_cbk] 0-storage-client-0: remote operation failed: No data available. Path: /home/tmp/mc-1181 (00000000-0000-0000-0000-000000000000)
[2013-08-16 03:01:06.808057] I [afr-self-heal-entry.c:607:afr_sh_entry_expunge_entry_cbk] 0-storage-replicate-0: looking up /home/tmp/mc-1181 under storage-client-0 failed (No data available)
[2013-08-16 03:01:06.808115] W [client-rpc-fops.c:2624:client3_3_lookup_cbk] 0-storage-client-0: remote operation failed: No data available. Path: /home/tmp/mc-3031 (00000000-0000-0000-0000-000000000000)
[2013-08-16 03:01:06.808138] I [afr-self-heal-entry.c:607:afr_sh_entry_expunge_entry_cbk] 0-storage-replicate-0: looking up /home/tmp/mc-3031 under storage-client-0 failed (No data available)
[2013-08-16 03:01:06.808176] W [client-rpc-fops.c:2624:client3_3_lookup_cbk] 0-storage-client-0: remote operation failed: No data available. Path: /home/tmp/doctrine_schema_22469.yml (00000000-0000-0000-0000-000000000000)
[2013-08-16 03:01:06.808193] I [afr-self-heal-entry.c:607:afr_sh_entry_expunge_entry_cbk] 0-storage-replicate-0: looking up /home/tmp/doctrine_schema_22469.yml under storage-client-0 failed (No data available)
[2013-08-16 03:01:06.809869] W [client-rpc-fops.c:2624:client3_3_lookup_cbk] 0-storage-client-0: remote operation failed: No data available. Path: /home/tmp/mc-2290 (00000000-0000-0000-0000-000000000000)
[2013-08-16 03:01:06.809910] I [afr-self-heal-entry.c:607:afr_sh_entry_expunge_entry_cbk] 0-storage-replicate-0: looking up /home/tmp/mc-2290 under storage-client-0 failed (No data available)
[2013-08-16 03:01:06.809972] W [client-rpc-fops.c:2624:client3_3_lookup_cbk] 0-storage-client-0: remote operation failed: No data available. Path: /home/tmp/mc-mad (00000000-0000-0000-0000-000000000000)
[2013-08-16 03:01:06.809987] I [afr-self-heal-entry.c:607:afr_sh_entry_expunge_entry_cbk] 0-storage-replicate-0: looking up /home/tmp/mc-mad under storage-client-0 failed (No data available)
[2013-08-16 03:01:06.813612] I [afr-self-heal-entry.c:1837:afr_sh_entry_common_lookup_done] 0-storage-replicate-0: /home/tmp/mc-1181: Skipping entry self-heal because of gfid absence
[2013-08-16 03:01:06.813681] I [afr-self-heal-entry.c:1837:afr_sh_entry_common_lookup_done] 0-storage-replicate-0: /home/tmp/.test.sh: Skipping entry self-heal because of gfid absence
[2013-08-16 03:01:06.813711] I [afr-self-heal-entry.c:1837:afr_sh_entry_common_lookup_done] 0-storage-replicate-0: /home/tmp/mc-2290: Skipping entry self-heal because of gfid absence
[2013-08-16 03:01:06.813740] I [afr-self-heal-entry.c:1837:afr_sh_entry_common_lookup_done] 0-storage-replicate-0: /home/tmp/mc-3031: Skipping entry self-heal because of gfid absence
[2013-08-16 03:01:06.816162] I [afr-self-heal-entry.c:1837:afr_sh_entry_common_lookup_done] 0-storage-replicate-0: /home/tmp/mc-mad: Skipping entry self-heal because of gfid absence
[2013-08-16 03:01:06.816261] I [afr-self-heal-entry.c:1837:afr_sh_entry_common_lookup_done] 0-storage-replicate-0: /home/tmp/doctrine_schema_22469.yml: Skipping entry self-heal because of gfid absence
[2013-08-16 03:01:06.817643] E [afr-self-heal-common.c:2212:afr_self_heal_completion_cbk] 0-storage-replicate-0: background  entry self-heal failed on /home/tmp

Ok then can i do next

before  run  rsync  on  peer  i will shutdown glusterfs-server on both
peers..  then  I will do rsync, and after finishing rsync i will start
glusterfs-server on bith peer ,,, but i do not think and files will be
updated on second peer




-------------------------------------------------------
Старший Системный Администратор
Алексей Шалин
ОсОО "Хостер kg" - http://www.hoster.kg
ул. Ахунбаева 123 (здание БГТС)
help at hoster.kg