[Gluster-users] favorite-child and self-heal BUG ?

[artur.k]

>>
>>> As Keith said you should not modify the backend directly.
>>>
>>
>> if the disc will fall on a single server glaster remove files from the second (good server) ?
>>
>>
>>
> noc-xx-2:/mnt/glusterfs# echo 7 > 7
>> bash: 7: Input/output error
>> noc-xx-2:/mnt/glusterfs#
>> noc-xx-2:/mnt/glusterfs# echo 7 > 7
>> bash: 7: Input/output error
>> noc-xx-2:/mnt/glusterfs# echo 7 > 7
>> noc-xx-2:/mnt/glusterfs#
>> noc-xx-2:/mnt/glusterfs#
>>
>>
>> ehh... if the 7 file exist only once server. on second server deleted by rm -f  trac-xx-1:/var/storage/glusterfs/7
>> :D
>>
>>

>Artur,

> Can you explain once again the problem you are seeing? Step by step?

> Krishna




1.
Both servers are running. 

noc-xx-2:/mnt/glusterfs# touch 1
noc-xx-2:/mnt/glusterfs# touch 2
noc-xx-2:/mnt/glusterfs# touch 3
noc-xx-2:/mnt/glusterfs# touch 4
noc-xx-2:/mnt/glusterfs# touch 5
noc-xx-2:/mnt/glusterfs# touch 6
noc-xx-2:/mnt/glusterfs# touch 7
noc-xx-2:/mnt/glusterfs# touch 8
noc-xx-2:/mnt/glusterfs# touch 9
noc-xx-2:/mnt/glusterfs# ls
1  2  3  4  5  6  7  8  9


2.

I disable the server set as "favorite child" 

trac-xx-1:/var/storage/glusterfs# /etc/init.d/glusterfs-server stop
Stopping glusterfs server: glusterfsd.

trac-xx-1:/var/storage/glusterfs# rm -rf 8 9
trac-xx-1:/var/storage/glusterfs# ls
1  2  3  4  5  6  7
trac-xx-1:/var/storage/glusterfs# /etc/init.d/glusterfs-server start
Starting glusterfs server: glusterfsd.

3.

noc-xx-2:/mnt/glusterfs# ls
1  2  3  4  5  6  7  8  9
noc-xx-2:/mnt/glusterfs#
noc-xx-2:/mnt/glusterfs#
noc-xx-2:/mnt/glusterfs# ls
1  2  3  4  5  6  7


Hmmm...readings are taken from the default server designated as "favorite child"  ??

noc-xx-2:/mnt/glusterfs# cat 8
cat: 8: Input/output error
noc-xx-2:/mnt/glusterfs# cat 8
cat: 8: Input/output error
noc-xx-2:/mnt/glusterfs# cat 8
noc-xx-2:/mnt/glusterfs#
noc-xx-2:/mnt/glusterfs#
noc-xx-2:/mnt/glusterfs# cat 8
noc-xx-2:/mnt/glusterfs# cat 9
cat: 9: Input/output error
noc-xx-2:/mnt/glusterfs# cat 9
cat: 9: Input/output error
noc-xx-2:/mnt/glusterfs# cat 9
noc-xx-2:/mnt/glusterfs# cat 9

4.

trac-xx-1:/var/storage/glusterfs# ls
1  2  3  4  5  6  7  8  9


5.

trac-xx-2:/var/storage/glusterfs# ls
1  2  3  4  5  6  7  8  9


As you can see the files replicated themselves but only after the third attempt to open the file on a client. As I understand I shouldn't remove the file directly on the server but such situation might occur. Perhaps the scenarion should be addressed somehow? 


logs:
2009-01-09 09:20:19 E [afr-self-heal-data.c:778:afr_sh_data_fix] afr: Unable to resolve conflicting data of /8. Please resolve manually by deleting the file /8 from all but the preferred subvolume. Please consider 'option favorite-child <>'
2009-01-09 09:20:19 W [afr.c:618:afr_open] afr: returning EIO, file has to be manually corrected in backend
2009-01-09 09:20:19 E [fuse-bridge.c:662:fuse_fd_cbk] glusterfs-fuse: 1091: OPEN() /8 => -1 (Input/output error)
2009-01-09 09:20:20 W [afr.c:618:afr_open] afr: returning EIO, file has to be manually corrected in backend
2009-01-09 09:20:20 E [fuse-bridge.c:662:fuse_fd_cbk] glusterfs-fuse: 1092: OPEN() /8 => -1 (Input/output error)
2009-01-09 09:20:21 W [afr-self-heal-data.c:764:afr_sh_data_fix] afr: Picking favorite child client1 as authentic source to resolve conflicting data of /8
2009-01-09 09:20:21 W [afr-self-heal-data.c:615:afr_sh_data_open_cbk] afr: sourcing file /8 from client1 to other sinks
2009-01-09 09:22:10 W [afr-self-heal-common.c:843:sh_missing_entries_lookup_cbk] afr: path /9 on subvolume client1 => -1 (No such file or directory)
2009-01-09 09:22:10 E [afr-self-heal-data.c:778:afr_sh_data_fix] afr: Unable to resolve conflicting data of /9. Please resolve manually by deleting the file /9 from all but the preferred subvolume. Please consider 'option favorite-child <>'
2009-01-09 09:22:10 W [afr.c:618:afr_open] afr: returning EIO, file has to be manually corrected in backend
2009-01-09 09:22:10 E [fuse-bridge.c:662:fuse_fd_cbk] glusterfs-fuse: 1114: OPEN() /9 => -1 (Input/output error)
2009-01-09 09:22:11 W [afr.c:618:afr_open] afr: returning EIO, file has to be manually corrected in backend
2009-01-09 09:22:11 E [fuse-bridge.c:662:fuse_fd_cbk] glusterfs-fuse: 1115: OPEN() /9 => -1 (Input/output error)
2009-01-09 09:22:12 W [afr-self-heal-data.c:764:afr_sh_data_fix] afr: Picking favorite child client1 as authentic source to resolve conflicting data of /9
2009-01-09 09:22:12 W [afr-self-heal-data.c:615:afr_sh_data_open_cbk] afr: sourcing file /9 from client1 to other sinks