[Bugs] [Bug 1240658] New: gluster volume heal full does not heal data and files inaccessible from mount point

[bugzilla at redhat.com]

https://bugzilla.redhat.com/show_bug.cgi?id=1240658

            Bug ID: 1240658
           Summary: gluster volume heal full does not heal data and files
                    inaccessible from mount point
           Product: Red Hat Gluster Storage
           Version: 3.1
         Component: gluster-afr
          Assignee: pkarampu at redhat.com
          Reporter: ravishankar at redhat.com
        QA Contact: storage-qa-internal at redhat.com
                CC: bugs at gluster.org, gluster-bugs at redhat.com
        Depends On: 1112158
             Group: redhat



+++ This bug was initially created as a clone of Bug #1112158 +++

Description of problem:
When the file and gfid link is deleted from the backend of one of the replica
pairs, they become inaccessible from the mount point and 'gluster volume heal
<volname> full' fails to trigger the heal.

Version-Release number of selected component (if applicable):


How reproducible:
Always on a 2 node setup (seems to work fine if volume is created on one node
only).

Steps to Reproduce:
1. Create 1x2 replica, fuse mount it.
2. mkdir -p /fuse_mnt/dir1/dir2
3. rm -f /brick2/.glusterfs/<gfid-link of dir1>
4. rmdir /brick2/dir1/dir2
5. gluster v heal <volname> full

After some time:
6.[root at ravi4 fuse_mnt]# ls -lR
.:
total 0
drwxr-xr-x 2 root root 6 Jun 23  2014 dir1

./dir1:
total 0

Actual results:
dir2 is missing from mount point and in brick2

Expected results:
dir2 must be accessible since it is present in brick1 and heal full must
recreate it in brick2

--- Additional comment from Ravishankar N on 2014-07-16 06:11:57 EDT ---

The repro steps mentioned in the bug description is not accurate as the brick2
was not killed/restarted. The correct steps would be:

1. Create 1x2 replica on 2 nodes, fuse mount it.
2. mkdir -p /fuse_mnt/dir1/dir2
3. kill one brick process (say brick2)
3. rm -rf /brick2/*
4. rm -rf /brick2/.glusterfs
5. restart brick2 
6. gluster v heal <volname> full

During multiple trial runs, it is observed that the heal full is not happening
on the node having the highest  UUID (i.e if brick2's node has highes UUID,
then data is not healed to it).

--- Additional comment from Ravishankar N on 2015-06-25 00:35:22 EDT ---

Note: Since full heals are done in replace-brick or increasing the replica
count scenarios. 

For replace-brick, http://review.gluster.org/#/c/10076/ and
http://review.gluster.org/#/c/10448/ should fix the issue. The patches mark the
replaced brick as needing heal via AFR changelog xattrs. This should cause the
self-heal daemon to automatically trigger heals without the need to run the
`heal <volname> full` command.

We would need to do a similar fix for replica count increase (add-brick
command) as well.


Referenced Bugs:

https://bugzilla.redhat.com/show_bug.cgi?id=1112158
[Bug 1112158] gluster volume heal full does not heal data and files
inaccessible from mount point
-- 
You are receiving this mail because:
You are on the CC list for the bug.
Unsubscribe from this bug https://bugzilla.redhat.com/token.cgi?t=vZlAmtifyd&a=cc_unsubscribe