[Bugs] [Bug 1256245] New: AFR: gluster v restart force or brick process restart doesn't heal the files

Mon Aug 24 07:18:23 UTC 2015

https://bugzilla.redhat.com/show_bug.cgi?id=1256245

            Bug ID: 1256245
           Summary: AFR: gluster v restart force or brick process restart
                    doesn't heal the files
           Product: GlusterFS
           Version: 3.6.5
         Component: replicate
          Keywords: Triaged
          Severity: urgent
          Assignee: bugs at gluster.org
          Reporter: ravishankar at redhat.com
                CC: annair at redhat.com, ashah at redhat.com,
                    asrivast at redhat.com, bugs at gluster.org,
                    gluster-bugs at redhat.com, ravishankar at redhat.com,
                    ssampat at redhat.com
        Depends On: 1239021, 1253309
            Blocks: 1251815, 1255690, 1223636

+++ This bug was initially created as a clone of Bug #1253309 +++

Description of problem:

When one of the replica brick is down and do some file operation, gluster vol
restart or brick process restart doesn't heal the files which needs to be
healed.

Version-Release number of selected component (if applicable):

glusterfs-3.7.1-7.el6rhs.x86_64

How reproducible:

100%

Steps to Reproduce:

1. Create 2*2 distribute replicate volume
2. Do fuse mount 
3. create some files on mount point
4. kill one of the replica brick
5. rename the file from the mount point
6. check gluster v heal <volname> info
7. restart the volume or restart the brick process

Actual results:

Files are not healed

Expected results:

volume restart or brick process restart should heal the files which need to be
healed

Additional info:

Volume Name: vol0
Type: Distributed-Replicate
Volume ID: 53c64343-c537-428c-b7b7-a45f198c42a0
Status: Started
Number of Bricks: 2 x 2 = 4
Transport-type: tcp
Bricks:
Brick1: 10.70.33.214:/rhs/brick1/b001
Brick2: 10.70.33.219:/rhs/brick1/b002
Brick3: 10.70.33.225:/rhs/brick1/b003
Brick4: 10.70.44.13:/rhs/brick1/b004
Options Reconfigured:
performance.readdir-ahead: on
features.uss: enable
features.quota: on
features.inode-quota: on
features.quota-deem-statfs: on
server.allow-insecure: on
features.barrier: disable
cluster.enable-shared-storage: enable

--- Additional comment from Ravishankar N on 2015-07-03 05:45:57 EDT ---

Currently in AFR-v2, when a CHILD_UP notification is received, the index heal
is triggered only on that particular child. The fix is to trigger the index
heal on all local children.

While this is a bug, it is not a blocker because the files will eventually get
healed in 10 minutes (default heal timeout value) or when the heal command is
explicitly launched via the gluster CLI.

--- Additional comment from Ravishankar N on 2015-08-13 09:15:30 EDT ---

http://review.gluster.org/#/c/11912/

--- Additional comment from Anand Avati on 2015-08-14 05:14:59 EDT ---

REVIEW: http://review.gluster.org/11912 (afr: launch index heal on local
subvols up on a child-up event) posted (#2) for review on master by Pranith
Kumar Karampuri (pkarampu at redhat.com)

--- Additional comment from Anand Avati on 2015-08-14 05:44:03 EDT ---

REVIEW: http://review.gluster.org/11912 (afr: launch index heal on local
subvols up on a child-up event) posted (#3) for review on master by Pranith
Kumar Karampuri (pkarampu at redhat.com)

--- Additional comment from Anand Avati on 2015-08-21 01:29:18 EDT ---

REVIEW: http://review.gluster.org/11912 (afr: launch index heal on local
subvols up on a child-up event) posted (#4) for review on master by Ravishankar
N (ravishankar at redhat.com)

--- Additional comment from Anand Avati on 2015-08-21 06:50:18 EDT ---

COMMIT: http://review.gluster.org/11912 committed in master by Pranith Kumar
Karampuri (pkarampu at redhat.com) 
------
commit e4cefd6c5915dd47c6b42098236df3901665f93a
Author: Ravishankar N <ravishankar at redhat.com>
Date:   Thu Aug 13 18:33:08 2015 +0530

    afr: launch index heal on local subvols up on a child-up event

    Problem:
    When a replica's child goes down and comes up, the index heal is
    triggered only on the child that just came up. This does not serve the
    intended purpose as the list of files that need to be healed
    to this child is actually captured on the other child of the replica.

    Fix:
    Launch index-heal on all local children of the replica xlator which just
    received a child up. Note that afr_selfheal_childup() eventually calls
    afr_shd_index_healer() which will not run the heal on non-local
    children.

    Signed-off-by: Ravishankar N <ravishankar at redhat.com>

    Change-Id: Ia23e47d197f983c695ec0bcd283e74931119ee55
    BUG: 1253309
    Reviewed-on: http://review.gluster.org/11912
    Tested-by: NetBSD Build System <jenkins at build.gluster.org>
    Tested-by: Gluster Build System <jenkins at build.gluster.com>
    Reviewed-by: Krutika Dhananjay <kdhananj at redhat.com>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu at redhat.com>

Referenced Bugs:

https://bugzilla.redhat.com/show_bug.cgi?id=1223636
[Bug 1223636] 3.1 QE Tracker
https://bugzilla.redhat.com/show_bug.cgi?id=1239021
[Bug 1239021] AFR: gluster v restart force or brick process restart doesn't
heal the files
https://bugzilla.redhat.com/show_bug.cgi?id=1251815
[Bug 1251815] [TRACKER] RHGS 3.1 Update 1 (z-stream) Tracker BZ
https://bugzilla.redhat.com/show_bug.cgi?id=1253309
[Bug 1253309] AFR: gluster v restart force or brick process restart doesn't
heal the files
https://bugzilla.redhat.com/show_bug.cgi?id=1255690
[Bug 1255690] AFR: gluster v restart force or brick process restart doesn't
heal the files
-- 
You are receiving this mail because:
You are on the CC list for the bug.
You are the assignee for the bug.