[Bugs] [Bug 1732776] New: I/O error on writes to a disperse volume when replace-brick is executed

[bugzilla at redhat.com]

https://bugzilla.redhat.com/show_bug.cgi?id=1732776

            Bug ID: 1732776
           Summary: I/O error on writes to a disperse volume when
                    replace-brick is executed
           Product: Red Hat Gluster Storage
           Version: rhgs-3.4
            Status: NEW
         Component: disperse
          Severity: high
          Assignee: aspandey at redhat.com
          Reporter: aspandey at redhat.com
        QA Contact: nchilaka at redhat.com
                CC: bugs at gluster.org, jahernan at redhat.com,
                    rhs-bugs at redhat.com, sankarshan at redhat.com,
                    storage-qa-internal at redhat.com
        Depends On: 1699866
            Blocks: 1699917
  Target Milestone: ---
    Classification: Red Hat



+++ This bug was initially created as a clone of Bug #1699866 +++

Description of problem:

An I/O error happens when files are being created and written to a disperse
volume when a replace-brick is executed.

Version-Release number of selected component (if applicable): mainline

How reproducible:

Always


Steps to Reproduce:
1. Create a disperse volume
2. Kill one brick
3. Open fd on a subdirectory
4. Do a replace brick of the killed brick
5. Write on the previous file

Actual results:

The write fails with I/O error

Expected results:

The write should succeed

Additional info:

--- Additional comment from Xavi Hernandez on 2019-04-15 12:22:58 UTC ---

The problem happens because a reopen is attempted on all available bricks and
any error it finds is propagated to the main fop.

Basically, when a write fop is sent and ec discovers that there's a brick that
has come up again but doesn't have the fd open, it tries to open it. It could
happen that the file was created when the brick was down and self-heal has not
yet recovered it. In this case the open will fail with ENOENT. This should be
ok, since the other bricks are perfectly fine to successfully process the write
with enough quorum, but this error is not ignored and it's propagated to the
main fop, causing it to fail even before attempting the write.

--- Additional comment from Worker Ant on 2019-04-15 12:24:52 UTC ---

REVIEW: https://review.gluster.org/22558 (cluster/ec: fix fd reopen) posted
(#2) for review on master by Xavi Hernandez

--- Additional comment from Worker Ant on 2019-04-16 09:31:09 UTC ---

REVIEW: https://review.gluster.org/22574 (tests: Heal should fail when
read/write fails) merged (#2) on master by Pranith Kumar Karampuri

--- Additional comment from Worker Ant on 2019-04-23 11:29:39 UTC ---

REVIEW: https://review.gluster.org/22558 (cluster/ec: fix fd reopen) merged
(#7) on master by Pranith Kumar Karampuri


Referenced Bugs:

https://bugzilla.redhat.com/show_bug.cgi?id=1699866
[Bug 1699866] I/O error on writes to a disperse volume when replace-brick is
executed
https://bugzilla.redhat.com/show_bug.cgi?id=1699917
[Bug 1699917] I/O error on writes to a disperse volume when replace-brick is
executed
-- 
You are receiving this mail because:
You are on the CC list for the bug.