<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p><br>
</p>
<br>
<div class="moz-cite-prefix">On 07/25/2018 09:06 AM, Raghavendra
Gowdappa wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAFkORY92MpGPTHvoOFg086rro30Sv1cApXzStyBZC79rfgk7pQ@mail.gmail.com">
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Tue, Jul 24, 2018 at 6:54 PM,
Ravishankar N <span dir="ltr"><<a
href="mailto:ravishankar@redhat.com" target="_blank"
moz-do-not-send="true">ravishankar@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"><span class="">
<p><br>
</p>
<br>
<div class="m_-3667789299069000167moz-cite-prefix">On
07/24/2018 06:30 PM, Ravishankar N wrote:<br>
</div>
<blockquote type="cite">
<p><br>
</p>
<br>
<div class="m_-3667789299069000167moz-cite-prefix">On
07/24/2018 02:56 PM, Raghavendra Gowdappa wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>All,<br>
<br>
</div>
I was trying to debug regression failures on [1]
and observed that split-brain-resolution.t was
failing consistently.<br>
<div>
<div>
<div><br>
=========================<br>
TEST 45 (line 88): 0
get_pending_heal_count patchy<br>
./tests/basic/afr/split-brain-<wbr>resolution.t
.. 45/45 RESULT 45: 1<br>
./tests/basic/afr/split-brain-<wbr>resolution.t
.. Failed 17/45 subtests <br>
<br>
Test Summary Report<br>
-------------------<br>
./tests/basic/afr/split-brain-<wbr>resolution.t
(Wstat: 0 Tests: 45 Failed: 17)<br>
Failed tests: 24-26, 28-36, 41-45<br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>On probing deeper, I observed a curious
fact - on most of the failures stat was
not served from md-cache, but instead was
wound down to afr which failed stat with
EIO as the file was in split brain. So, I
did another test:</div>
<div>* disabled md-cache</div>
<div>* mount glusterfs with
attribute-timeout 0 and entry-timeout 0</div>
<div><br>
</div>
<div>Now the test fails always. So, I think
the test relied on stat requests being
absorbed either by kernel attribute cache
or md-cache. When its not happening stats
are reaching afr and resulting in failures
of cmds like getfattr etc. </div>
</div>
</div>
</div>
</blockquote>
<br>
This indeed seems to be the case. Is there any way
we can avoid the stat? When a getfattr is performed
on the mount, aren't lookup + getfattr are the only
fops that need to be hit in gluster? <br>
</blockquote>
<br>
</span> Or should afr allow (f)stat even for replica-2
split-brains because it is allowing lookup anyway
(lookup cbk contains stat information from one of its
children) ?<br>
</div>
</blockquote>
<div><br>
</div>
<div>I think the question here should be what kind of access
we've to provide for files in split-brain. Once that
broader question is answered, we should think about what
fops come under those kinds of access. If
setfattr/getfattr cmd access has to be provided I guess
lookup, stat, setxattr, getxattr need to work with
split-brain files.<br>
</div>
</div>
</div>
</div>
</blockquote>
<br>
Ideally, the only fop that should be allowed access is checking
whether the file exists or not (i.e. lookup), subject to quorum
checks. All others should be denied. This is how it works as of
today too but we (afr) overloaded setfattr and getfattr with virtual
xattrs to allow examining and resolving split-brain from the mount,
which is now failing in the .t because of the stat failing like you
pointed out. I think we should allow (f)stat too for replica-2 case
even when there are no good copies (i.e. read_subvol) to support the
mount based split-brain resolution method. Pranith, what do you
think?<br>
<br>
-Ravi<br>
<br>
<br>
<blockquote type="cite"
cite="mid:CAFkORY92MpGPTHvoOFg086rro30Sv1cApXzStyBZC79rfgk7pQ@mail.gmail.com">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div> <br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"> -Ravi<span class=""><br>
<blockquote type="cite"> -Ravi<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div>
<div>
<div>Thoughts?<br>
</div>
<div><br>
</div>
<div>[1] <a
href="https://review.gluster.org/#/c/20549/"
target="_blank" moz-do-not-send="true">https://review.gluster.org/#/<wbr>c/20549/</a><br>
</div>
</div>
</div>
</div>
<br>
<fieldset
class="m_-3667789299069000167mimeAttachmentHeader"></fieldset>
<br>
<pre>______________________________<wbr>_________________
Gluster-devel mailing list
<a class="m_-3667789299069000167moz-txt-link-abbreviated" href="mailto:Gluster-devel@gluster.org" target="_blank" moz-do-not-send="true">Gluster-devel@gluster.org</a>
<a class="m_-3667789299069000167moz-txt-link-freetext" href="https://lists.gluster.org/mailman/listinfo/gluster-devel" target="_blank" moz-do-not-send="true">https://lists.gluster.org/<wbr>mailman/listinfo/gluster-devel</a></pre>
</blockquote>
<br>
<br>
<fieldset
class="m_-3667789299069000167mimeAttachmentHeader"></fieldset>
<br>
<pre>______________________________<wbr>_________________
Gluster-devel mailing list
<a class="m_-3667789299069000167moz-txt-link-abbreviated" href="mailto:Gluster-devel@gluster.org" target="_blank" moz-do-not-send="true">Gluster-devel@gluster.org</a>
<a class="m_-3667789299069000167moz-txt-link-freetext" href="https://lists.gluster.org/mailman/listinfo/gluster-devel" target="_blank" moz-do-not-send="true">https://lists.gluster.org/<wbr>mailman/listinfo/gluster-devel</a></pre>
</blockquote>
<br>
</span></div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<br>
</body>
</html>