<div dir="ltr"><div>AFR had similar problem 2 years back and we started persisting afr children and graph always gets generated with what the xattrs will be in that order as an option. You can check afr_pending_xattrs_init() for it. On the glusterd side we give an identifer for each of the bricks.<br><br>The main reason we did this is because we didn't want to change this logic if new xlators get introduced in between client xlator and afr. What will you do if you add one more xlator between readdir-ahead and dht in future?<br><br></div><div>Now that I am seeing this mail and also Facebook guys also expressed that there should be a way for gluster to identify if it is talking to right bricks recently when they met us. May be we should let each xlator choose an identifier for its subvolumes and persist it on the bricks and use that instead of depending on mgmt to do the right things.<br><br></div><div>I understand this patch is already solving your problem. I see that it is merged as well but it is not future proof.<br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, May 17, 2017 at 10:03 AM, Raghavendra Gowdappa <span dir="ltr"><<a href="mailto:rgowdapp@redhat.com" target="_blank">rgowdapp@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">dht: Add readdir-ahead in rebalance graph if parallel-readdir is on<br>
<br>
Issue:<br>
The value of linkto xattr is generally the name of the dht's<br>
next subvol, this requires that the next subvol of dht is not<br>
changed for the life time of the volume. But with parallel<br>
readdir enabled, the readdir-ahead loaded below dht, is optional.<br>
The linkto xattr for first subvol, when:<br>
- parallel readdir is enabled : "<volname>-readdir-head-0"<br>
- plain distribute volume : "<volname>-client-0"<br>
- distribute replicate volume : "<volname>-afr-0"<br>
<br>
The value of linkto xattr is "<volname>-readdir-head-0" when<br>
parallel readdir is enabled, and is "<volname>-client-0" if<br>
its disabled. But the dht_lookup takes care of healing if it<br>
cannot identify which linkto subvol, the xattr points to.<br>
<br>
In dht_lookup_cbk, if linkto xattr is found to be "<volname>-client-0"<br>
and parallel readdir is enabled, then it cannot understand the<br>
value "<volname>-client-0" as it expects "<volname>-readdir-head-0".<br>
In that case, dht_lookup_everywhere is issued and then the linkto file<br>
is unlinked and recreated with the right linkto xattr. The issue is<br>
when parallel readdir is enabled, mount point accesses the file<br>
that is currently being migrated. Since rebalance process doesn't<br>
have parallel-readdir feature, it expects "<volname>-client-0"<br>
where as mount expects "<volname>-readdir-head-0". Thus at some point<br>
either the mount or rebalance will fail.<br>
<br>
Solution:<br>
Enable parallel-readdir for rebalance as well and then do not<br>
allow enabling/disabling parallel-readdir if rebalance is in<br>
progress.<br>
<br>
Change-Id: I241ab966bdd850e667f7768840540<wbr>546f5289483<br>
BUG: 1436090<br>
Signed-off-by: Poornima G <<a href="mailto:pgurusid@redhat.com">pgurusid@redhat.com</a>><br>
Reviewed-on: <a href="https://review.gluster.org/17056" rel="noreferrer" target="_blank">https://review.gluster.org/<wbr>17056</a><br>
Smoke: Gluster Build System <<a href="mailto:jenkins@build.gluster.org">jenkins@build.gluster.org</a>><br>
NetBSD-regression: NetBSD Build System <<a href="mailto:jenkins@build.gluster.org">jenkins@build.gluster.org</a>><br>
CentOS-regression: Gluster Build System <<a href="mailto:jenkins@build.gluster.org">jenkins@build.gluster.org</a>><br>
<br>
<br>
Patch: <a href="https://review.gluster.org/17056" rel="noreferrer" target="_blank">https://review.gluster.org/<wbr>17056</a><br>
owners: Me and Poornima<br>
<br>
PS: readdir-ahead is loaded in rebalance graph on top of each subvolume of DHT.<br>
<br>
regards,<br>
Raghavendra<br>
______________________________<wbr>_________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a><br>
<a href="http://lists.gluster.org/mailman/listinfo/gluster-devel" rel="noreferrer" target="_blank">http://lists.gluster.org/<wbr>mailman/listinfo/gluster-devel</a><br>
</blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">Pranith<br></div></div>
</div>