<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jun 13, 2018 at 8:00 AM, Raghavendra Gowdappa <span dir="ltr"><<a href="mailto:rgowdapp@redhat.com" target="_blank">rgowdapp@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span class="">On Wed, Jun 13, 2018 at 7:21 AM, Poornima Gurusiddaiah <span dir="ltr"><<a href="mailto:pgurusid@redhat.com" target="_blank">pgurusid@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto"><span class="m_-617273402794290470gmail-"><div><br><br><div class="gmail_quote"><div dir="ltr">On Wed, Jun 13, 2018, 5:22 AM Vijay Bellur <<a href="mailto:vbellur@redhat.com" target="_blank">vbellur@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jun 11, 2018 at 7:44 PM, Raghavendra Gowdappa <span dir="ltr"><<a href="mailto:rgowdapp@redhat.com" rel="noreferrer" target="_blank">rgowdapp@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><div>All,<br></div><div><br></div>This is an option in open-behind, which lets fuse native mounts to use anonymous fds. The reasoning being since anonymous fds are stateless, overhead of open is avoided and hence better performance. However, bugs filed [1][2] seemed to indicate contrary results.<br><br>Also, using anonymous fds affects other xlators which rely on per fd state [3].<br><br>So, this brings to the point do anonymous-fds actually improve performance on native fuse mounts? If not, we can disable them. May be they are useful for light weight metadata operations like fstat, but the workload should only be limited to them. Note that anonymous fds are used by open-behind by only two fops - readv and fstat. But, [1] has shown that they actually regress performance for sequential reads.<br></div></div></div></blockquote><div><br></div><div><br></div><div>Perhaps a more intelligent open-behind based on size could help? IIRC, open-behind was originally developed to improve latency for small file operations.</div></div></div></div></blockquote></div></div></span></div></blockquote><div><br></div></span><div>It looks like Quick-read accounts to larger share of performance impact when compared to open-behind. Milind is scheduling some runs with the combination of "open-behind off, quick-read on" to asses the actual perf impact of open-behind for the workload of small file reads. I hope we'll have enough data to arrive at usefulness of open-behind by then.<br></div></div></div></div></blockquote><div><br></div><div>The test data will take some time as other high priority tasks pre-empted this effort.</div><div> <br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div></div><span class=""><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto"><span class="m_-617273402794290470gmail-"><div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div> For large files, it is unnecessary and can affect read-ahead behavior as observed in the referenced bugs. Could we alter the behavior to disable open-behind for those files which are bigger than a configurable size threshold?</div></div></div></div></blockquote></div></div></span><div dir="auto">+1, this sounds like a perfect solution which doesn't give out the benefits (may be in few cases) but also doesn't reduce the performance in small file read. We could enable open behind only for fd with rd-only, and if the size is less than or equal to the quick-read file size.</div></div></blockquote><div><br></div></span><div>Yes, that's one solution we thought off too (aligning open-behind's decision on quick-read size). But a slight variation of this approach (opens early for files of size > 256KB and tested on files of size GBs) has been tried and didn't yield expected results [1].</div><div><br></div><div>[1] <a href="https://review.gluster.org/#/c/17377/" target="_blank">https://review.gluster.org/#/<wbr>c/17377/</a></div><span class=""><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto"><div dir="auto"><br></div><div dir="auto">Regards,</div><div dir="auto">Poornima</div><span class="m_-617273402794290470gmail-"><div dir="auto"><br></div><div dir="auto"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><br></div><div>Thanks,</div><div>Vijay</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><br>[1] <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1419807" rel="noreferrer" target="_blank">https://bugzilla.redhat.com/sh<wbr>ow_bug.cgi?id=1419807</a><br>[2] <a href="https://bugzilla.redhat.com/1489513" rel="noreferrer" target="_blank">https://bugzilla.redhat.com/14<wbr>89513</a>, "read-ahead underperrforms expectations"<br> open-behind without patch (MiB/s) with patch (MiB/s)<br> on 132.87 133.51<br> off 139.70 139.77<br><br>[3] <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1084508" rel="noreferrer" target="_blank">https://bugzilla.redhat.com/sh<wbr>ow_bug.cgi?id=1084508</a><br></div><div><br></div><div>PS: Anonymous fds are stateless fds, where a client like native fuse mount doesn't do an explicit open. Instead, bricks do the open on-demand during fops which need an fd (like readv, fstat etc).<br></div><div><br></div>regards,<br></div>Raghavendra<br></div>
<br>______________________________<wbr>_________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@gluster.org" rel="noreferrer" target="_blank">Gluster-devel@gluster.org</a><br>
<a href="http://lists.gluster.org/mailman/listinfo/gluster-devel" rel="noreferrer noreferrer" target="_blank">http://lists.gluster.org/mailm<wbr>an/listinfo/gluster-devel</a><br></blockquote></div><br></div></div>
______________________________<wbr>_________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@gluster.org" rel="noreferrer" target="_blank">Gluster-devel@gluster.org</a><br>
<a href="http://lists.gluster.org/mailman/listinfo/gluster-devel" rel="noreferrer noreferrer" target="_blank">http://lists.gluster.org/mailm<wbr>an/listinfo/gluster-devel</a></blockquote></div></div></span></div>
</blockquote></span></div><br></div></div>
</blockquote></div><br></div></div>