[Gluster-devel] Disabling use of anonymous fds in open-behind

[Raghavendra Gowdappa]

On Wed, Jun 13, 2018 at 7:21 AM, Poornima Gurusiddaiah <pgurusid at redhat.com>
wrote:

>
>
> On Wed, Jun 13, 2018, 5:22 AM Vijay Bellur <vbellur at redhat.com> wrote:
>
>>
>>
>> On Mon, Jun 11, 2018 at 7:44 PM, Raghavendra Gowdappa <
>> rgowdapp at redhat.com> wrote:
>>
>>> All,
>>>
>>> This is an option in open-behind, which lets fuse native mounts to use
>>> anonymous fds. The reasoning being since anonymous fds are stateless,
>>> overhead of open is avoided and hence better performance. However, bugs
>>> filed [1][2] seemed to indicate contrary results.
>>>
>>> Also, using anonymous fds affects other xlators which rely on per fd
>>> state [3].
>>>
>>> So, this brings to the point do anonymous-fds actually improve
>>> performance on native fuse mounts? If not, we can disable them. May be they
>>> are useful for light weight metadata operations like fstat, but the
>>> workload should only be limited to them. Note that anonymous fds are used
>>> by open-behind by only two fops - readv and fstat. But, [1] has shown that
>>> they actually regress performance for sequential reads.
>>>
>>
>>
>> Perhaps a more intelligent open-behind based on size could help? IIRC,
>> open-behind was originally developed to improve latency for small file
>> operations.
>>
>
It looks like Quick-read accounts to larger share of performance impact
when compared to open-behind. Milind is scheduling some runs with the
combination of "open-behind off, quick-read on" to asses the actual perf
impact of open-behind for the workload of small file reads. I hope we'll
have enough data to arrive at usefulness of open-behind by then.

For large files, it is unnecessary and can affect read-ahead behavior as
>> observed in the referenced bugs. Could we alter the behavior to disable
>> open-behind for those files which are bigger than a configurable size
>> threshold?
>>
> +1, this sounds like a perfect solution which doesn't give out the
> benefits (may be in few cases) but also doesn't reduce the performance in
> small file read. We could enable open behind only for fd with rd-only, and
> if the size is less than or equal to the quick-read file size.
>

Yes, that's one solution we thought off too (aligning open-behind's
decision on quick-read size). But a slight variation of this approach
(opens early for files of size > 256KB and tested on files of size GBs) has
been tried and didn't yield expected results [1].

[1] https://review.gluster.org/#/c/17377/


> Regards,
> Poornima
>
>
>> Thanks,
>> Vijay
>>
>>
>>> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1419807
>>> [2] https://bugzilla.redhat.com/1489513, "read-ahead underperrforms
>>> expectations"
>>>       open-behind without patch (MiB/s) with patch (MiB/s)
>>>           on          132.87                133.51
>>>           off         139.70                139.77
>>>
>>> [3] https://bugzilla.redhat.com/show_bug.cgi?id=1084508
>>>
>>> PS: Anonymous fds are stateless fds, where a client like native fuse
>>> mount doesn't do an explicit open. Instead, bricks do the open on-demand
>>> during fops which need an fd (like readv, fstat etc).
>>>
>>> regards,
>>> Raghavendra
>>>
>>> _______________________________________________
>>> Gluster-devel mailing list
>>> Gluster-devel at gluster.org
>>> http://lists.gluster.org/mailman/listinfo/gluster-devel
>>>
>>
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at gluster.org
>> http://lists.gluster.org/mailman/listinfo/gluster-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-devel/attachments/20180613/054ed6b4/attachment.html>