[Gluster-devel] Disabling use of anonymous fds in open-behind

[Raghavendra Gowdappa]

On Wed, Jun 13, 2018 at 8:00 AM, Raghavendra Gowdappa <rgowdapp at redhat.com>
wrote:

>
>
> On Wed, Jun 13, 2018 at 7:21 AM, Poornima Gurusiddaiah <
> pgurusid at redhat.com> wrote:
>
>>
>>
>> On Wed, Jun 13, 2018, 5:22 AM Vijay Bellur <vbellur at redhat.com> wrote:
>>
>>>
>>>
>>> On Mon, Jun 11, 2018 at 7:44 PM, Raghavendra Gowdappa <
>>> rgowdapp at redhat.com> wrote:
>>>
>>>> All,
>>>>
>>>> This is an option in open-behind, which lets fuse native mounts to use
>>>> anonymous fds. The reasoning being since anonymous fds are stateless,
>>>> overhead of open is avoided and hence better performance. However, bugs
>>>> filed [1][2] seemed to indicate contrary results.
>>>>
>>>> Also, using anonymous fds affects other xlators which rely on per fd
>>>> state [3].
>>>>
>>>> So, this brings to the point do anonymous-fds actually improve
>>>> performance on native fuse mounts? If not, we can disable them. May be they
>>>> are useful for light weight metadata operations like fstat, but the
>>>> workload should only be limited to them. Note that anonymous fds are used
>>>> by open-behind by only two fops - readv and fstat. But, [1] has shown that
>>>> they actually regress performance for sequential reads.
>>>>
>>>
>>>
>>> Perhaps a more intelligent open-behind based on size could help? IIRC,
>>> open-behind was originally developed to improve latency for small file
>>> operations.
>>>
>>
> It looks like Quick-read accounts to larger share of performance impact
> when compared to open-behind. Milind is scheduling some runs with the
> combination of "open-behind off, quick-read on" to asses the actual perf
> impact of open-behind for the workload of small file reads. I hope we'll
> have enough data to arrive at usefulness of open-behind by then.
>

The test data will take some time as other high priority tasks pre-empted
this effort.


> For large files, it is unnecessary and can affect read-ahead behavior as
>>> observed in the referenced bugs. Could we alter the behavior to disable
>>> open-behind for those files which are bigger than a configurable size
>>> threshold?
>>>
>> +1, this sounds like a perfect solution which doesn't give out the
>> benefits (may be in few cases) but also doesn't reduce the performance in
>> small file read. We could enable open behind only for fd with rd-only, and
>> if the size is less than or equal to the quick-read file size.
>>
>
> Yes, that's one solution we thought off too (aligning open-behind's
> decision on quick-read size). But a slight variation of this approach
> (opens early for files of size > 256KB and tested on files of size GBs) has
> been tried and didn't yield expected results [1].
>
> [1] https://review.gluster.org/#/c/17377/
>
>
>> Regards,
>> Poornima
>>
>>
>>> Thanks,
>>> Vijay
>>>
>>>
>>>> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1419807
>>>> [2] https://bugzilla.redhat.com/1489513, "read-ahead underperrforms
>>>> expectations"
>>>>       open-behind without patch (MiB/s) with patch (MiB/s)
>>>>           on          132.87                133.51
>>>>           off         139.70                139.77
>>>>
>>>> [3] https://bugzilla.redhat.com/show_bug.cgi?id=1084508
>>>>
>>>> PS: Anonymous fds are stateless fds, where a client like native fuse
>>>> mount doesn't do an explicit open. Instead, bricks do the open on-demand
>>>> during fops which need an fd (like readv, fstat etc).
>>>>
>>>> regards,
>>>> Raghavendra
>>>>
>>>> _______________________________________________
>>>> Gluster-devel mailing list
>>>> Gluster-devel at gluster.org
>>>> http://lists.gluster.org/mailman/listinfo/gluster-devel
>>>>
>>>
>>> _______________________________________________
>>> Gluster-devel mailing list
>>> Gluster-devel at gluster.org
>>> http://lists.gluster.org/mailman/listinfo/gluster-devel
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-devel/attachments/20180615/c44f4e6e/attachment.html>