[Gluster-devel] On making performance.parallel-readdir as a default option

[Raghavendra Gowdappa]

On Fri, Sep 21, 2018 at 11:25 PM Raghavendra Gowdappa <rgowdapp at redhat.com>
wrote:

> Hi all,
>
> We've a feature performance.parallel-readdir [1] that is known to improve
> performance of readdir operations [2][3][4]. The option is especially
> useful when distribute scale is relatively large (>10) and is known to
> improve performance of readdir operations even on smaller scale of
> distribute count 1 [4].
>
> However, this option is not enabled by default. I am here proposing to
> make this as a default feature.
>
> But, there are some important things to be addressed in readdir-ahead
> (which is core part of parallel-readdir), before we can do so:
>
> To summarize issues with readdir-ahead:
> * There seems to be one prominent problem of missing dentries with
> parallel-readdir. There was one problem discussed on tech-list just
> yesterday. I've heard about this recurrently earlier too. Not sure whether
> this is the problem of missing unlink/rmdir/create etc fops (see below) in
> readdir-ahead. ATM, no RCA.
> * fixes to maintain stat-consistency in dentries pre-fetched have not made
> into downstream yet (though merged upstream [5]).
> * readdir-ahead doesn't implement directory modification fops like
> rmdir/create/symlink/link/unlink/rename. This means cache won't be updated
> wiith newer content, even on single mount till its consumed by application
> or purged.
> * dht linkto-files should store relative positions of subvolumes instead
> of absolute subvolume name, so that changes to immediate child won't render
> them stale.
> * Features parallel-readdir depends on to be working should be enabled
> automatically even though they were off earlier when parallel-readdir is
> enabled [6].
>
> I've listed important known issues above. But we can discuss which are the
> blockers for making this feature as a default.
>
> Thoughts?
>
> [1] http://review.gluster.org/#/c/16090/
> [2]
> https://events.static.linuxfound.org/sites/events/files/slides/Gluster_DirPerf_Vault2017_0.pdf
> (sections on small directory)
> [3] https://bugzilla.redhat.com/show_bug.cgi?id=1628807#c35
> <https://bugzilla.redhat.com/show_bug.cgi?id=1628807#c34>
> [4] https://www.spinics.net/lists/gluster-users/msg34956.html
> [5] http://review.gluster.org/#/c/glusterfs/+/20639/
> [6] https://bugzilla.redhat.com/show_bug.cgi?id=1631406
>
> regards,
> Raghavendra
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-devel/attachments/20180921/12442018/attachment.html>