[Gluster-devel] opendir/readdir helper

[Brian Foster]

On 02/01/2013 08:27 AM, Jeff Darcy wrote:
> As we all know, directory-listing performance (or lack thereof) is a bit
> of a sore spot for many GlusterFS users, because it's one of the few
> places where FUSE really does make a difference. 

Hey Jeff,

Just an update on this topic... I slapped some prototype code together
to play around with directory prefetching [1] and ran a few basic tests.
The results could probably still be considered slow, but there is a nice
bump in performance. I'm seeing roughly 50% improvement in single large
directory read tests (numbers at [1]). I think this is mostly from using
larger requests.

There are still some interesting tests to run (I'd like to try Avati's
updates to support native fuse readdirp, compare results from increasing
the fuse readdirp buffer, and perhaps create a libgfapi driver to factor
the fuse overhead), but wanted throw this out there for reference.
Thoughts and ideas appreciated.

Brian

[1] - http://review.gluster.org/4519

 It will probably
> always be a sore spot even with the readdirp changes that are already
> under way.  The next step would be to add a FUSE enhancement to "inject"
> directory entries before they're requested, but that's a lot of work for
> an uncertain outcome.  The FUSE haters among the kernel leadership would
> probably reject such changes without serious consideration, and even if
> I'm wrong about that it's likely to be a long time before they make it
> into the various distributions (not to mention non-Linux platform issues).
> 
> So, let's think outside the box for a bit.  What about an LD_PRELOAD
> helper?  Believe me, I know all about the problems with LD_PRELOAD, but
> I still can't think of any reasonable use case that requires readdir to
> work across a fork (for example).  The basic idea is that the LD_PRELOAD
> would catch calls to opendir/readdir and match them against paths
> matching GlusterFS volumes.  If a match is found, then it would use
> libgfapi to serve the results, without any FUSE involvement and with
> massive prefetching goodness etc.  Without a match, the helper would
> naturally fall back to the system functions.
> 
> I suspect that this approach would make listings on very large single
> directories many times faster than would ever be possible with FUSE. For
> deeply nested directories we'd need to add some more complexity so that
> we're not going through the whole connection-establishment path
> (including authentication etc.) for each directory separately, but
> that's all pretty well understood pain for pretty obvious gain.
> 
> Any other thoughts?
> 
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> https://lists.nongnu.org/mailman/listinfo/gluster-devel