[Gluster-devel] Using kernel NFS with a gluster fuse mount

[Jacobson, Erik]

Dear team,

First, a thank as always for making the gluster storage solution.

The question:
Years back, when we first started using Gluster, we had a requirement to provide NFS services. Documentation and discussions at the time made us feel as if we should not use the kernel NFS server from a fuse mount but rather use Ganesha. Recently I have seen people are using Linux kernel NFS server with gluster fuse mounts. It has been working well in our test environment. What are the drawbacks to exporting a fuse-mounted glusterfs filesystem using the Linux NFS server?



Optional background reading for the question:
At the time, and still today, Ganesha doesn’t serve our specific workload fast enough (no disrespect to the great people working on Ganesha). We therefore used and continue to use Gluster NFS.

Recently, we have begun to have issues with gluster NFS mounts when the nfs performance io cache is enabled. Files under 200M or so, in certain situations and systems but not all systems, will get an IO error on the NFS mount. The fuse access is fine. Turning off the cache makes the server nearly unusable for our specific workload and scale. We may write about that separately but we’re behind in gluster version and know we need to get current before we ask for help.

This problem at an important site had us looking for workarounds. We are using Ganesha as the work-around, but it is significantly slower. Internal testing showed exporting with kernel NFS was very slick. Hence the question.

Gluster is used for several things but for NFS, it’s largely a collection of squashfs files that represent root filesystems that is the most load.

Thank you all!!!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-devel/attachments/20240310/10152f25/attachment.html>