[Gluster-devel] [RFC] FUSE bridge based on GlusterFS API

[Jeff Darcy]

This looks like an interesting direction, and an amazing piece of work.
Some of my response is probably going to seem defensive or critical, so
I just wanted to get that out there first.

> The rationale for this is that FUSE bridge currently present in
> GlusterFS code does not use GlusterFS API, and that is considered to
> be wrong, and there are some plans to replace it with modern solution.
> xglfs code could potentially go under glusterfs tree if the developers
> decide it should happen. Also, it could be rpm'ed and suggested to
> Fedora users.

"Considered wrong" might be overstating the case.  It might be useful to
keep in mind that the fuse-bridge code predates GFAPI by a considerable
amount.  In fact, significant parts of GFAPI were borrowed from the
existing fuse-bridge code.  At the time, there was a choice between
FUSE's path-based and inode-based APIs.  Using the path-based API has
always been easier, requiring less code.  We can see this effect in the
fact that xglfs is a little less than 1/3 as code as fuse-bridge.

On the other hand, and again at the time, there were some pretty good
reasons to eschew the path-based API.  I don't remember all of those
reasons (some of them predate even my involvement with the project) but
I'm pretty sure performance was chief among them.  The path-based API is
completely synchronous, which would have been utterly disastrous for
performance prior to syncops (which of course came later).  Even with
syncops, it's not clear whether that gap has been or can be closed.  If
we ever wanted to consider switching fully to the path-based API, we'd
certainly need to examine that issue closely.  Other issues that
differentiate the two APIs might include:

  * Access to unlinked files (which have no path).

  * Levels of control over various forms of caching.

  * Ability to use reverse invalidation.

  * Ability to support SELinux (which does nasty stuff during mount).

  * Other ops that might be present only in the inode API.

  * Security.

Perhaps we should ping Miklos Szeredi (kernel FUSE maintainer, now at
Red Hat) about some of these.

> Some things remain not so clear to me:
> 
> * FUSE .bmap fop (wtf is that?);
> * trickery with .fgetattr (do we need that trickery?);

Not sure what you mean here.  Do you mean fgetxattr?

> * .flush fop (no GlusterFS equivalent?);
> * fsync/fdatasync difference for GlusterFS;
> * .fsyncdir fop (again, wtf?);

I suspect these are related to the path-based vs. inode-based issue.
Fact is, the VFS calls and syscalls have never lined up entirely, and it
shows up in differences like these.

> * WHERE IS MY glfs_truncate()?

That just seems like a bug.  There should be one.