[Gluster-users] [Gluster-devel] [RFC] FUSE bridge based on GlusterFS API

[Oleksandr Natalenko]

On четвер, 7 квітня 2016 р. 16:12:07 EEST Jeff Darcy wrote:
> "Considered wrong" might be overstating the case.  It might be useful to
> keep in mind that the fuse-bridge code predates GFAPI by a considerable
> amount.  In fact, significant parts of GFAPI were borrowed from the
> existing fuse-bridge code.  At the time, there was a choice between
> FUSE's path-based and inode-based APIs.  Using the path-based API has
> always been easier, requiring less code.  We can see this effect in the
> fact that xglfs is a little less than 1/3 as code as fuse-bridge.

Sure, API came years later after FUSE bridge, and I understand that. Also, 
that explains to me some kind of code doubling, when FUSE bridge is 
(completely) separate entity from API doing pretty much the same (as I 
thought) thing.

> On the other hand, and again at the time, there were some pretty good
> reasons to eschew the path-based API.  I don't remember all of those
> reasons (some of them predate even my involvement with the project) but
> I'm pretty sure performance was chief among them.  The path-based API is
> completely synchronous, which would have been utterly disastrous for
> performance prior to syncops (which of course came later).  Even with
> syncops, it's not clear whether that gap has been or can be closed.  If
> we ever wanted to consider switching fully to the path-based API, we'd
> certainly need to examine that issue closely.  Other issues that
> differentiate the two APIs might include:
> 
>   * Access to unlinked files (which have no path).
> 
>   * Levels of control over various forms of caching.
> 
>   * Ability to use reverse invalidation.
> 
>   * Ability to support SELinux (which does nasty stuff during mount).
> 
>   * Other ops that might be present only in the inode API.
> 
>   * Security.
> 
> Perhaps we should ping Miklos Szeredi (kernel FUSE maintainer, now at
> Red Hat) about some of these.

Also, Soumya pointed me to handles API (/usr/include/glusterfs/api/glfs-
handles.h). If I got it correctly, probably, it could be used instead of path-
based API for FUSE bridge? I have briefly looked at it, but the article about 
NFS handles (again, supplied to me by Soumya) remains unread so far :). Does 
handles API represend inode API you are talking about? Then, also, we 
shouldn't use highlevel FUSE API and stick to lowlevel instead as it (AFAIK, 
correct me if I'm wrong) operates on inodes as well.

> > * FUSE .bmap fop (wtf is that?);
> > * trickery with .fgetattr (do we need that trickery?);
> 
> Not sure what you mean here.  Do you mean fgetxattr?

I mean .fgetattr calling .getattr for / and glfs_fstat() for everything else. 
Not sure why it happens. BBFS code says:

===
    // On FreeBSD, trying to do anything with the mountpoint ends up
    // opening it, and then using the FD for an fgetattr.  So in the
    // special case of a path of "/", I need to do a getattr on the
    // underlying root directory instead of doing the fgetattr().
===

Probably, I just wanted to note that (in case my toy could become portable 
across *nixes).

> 
> > * .flush fop (no GlusterFS equivalent?);
> > * fsync/fdatasync difference for GlusterFS;
> > * .fsyncdir fop (again, wtf?);
> 
> I suspect these are related to the path-based vs. inode-based issue.
> Fact is, the VFS calls and syscalls have never lined up entirely, and it
> shows up in differences like these.

I definitely need help with handles API if that is the right thing to address 
to.

> That just seems like a bug.  There should be one.

That is definitely the bug. /usr/include/glusterfs/api/glfs.h clearly defines 
it:

===
int glfs_truncate (glfs_t *fs, const char *path, off_t length) __THROW
        GFAPI_PUBLIC(glfs_truncate, 3.4.0);
===

But linking executable with a call to glfs_truncate() results in error:

===
CMakeFiles/xglfs.dir/xglfs_truncate.c.o: In function `xglfs_truncate':
/home/pf/work/devel/own/xglfs/xglfs_truncate.c:31: undefined reference to 
`glfs_truncate'
===

The bug was discussed more than year ago [1], but it seems there is no 
solution so far.

Thanks.

Regards,
  post-factum

[1] http://irclog.perlgeek.de/gluster-dev/2015-01-25/text