[Gluster-devel] inode linking in GlusterFS NFS server

[Anand Avati]

On Mon, Jul 7, 2014 at 12:48 PM, Raghavendra Bhat <rabhat at redhat.com> wrote:

>
> Hi,
>
> As per my understanding nfs server is not doing inode linking in readdirp
> callback. Because of this there might be some errors while dealing with
> virtual inodes (or gfids). As of now meta, gfid-access and snapview-server
> (used for user serviceable snapshots) xlators makes use of virtual inodes
> with random gfids. The situation is this:
>
> Say User serviceable snapshot feature has been enabled and there are 2
> snapshots ("snap1" and "snap2"). Let /mnt/nfs be the nfs mount. Now the
> snapshots can be accessed by entering .snaps directory.  Now if snap1
> directory is entered and *ls -l* is done (i.e. "cd /mnt/nfs/.snaps/snap1"
> and then "ls -l"),  the readdirp fop is sent to the snapview-server xlator
> (which is part of a daemon running for the volume), which talks to the
> corresponding snapshot volume and gets the dentry list. Before unwinding it
> would have generated random gfids for those dentries.
>
> Now nfs server upon getting readdirp reply, will associate the gfid with
> the filehandle created for the entry. But without linking the inode, it
> would send the readdirp reply back to nfs client. Now next time when nfs
> client makes some operation on one of those filehandles, nfs server tries
> to resolve it by finding the inode for the gfid present in the filehandle.
> But since the inode was not linked in readdirp, inode_find operation fails
> and it tries to do a hard resolution by sending the lookup operation on
> that gfid to the normal main graph. (The information on whether the call
> should be sent to main graph or snapview-server would be present in the
> inode context. But here the lookup has come on a gfid with a newly created
> inode where the context is not there yet. So the call would be sent to the
> main graph itself). But since the gfid is a randomly generated virtual gfid
> (not present on disk), the lookup operation fails giving error.
>
> As per my understanding this can happen with any xlator that deals with
> virtual inodes (by generating random gfids).
>
> I can think of these 2 methods to handle this:
> 1)  do inode linking for readdirp also in nfs server
> 2)  If lookup operation fails, snapview-client xlator (which actually
> redirects the fops on snapshot world to snapview-server by looking into the
> inode context) should check if the failed lookup is a nameless lookup. If
> so, AND the gfid of the inode is NULL AND lookup has come from main graph,
> then instead of unwinding the lookup with failure, send it to
> snapview-server which might be able to find the inode for the gfid (as the
> gfid was generated by itself, it should be able to find the inode for that
> gfid unless and until it has been purged from the inode table).
>
>
> Please let me know if I have missed anything. Please provide feedback.
>
>

That's right. NFS server should be linking readdirp_cbk inodes just like
FUSE or protocol/server. It has been OK without virtual gfids thus far.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-devel/attachments/20140707/8fc24b6c/attachment-0001.html>