[Gluster-users] Passing noforget option to glusterfs native client mounts

[Anirban Ghoshal]

Thanks, Harshavardhana, Anand for the tips!

I checked out parts of the linux-2.6.34 (the one we are using down here) knfsd/fuse code. I understood (hopefully, rightly) that when we export a fuse directory over NFS and specify an fsid, the handle is constructed somewhat like this:

fh_size (4 bytes) - fh_version and stuff (4 bytes) - fsid, from export parms (4 bytes) - nodeid (8 bytes) - generation number (4 bytes) - parent nodeid (8 bytes) - parent generation (4 bytes).

So, since Anand mentions that nodeid's for glusterfs are just the inode_t addresses on servers, I can now relate to the fact that the file handles might not even survive failovers in any and every case, even with the fsid constant. 

That's why I was so confused.. I never faced an issue with stale file handles during failover yet! Maybe something to do with the order in which files were created on the replica server following heal commencement (our data is quite static btw) - like, if you malloc identical things on two identical platforms by running the same executable on each, you get allocations at the exact same virtual addresses...

However, now that I understand at least in part how this works, Glusterfs NFS does seem a lot cleaner... Will also try out the Ganesha... 


Thanks!


On Tuesday, 24 December 2013 11:04 PM, Harshavardhana <harsha at harshavardhana.net> wrote:
 
On Tue, Dec 24, 2013 at 8:21 AM, Anirban Ghoshal

<chalcogen_eg_oxygen at yahoo.com> wrote:
> Hi, and Thanks a lot, Anand!
>
> I was initially searching for a good answer to why the glusterfs site lists
> knfsd as NOT compatible with the glusterfs.  So, now I know. :)
>
> Funnily enough, we didn't have a problem with the failover during our
> testing. We passed constant fsid's (fsid=xxx) while exporting our mounts and
> NFS mounts on client applications haven't called any of the file handles out
> stale while migrating the NFS service from one server to the other. Not sure
> why this happpens.

Using fsid is just a workaround always used to solve ESTALE on file
handles. The device major/minor numbers are embedded in the NFS file
handle, a problem when an NFS export is failed over or moved to
another node during failover is that these numbers change when the
resource is exported on the new node resulting in client to see a
"Stale NFS file handle" error. We need to make sure the embedded
number stays the same that is where the fsid export option - allowing
us to specify a coherent number across various clients.

GlusterNFS server is way cleaner solution for such consistency.

Another thing would be to take the next step, give a go for
'NFS-Ganesha' and 'GlusterFS' integration?

https://forge.gluster.org/nfs-ganesha-and-glusterfs-integration
http://www.gluster.org/2013/09/gluster-ganesha-nfsv4-initial-impressions/

Cheers
-- 
Religious confuse piety with mere ritual, the virtuous confuse
regulation with outcomes
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131225/173d112e/attachment.html>