[Gluster-users] Stale file handle with FUSE client

Thu Mar 13 08:48:57 UTC 2014

----- Original Message -----
> From: "Alan Orth" <alan.orth at gmail.com>
> To: "Pranith Kumar Karampuri" <pkarampu at redhat.com>
> Cc: "gluster-users" <gluster-users at gluster.org>
> Sent: Thursday, March 13, 2014 2:13:26 PM
> Subject: Re: [Gluster-users] Stale file handle with FUSE client
> 
> Hi, Pranith.
> 
> Let me re-mount the volumes on my nodes using use-readdirp=no and do
> some more testing.
> 
> Also, one of the suggestions in that bug thread was to drop caches.  I
> did this on one of the clients in question and then I was able to access
> the file immediately (in this case, exit, then `ssh -X` and .Xauthority
> was created properly)...

That is good news. Then most probably it is that bug. The problem was appearing because of a bug in fuse-kernel module in 6.4's kernel. I guess it is fixed in 6.5's kernel.

Pranith

> 
> Thanks,
> 
> Alan
> 
> On 03/13/2014 10:57 AM, Pranith Kumar Karampuri wrote:
> > Alan,
> >    Could you check if you can re-create the issue with use-readdirp=no
> >    option for the fuse mount?
> >
> > Pranith
> > ----- Original Message -----
> >> From: "Pranith Kumar Karampuri" <pkarampu at redhat.com>
> >> To: "Alan Orth" <alan.orth at gmail.com>
> >> Cc: "gluster-users" <gluster-users at gluster.org>
> >> Sent: Thursday, March 13, 2014 1:23:02 PM
> >> Subject: Re: [Gluster-users] Stale file handle with FUSE client
> >>
> >> Alan,
> >>    I could be wrong, but the issue looks so much like the following bug?
> >> https://bugzilla.redhat.com/show_bug.cgi?id=1041109
> >>
> >> Pranith
> >> ----- Original Message -----
> >>> From: "Alan Orth" <alan.orth at gmail.com>
> >>> To: "gluster-users" <gluster-users at gluster.org>
> >>> Sent: Thursday, March 13, 2014 1:04:43 PM
> >>> Subject: Re: [Gluster-users] Stale file handle with FUSE client
> >>>
> >>> Another invocation of this issue:
> >>>
> >>> E138: Can't write viminfo file /home/aorth/.viminfo!
> >>>
> >>> This happened after using vim on several machines which share the same
> >>> networked /home directory. Usage wasn't concurrent, but within 10 seconds
> >>> or
> >>> so of each other. The client's volume log looks like this:
> >>>
> >>> [2014-03-13 07:31:59.961040] W [client-rpc-fops.c:526:client3_3_stat_cbk]
> >>> 0-homes-client-0: remote operation failed: No such file or directory
> >>> [2014-03-13 07:31:59.963245] W [client-rpc-fops.c:471:client3_3_open_cbk]
> >>> 0-homes-client-0: remote operation failed: No such file or directory.
> >>> Path:
> >>> /aorth/.viminfo (0f674308-6948-4383-bce7-9598a153d837)
> >>> [2014-03-13 07:31:59.963450] W [client-rpc-fops.c:471:client3_3_open_cbk]
> >>> 0-homes-client-1: remote operation failed: No such file or directory.
> >>> Path:
> >>> /aorth/.viminfo (0f674308-6948-4383-bce7-9598a153d837)
> >>> [2014-03-13 07:31:59.964873] W [fuse-bridge.c:915:fuse_fd_cbk]
> >>> 0-glusterfs-fuse: 72528783: OPEN() /aorth/.viminfo => -1 (No such file or
> >>> directory)
> >>>
> >>> But of course the file exists:
> >>>
> >>> $ ls -l .viminfo
> >>> -rw-------. 1 aorth aorth 12002 Mar 13 10:29 .viminfo
> >>>
> >>> Any help would be appreciated!
> >>>
> >>> Alan
> >>>
> >>> On 03/12/2014 02:38 PM, Alan Orth wrote:
> >>>
> >>>
> >>> Hi, all.
> >>>
> >>> I am having a problem on my replicated setup where files which are
> >>> commonly
> >>> accessed from different clients (such as ~/.Xauthority) are returning
> >>> "Stale
> >>> file handle" warning and subsequent errors. Access isn't necessarily
> >>> concurrent, but within a minute or two.
> >>>
> >>> In this case, trying to log in with `ssh -X` to a few compute nodes in
> >>> our
> >>> cluster, after the second or third time I see these in the client's log
> >>> file:
> >>>
> >>> [2014-03-07 09:19:04.187240] W
> >>> [client-rpc-fops.c:2624:client3_3_lookup_cbk]
> >>> 0-homes-client-0: remote operation failed: Stale file handle. Path:
> >>> /aorth/.Xauthority (79f4688e-da1b-4c0e-ae45-ac339f09d581)
> >>> [2014-03-07 09:19:04.187585] W
> >>> [client-rpc-fops.c:2624:client3_3_lookup_cbk]
> >>> 0-homes-client-1: remote operation failed: Stale file handle. Path:
> >>> /aorth/.Xauthority (79f4688e-da1b-4c0e-ae45-ac339f09d581)
> >>> [2014-03-07 09:19:21.820247] W
> >>> [client-rpc-fops.c:2624:client3_3_lookup_cbk]
> >>> 0-homes-client-0: remote operation failed: Stale file handle. Path:
> >>> /aorth/.Xauthority (d5bfeb33-786f-45ca-a533-a23cf0c54216)
> >>> [2014-03-07 09:19:21.820410] W
> >>> [client-rpc-fops.c:2624:client3_3_lookup_cbk]
> >>> 0-homes-client-1: remote operation failed: Stale file handle. Path:
> >>> /aorth/.Xauthority (d5bfeb33-786f-45ca-a533-a23cf0c54216)
> >>> [2014-03-07 09:27:02.426186] W [client-rpc-fops.c:471:client3_3_open_cbk]
> >>> 0-homes-client-0: remote operation failed: No such file or directory.
> >>> Path:
> >>> /aorth/.Xauthority (323ef093-1e18-463b-8544-0faf8b631985)
> >>> [2014-03-07 09:27:02.426517] W [client-rpc-fops.c:471:client3_3_open_cbk]
> >>> 0-homes-client-1: remote operation failed: No such file or directory.
> >>> Path:
> >>> /aorth/.Xauthority (323ef093-1e18-463b-8544-0faf8b631985)
> >>> [2014-03-07 09:27:02.428410] W [fuse-bridge.c:915:fuse_fd_cbk]
> >>> 0-glusterfs-fuse: 69684128: OPEN() /aorth/.Xauthority => -1 (No such file
> >>> or
> >>> directory)
> >>>
> >>> And I've seen the issue happen also when accessing other shared files
> >>> within
> >>> small time spans, like ~/.cpan/CPAN/MyConfig.pm. Other times I don't see
> >>> any
> >>> entries in the log file at all...
> >>>
> >>> The volume in question (homes) is configured like this:
> >>>
> >>>
> >>> # gluster volume info homes
> >>>
> >>> Volume Name: homes
> >>> Type: Replicate
> >>> Volume ID: 38637fa1-7ef3-4e8d-ace9-8ff81bc8bed9
> >>> Status: Started
> >>> Number of Bricks: 1 x 2 = 2
> >>> Transport-type: tcp
> >>> Bricks:
> >>> Brick1: storage0:/mnt/gfs/storage0/sda1/homes
> >>> Brick2: storage1:/mnt/gfs/storage1/sdb1/homes
> >>> Options Reconfigured:
> >>> nfs.disable: on
> >>> Clients and servers are all running GlusterFS 3.4.2 on CentOS 6.4/5.
> >>>
> >>> Thanks,
> >>> --
> >>> Alan Orth alan.orth at gmail.com http://alaninkenya.org http://mjanja.co.ke
> >>> "I
> >>> have always wished for my computer to be as easy to use as my telephone;
> >>> my
> >>> wish has come true because I can no longer figure out how to use my
> >>> telephone." -Bjarne Stroustrup, inventor of C++
> >>> GPG public key ID: 0x8cb0d0acb5cd81ec209c6cdfbd1a0e09c2f836c0
> >>>
> >>> --
> >>> Alan Orth alan.orth at gmail.com http://alaninkenya.org http://mjanja.co.ke
> >>> "I
> >>> have always wished for my computer to be as easy to use as my telephone;
> >>> my
> >>> wish has come true because I can no longer figure out how to use my
> >>> telephone." -Bjarne Stroustrup, inventor of C++
> >>> GPG public key ID: 0x8cb0d0acb5cd81ec209c6cdfbd1a0e09c2f836c0
> >>>
> >>> _______________________________________________
> >>> Gluster-users mailing list
> >>> Gluster-users at gluster.org
> >>> http://supercolony.gluster.org/mailman/listinfo/gluster-users
> 
> --
> Alan Orth
> alan.orth at gmail.com
> http://alaninkenya.org
> http://mjanja.co.ke
> "I have always wished for my computer to be as easy to use as my telephone;
> my wish has come true because I can no longer figure out how to use my
> telephone." -Bjarne Stroustrup, inventor of C++
> GPG public key ID: 0x8cb0d0acb5cd81ec209c6cdfbd1a0e09c2f836c0
> 
> 
> 