[Gluster-users] Stale file handle with FUSE client

Thu Mar 13 09:22:55 UTC 2014

Pranith,

Ok, I've re-mounted several of my clients with the 'use-readdirp=no'
mount option and I'll keep an eye on them.  Also good to hear about RHEL
6.5 kernel, as moving to RHEL 6.5 is something I will be doing on my
cluster clients as I see moments of opportunity (already upgraded a few).

Cheers, and I'll keep the list posted.

Alan

On 03/13/2014 11:48 AM, Pranith Kumar Karampuri wrote:
>
> ----- Original Message -----
>> From: "Alan Orth" <alan.orth at gmail.com>
>> To: "Pranith Kumar Karampuri" <pkarampu at redhat.com>
>> Cc: "gluster-users" <gluster-users at gluster.org>
>> Sent: Thursday, March 13, 2014 2:13:26 PM
>> Subject: Re: [Gluster-users] Stale file handle with FUSE client
>>
>> Hi, Pranith.
>>
>> Let me re-mount the volumes on my nodes using use-readdirp=no and do
>> some more testing.
>>
>> Also, one of the suggestions in that bug thread was to drop caches.  I
>> did this on one of the clients in question and then I was able to access
>> the file immediately (in this case, exit, then `ssh -X` and .Xauthority
>> was created properly)...
> That is good news. Then most probably it is that bug. The problem was appearing because of a bug in fuse-kernel module in 6.4's kernel. I guess it is fixed in 6.5's kernel.
>
> Pranith
>
>> Thanks,
>>
>> Alan
>>
>> On 03/13/2014 10:57 AM, Pranith Kumar Karampuri wrote:
>>> Alan,
>>>    Could you check if you can re-create the issue with use-readdirp=no
>>>    option for the fuse mount?
>>>
>>> Pranith
>>> ----- Original Message -----
>>>> From: "Pranith Kumar Karampuri" <pkarampu at redhat.com>
>>>> To: "Alan Orth" <alan.orth at gmail.com>
>>>> Cc: "gluster-users" <gluster-users at gluster.org>
>>>> Sent: Thursday, March 13, 2014 1:23:02 PM
>>>> Subject: Re: [Gluster-users] Stale file handle with FUSE client
>>>>
>>>> Alan,
>>>>    I could be wrong, but the issue looks so much like the following bug?
>>>> https://bugzilla.redhat.com/show_bug.cgi?id=1041109
>>>>
>>>> Pranith
>>>> ----- Original Message -----
>>>>> From: "Alan Orth" <alan.orth at gmail.com>
>>>>> To: "gluster-users" <gluster-users at gluster.org>
>>>>> Sent: Thursday, March 13, 2014 1:04:43 PM
>>>>> Subject: Re: [Gluster-users] Stale file handle with FUSE client
>>>>>
>>>>> Another invocation of this issue:
>>>>>
>>>>> E138: Can't write viminfo file /home/aorth/.viminfo!
>>>>>
>>>>> This happened after using vim on several machines which share the same
>>>>> networked /home directory. Usage wasn't concurrent, but within 10 seconds
>>>>> or
>>>>> so of each other. The client's volume log looks like this:
>>>>>
>>>>> [2014-03-13 07:31:59.961040] W [client-rpc-fops.c:526:client3_3_stat_cbk]
>>>>> 0-homes-client-0: remote operation failed: No such file or directory
>>>>> [2014-03-13 07:31:59.963245] W [client-rpc-fops.c:471:client3_3_open_cbk]
>>>>> 0-homes-client-0: remote operation failed: No such file or directory.
>>>>> Path:
>>>>> /aorth/.viminfo (0f674308-6948-4383-bce7-9598a153d837)
>>>>> [2014-03-13 07:31:59.963450] W [client-rpc-fops.c:471:client3_3_open_cbk]
>>>>> 0-homes-client-1: remote operation failed: No such file or directory.
>>>>> Path:
>>>>> /aorth/.viminfo (0f674308-6948-4383-bce7-9598a153d837)
>>>>> [2014-03-13 07:31:59.964873] W [fuse-bridge.c:915:fuse_fd_cbk]
>>>>> 0-glusterfs-fuse: 72528783: OPEN() /aorth/.viminfo => -1 (No such file or
>>>>> directory)
>>>>>
>>>>> But of course the file exists:
>>>>>
>>>>> $ ls -l .viminfo
>>>>> -rw-------. 1 aorth aorth 12002 Mar 13 10:29 .viminfo
>>>>>
>>>>> Any help would be appreciated!
>>>>>
>>>>> Alan
>>>>>
>>>>> On 03/12/2014 02:38 PM, Alan Orth wrote:
>>>>>
>>>>>
>>>>> Hi, all.
>>>>>
>>>>> I am having a problem on my replicated setup where files which are
>>>>> commonly
>>>>> accessed from different clients (such as ~/.Xauthority) are returning
>>>>> "Stale
>>>>> file handle" warning and subsequent errors. Access isn't necessarily
>>>>> concurrent, but within a minute or two.
>>>>>
>>>>> In this case, trying to log in with `ssh -X` to a few compute nodes in
>>>>> our
>>>>> cluster, after the second or third time I see these in the client's log
>>>>> file:
>>>>>
>>>>> [2014-03-07 09:19:04.187240] W
>>>>> [client-rpc-fops.c:2624:client3_3_lookup_cbk]
>>>>> 0-homes-client-0: remote operation failed: Stale file handle. Path:
>>>>> /aorth/.Xauthority (79f4688e-da1b-4c0e-ae45-ac339f09d581)
>>>>> [2014-03-07 09:19:04.187585] W
>>>>> [client-rpc-fops.c:2624:client3_3_lookup_cbk]
>>>>> 0-homes-client-1: remote operation failed: Stale file handle. Path:
>>>>> /aorth/.Xauthority (79f4688e-da1b-4c0e-ae45-ac339f09d581)
>>>>> [2014-03-07 09:19:21.820247] W
>>>>> [client-rpc-fops.c:2624:client3_3_lookup_cbk]
>>>>> 0-homes-client-0: remote operation failed: Stale file handle. Path:
>>>>> /aorth/.Xauthority (d5bfeb33-786f-45ca-a533-a23cf0c54216)
>>>>> [2014-03-07 09:19:21.820410] W
>>>>> [client-rpc-fops.c:2624:client3_3_lookup_cbk]
>>>>> 0-homes-client-1: remote operation failed: Stale file handle. Path:
>>>>> /aorth/.Xauthority (d5bfeb33-786f-45ca-a533-a23cf0c54216)
>>>>> [2014-03-07 09:27:02.426186] W [client-rpc-fops.c:471:client3_3_open_cbk]
>>>>> 0-homes-client-0: remote operation failed: No such file or directory.
>>>>> Path:
>>>>> /aorth/.Xauthority (323ef093-1e18-463b-8544-0faf8b631985)
>>>>> [2014-03-07 09:27:02.426517] W [client-rpc-fops.c:471:client3_3_open_cbk]
>>>>> 0-homes-client-1: remote operation failed: No such file or directory.
>>>>> Path:
>>>>> /aorth/.Xauthority (323ef093-1e18-463b-8544-0faf8b631985)
>>>>> [2014-03-07 09:27:02.428410] W [fuse-bridge.c:915:fuse_fd_cbk]
>>>>> 0-glusterfs-fuse: 69684128: OPEN() /aorth/.Xauthority => -1 (No such file
>>>>> or
>>>>> directory)
>>>>>
>>>>> And I've seen the issue happen also when accessing other shared files
>>>>> within
>>>>> small time spans, like ~/.cpan/CPAN/MyConfig.pm. Other times I don't see
>>>>> any
>>>>> entries in the log file at all...
>>>>>
>>>>> The volume in question (homes) is configured like this:
>>>>>
>>>>>
>>>>> # gluster volume info homes
>>>>>
>>>>> Volume Name: homes
>>>>> Type: Replicate
>>>>> Volume ID: 38637fa1-7ef3-4e8d-ace9-8ff81bc8bed9
>>>>> Status: Started
>>>>> Number of Bricks: 1 x 2 = 2
>>>>> Transport-type: tcp
>>>>> Bricks:
>>>>> Brick1: storage0:/mnt/gfs/storage0/sda1/homes
>>>>> Brick2: storage1:/mnt/gfs/storage1/sdb1/homes
>>>>> Options Reconfigured:
>>>>> nfs.disable: on
>>>>> Clients and servers are all running GlusterFS 3.4.2 on CentOS 6.4/5.
>>>>>
>>>>> Thanks,
>>>>> --
>>>>> Alan Orth alan.orth at gmail.com http://alaninkenya.org http://mjanja.co.ke
>>>>> "I
>>>>> have always wished for my computer to be as easy to use as my telephone;
>>>>> my
>>>>> wish has come true because I can no longer figure out how to use my
>>>>> telephone." -Bjarne Stroustrup, inventor of C++
>>>>> GPG public key ID: 0x8cb0d0acb5cd81ec209c6cdfbd1a0e09c2f836c0
>>>>>
>>>>> --
>>>>> Alan Orth alan.orth at gmail.com http://alaninkenya.org http://mjanja.co.ke
>>>>> "I
>>>>> have always wished for my computer to be as easy to use as my telephone;
>>>>> my
>>>>> wish has come true because I can no longer figure out how to use my
>>>>> telephone." -Bjarne Stroustrup, inventor of C++
>>>>> GPG public key ID: 0x8cb0d0acb5cd81ec209c6cdfbd1a0e09c2f836c0
>>>>>
>>>>> _______________________________________________
>>>>> Gluster-users mailing list
>>>>> Gluster-users at gluster.org
>>>>> http://supercolony.gluster.org/mailman/listinfo/gluster-users
>> --
>> Alan Orth
>> alan.orth at gmail.com
>> http://alaninkenya.org
>> http://mjanja.co.ke
>> "I have always wished for my computer to be as easy to use as my telephone;
>> my wish has come true because I can no longer figure out how to use my
>> telephone." -Bjarne Stroustrup, inventor of C++
>> GPG public key ID: 0x8cb0d0acb5cd81ec209c6cdfbd1a0e09c2f836c0
>>
>>
>>

-- 
Alan Orth
alan.orth at gmail.com
http://alaninkenya.org
http://mjanja.co.ke
"I have always wished for my computer to be as easy to use as my telephone; my wish has come true because I can no longer figure out how to use my telephone." -Bjarne Stroustrup, inventor of C++
GPG public key ID: 0x8cb0d0acb5cd81ec209c6cdfbd1a0e09c2f836c0

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 884 bytes
Desc: OpenPGP digital signature
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140313/218146e4/attachment.sig>