[Gluster-devel] ls again

Jordi Moles jordi at cdmon.com
Wed Apr 9 09:54:08 UTC 2008 

Hi,

I restarted both nodes and clients, cause it got to the point where the 
filesystem was totally inaccessible, which means that a "ls" command 
didn't work anymore, i only could get that error saying that the system 
was no ready.

Anyway... without doing anything else... just restarting everything, now 
is back on again, but ls keeps being so slow.

no node don't report any log at all and clients write this:

*************

2008-04-09 09:23:49 E [fuse-bridge.c:459:fuse_entry_cbk] glusterfs-fuse: 
8738177: (34) /massiu1/tmp/1207732944.P9218.postfix01gluster01.cdmon.com 
=> -1 (2)
2008-04-09 09:23:49 E [fuse-bridge.c:459:fuse_entry_cbk] glusterfs-fuse: 
8738177: (34) /massiu1/tmp/1207732944.P9218.postfix01gluster01.cdmon.com 
=> -1 (2)
2008-04-09 09:33:46 E [unify.c:182:unify_lookup_cbk] ultim: grup3 returned 2
2008-04-09 09:33:46 E [unify.c:182:unify_lookup_cbk] ultim: nm returned 2
2008-04-09 09:33:46 E [fuse-bridge.c:459:fuse_entry_cbk] glusterfs-fuse: 
9476489: (34) /massiu1/tmp/1207733533.P7403.postfix02gluster01.cdmon.com 
=> -1 (2)
2008-04-09 09:33:46 E [fuse-bridge.c:459:fuse_entry_cbk] glusterfs-fuse: 
9476489: (34) /massiu1/tmp/1207733533.P7403.postfix02gluster01.cdmon.com 
=> -1 (2)
2008-04-09 09:33:46 E [unify.c:182:unify_lookup_cbk] ultim: grup1 returned 2
2008-04-09 09:33:46 E [unify.c:182:unify_lookup_cbk] ultim: nm returned 2
2008-04-09 09:33:46 E [fuse-bridge.c:459:fuse_entry_cbk] glusterfs-fuse: 
9476498: (34) /massiu1/tmp/1207733537.P7325.postfix02gluster01.cdmon.com 
=> -1 (2)
2008-04-09 09:33:46 E [fuse-bridge.c:459:fuse_entry_cbk] glusterfs-fuse: 
9476498: (34) /massiu1/tmp/1207733537.P7325.postfix02gluster01.cdmon.com 
=> -1 (2)

*************

Do you have any idea? Any suggestion about what can i possibly check or 
change to make it work? Should i go back to an earlier version?

Thanks.

En/na Jordi Moles ha escrit:
> Hi,
>
> yes, i realised that. The thing is... as i said... nothing has 
> changed, a part from gluster version. Disk were and still are ok. Just 
> in case... i performed a whole fsck of each disk, but no error was 
> reported.
>
> is there any other explanation for that issue?
>
> En/na Amar S. Tumballi ha escrit:
>> Hi Jordi,
>>  Can you check on the backend if your disks are ok? because from 
>> posix level
>> itself, you are getting EIO. (Input/Output Error)
>>
>> Regards,
>> Amar
>>
>> On Mon, Apr 7, 2008 at 9:15 AM, Jordi Moles <jordi at cdmon.com> wrote:
>>
>>  
>>> Hi everyone,
>>>
>>> i was about to write a new message to this list to explain my new
>>> experiencies with "the ls problem".
>>>
>>> This morning i updated both fuse and glusterfs to the latest tla 
>>> versions,
>>> i used to have mainline2.5-patch 690, now i've got patch 736.
>>>
>>> The thing is that i used to have "ls slow" commands when there was 
>>> "some"
>>> activity in the filesystem.
>>>
>>> But with today's update, the system is even slowler, with an empty
>>> filesystem, with no activity at all.... ls is really really slow.
>>>
>>> is that normal?
>>>
>>> these are extracts from my log files:
>>>
>>> postfix-client
>>> ***************
>>> 2008-04-07 16:09:52 E [unify.c:177:unify_lookup_cbk] ultim: grup1 
>>> returned
>>> 5
>>> 2008-04-07 16:09:52 E [fuse-bridge.c:459:fuse_entry_cbk] 
>>> glusterfs-fuse:
>>> 483446: (34) /dummymassiu2/new/
>>> 1207584588.V12I1e6810M24355.postfix01gluster01.cdmon.com,S=1312 => 
>>> -1 (5)
>>> 2008-04-07 16:09:52 E [unify.c:177:unify_lookup_cbk] ultim: grup1 
>>> returned
>>> 5
>>> 2008-04-07 16:09:52 E [fuse-bridge.c:459:fuse_entry_cbk] 
>>> glusterfs-fuse:
>>> 483450: (34) /dummymassiu2/new/
>>> 1207584588.V12I1e6810M24355.postfix01gluster01.cdmon.com,S=1312 => 
>>> -1 (5)
>>> 2008-04-07 16:09:53 E [unify.c:177:unify_lookup_cbk] ultim: grup1 
>>> returned
>>> 5
>>> 2008-04-07 16:09:53 E [fuse-bridge.c:459:fuse_entry_cbk] 
>>> glusterfs-fuse:
>>> 483784: (34) /dummymassiu6/new/
>>> 1207584588.V12I124a4aM313591.postfix01gluster01.cdmon.com,S=1313 => 
>>> -1 (5)
>>> 2008-04-07 16:09:53 E [unify.c:177:unify_lookup_cbk] ultim: grup1 
>>> returned
>>> 5
>>> 2008-04-07 16:09:53 E [fuse-bridge.c:459:fuse_entry_cbk] 
>>> glusterfs-fuse:
>>> 483788: (34) /dummymassiu6/new/
>>> 1207584588.V12I124a4aM313591.postfix01gluster01.cdmon.com,S=1313 => 
>>> -1 (5)
>>> ***************
>>>
>>> nodes
>>> ***************
>>> 2008-04-07 16:10:29 W [posix.c:92:posix_lookup] esp: lstat on
>>> /dummymassiu6/new/maildirsize: Input/output error
>>> 2008-04-07 16:10:29 W [posix.c:92:posix_lookup] esp: lstat on
>>> /dummymassiu6/new/
>>> 1207584613.V12I124a4aM742789.postfix01gluster01.cdmon.com,S=1312:
>>> Input/output error
>>> 2008-04-07 16:10:29 W [posix.c:92:posix_lookup] esp: lstat on
>>> /dummymassiu6/new/maildirsize: Input/output error
>>> 2008-04-07 16:10:30 W [posix.c:92:posix_lookup] esp: lstat on
>>> /dummymassiu2/new/
>>> 1207584622.V12I1e6810M709269.postfix02gluster01.cdmon.com,S=1310:
>>> Input/output error
>>> 2008-04-07 16:10:30 W [posix.c:92:posix_lookup] esp: lstat on
>>> /dummymassiu2/new/
>>> 1207584622.V12I1e6810M709269.postfix02gluster01.cdmon.com,S=1310:
>>> Input/output error
>>> 2008-04-07 16:10:30 W [posix.c:92:posix_lookup] esp: lstat on
>>> /dummymassiu6/new/
>>> 1207584610.V12I124a51M867739.postfix02gluster01.cdmon.com,S=1310:
>>> Input/output error
>>> 2008-04-07 16:10:30 W [posix.c:92:posix_lookup] esp: lstat on
>>> /dummymassiu6/new/maildirsize: Input/output error
>>> 2008-04-07 16:10:30 W [posix.c:92:posix_lookup] esp: lstat on
>>> /dummymassiu6/new/maildirsize: Input/output error
>>> 2008-04-07 16:10:30 W [posix.c:92:posix_lookup] esp: lstat on
>>> /dummymassiu6/new/
>>> 1207584616.V12I124a52M181177.postfix02gluster01.cdmon.com,S=1311:
>>> Input/output error
>>> 2008-04-07 16:10:30 W [posix.c:92:posix_lookup] esp: lstat on
>>> /dummymassiu6/new/
>>> 1207584610.V12I124a51M867739.postfix02gluster01.cdmon.com,S=1310:
>>> Input/output error
>>> ***************
>>>
>>> Nothing a part from the glusterfs version has changed... i mean... conf
>>> files are the same.... same machines.... and so on.....
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> En/na Reinis Rozitis ha escrit:
>>>
>>>  ls commands on modern GNU/Linux systems run xattr calls on each 
>>> entry for
>>>    
>>>>>>           
>>>>> ACL purposes. This can turn out to be pretty expensive in terms of
>>>>>         
>>>> performance.
>>>>
>>>> Thanks,
>>>> thats what I thought of. Maybe worth to add to the FAQ or some other
>>>> docs so
>>>> it doesnt turn out as WTF - factor.
>>>>
>>>> rr
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Gluster-devel mailing list
>>>> Gluster-devel at nongnu.org
>>>> http://lists.nongnu.org/mailman/listinfo/gluster-devel
>>>>
>>>>
>>>>       
>>>
>>> _______________________________________________
>>> Gluster-devel mailing list
>>> Gluster-devel at nongnu.org
>>> http://lists.nongnu.org/mailman/listinfo/gluster-devel
>>>
>>>     
>>
>>
>>
>>   
>
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> http://lists.nongnu.org/mailman/listinfo/gluster-devel

More information about the Gluster-devel mailing list