[Gluster-users] 'ls' some file will hang, and log will keep growing with the same error

[Manhong Dai]

I made some progress with these tests.

1, a fuse mount with gluster version 5.1 doesn't have this issue. All 
files can listed with 'ls', 'ls -l', 'find . -ls ', etc.

2, The directories after running 'find . -ls' with gluster fuse client 
version 5.1 don't have this issue on gluster fuse client version 6.5 
anymore. Actually I am using this trick now to 'fix' the whole filesystem.

3, 'ls -f', 'find . ' always work on both  version 5.1  and 6.5. but 
'ls', 'ls -l', 'find . -ls' is the trouble-maker

It seems to me that version 6.5 fuse mount has a bug.


Best,

Manhong

On 9/10/19 8:59 AM, Manhong Dai wrote:
> Hi,
>
>     Under some folder, running 'ls' will hang. And the client log will 
> keep growing until there is no space left or the glusterfs process is 
> killed.
>
> [2019-09-10 12:46:30.032136] I [dict.c:559:dict_get] 
> (-->/usr/lib/glusterfs/6.5/xlator/protocol/client.so(+0x5c96e) 
> [0x7f7c6b62a96e] 
> -->/usr/lib/glusterfs/6.5/xlator/cluster/distribute.so(+0x41f01) 
> [0x7f7c6b54ef01] -->/usr/lib/libglusterfs.so.0(dict_get+0x8c) 
> [0x7f7c71194b2c] ) 0-dict: !this || key=trusted.glusterfs.dht.mds 
> [Invalid argument]
> [2019-09-10 12:46:30.033706] I [dict.c:559:dict_get] 
> (-->/usr/lib/glusterfs/6.5/xlator/protocol/client.so(+0x5c96e) 
> [0x7f7c6b62a96e] 
> -->/usr/lib/glusterfs/6.5/xlator/cluster/distribute.so(+0x41f01) 
> [0x7f7c6b54ef01] -->/usr/lib/libglusterfs.so.0(dict_get+0x8c) 
> [0x7f7c71194b2c] ) 0-dict: !this || key=trusted.glusterfs.dht.mds 
> [Invalid argument]
> [2019-09-10 12:46:30.035249] I [dict.c:559:dict_get] 
> (-->/usr/lib/glusterfs/6.5/xlator/protocol/client.so(+0x5c96e) 
> [0x7f7c6b62a96e] 
> -->/usr/lib/glusterfs/6.5/xlator/cluster/distribute.so(+0x41f01) 
> [0x7f7c6b54ef01] -->/usr/lib/libglusterfs.so.0(dict_get+0x8c) 
> [0x7f7c71194b2c] ) 0-dict: !this || key=trusted.glusterfs.dht.mds 
> [Invalid argument]
> [2019-09-10 12:46:30.036739] I [dict.c:559:dict_get] 
> (-->/usr/lib/glusterfs/6.5/xlator/protocol/client.so(+0x5c96e) 
> [0x7f7c6b62a96e] 
> -->/usr/lib/glusterfs/6.5/xlator/cluster/distribute.so(+0x41f01) 
> [0x7f7c6b54ef01] -->/usr/lib/libglusterfs.so.0(dict_get+0x8c) 
> [0x7f7c71194b2c] ) 0-dict: !this || key=trusted.glusterfs.dht.mds 
> [Invalid argument]
>
>     The gluster version is 6.5. Running on an Arch Linux, which was 
> updated to the latest version yesterday.
>
> Best,
>
> Manhong
>