[Gluster-users] Structure needs cleaning on some files

[Johan Huysmans]

Hi All,

When reading some files we get this error:
md5sum: /path/to/file.xml: Structure needs cleaning

in /var/log/glusterfs/mnt-sharedfs.log we see these errors:
[2013-12-10 08:07:32.256910] W 
[client-rpc-fops.c:526:client3_3_stat_cbk] 1-testvolume-client-0: remote 
operation failed: No such file or directory
[2013-12-10 08:07:32.257436] W 
[client-rpc-fops.c:526:client3_3_stat_cbk] 1-testvolume-client-1: remote 
operation failed: No such file or directory
[2013-12-10 08:07:32.259356] W [fuse-bridge.c:705:fuse_attr_cbk] 
0-glusterfs-fuse: 8230: STAT() /path/to/file.xml => -1 (Structure needs 
cleaning)

We are using gluster 3.4.1-3 on CentOS6.
Our servers are 64-bit, our clients 32-bit (we are already using 
--enable-ino32 on the mountpoint)

This is my gluster configuration:
Volume Name: testvolume
Type: Replicate
Volume ID: ca9c2f87-5d5b-4439-ac32-b7c138916df7
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: SRV-1:/gluster/brick1
Brick2: SRV-2:/gluster/brick2
Options Reconfigured:
performance.force-readdirp: on
performance.stat-prefetch: off
network.ping-timeout: 5

And this is how the applications work:
We have 2 client nodes who both have a fuse.glusterfs mountpoint.
On 1 client node we have a application which writes files.
On the other client node we have a application which reads these files.
On the node where the files are written we don't see any problem, and 
can read that file without problems.
On the other node we have problems (error messages above) reading that file.
The problem occurs when we perform a md5sum on the exact file, when 
perform a md5sum on all files in that directory there is no problem.


How can we solve this problem as this is annoying.
The problem occurs after some time (can be days), an umount and mount of 
the mountpoint solves it for some days.
Once it occurs (and we don't remount) it occurs every time.


I hope someone can help me with this problems.

Thanks,
Johan Huysmans