[Gluster-devel] volume rebalance still broken

[Emmanuel Dreyfus]

Hi

I am back with my volume rebalance problem on NetBSD. The logs told
me I had error when handling extended attributes. The problem were caused
by differences between the Linux and NetBSD extended attribute API. I 
fixed it by implementing the Linux API à la <sys/xattr.h> in NetBSD, 
hopefully this will make my life easier in the future.

gluster volume rebalance now run without a hitch and logs report no
errors. However, I have a problem with the file that existed on the
volume prior the first rebalance operation:

client# ls -l old_file
-rw-r--r--  1 root  wheel  893 May 10 11:47 old_file
client# cat old_file
cat: old_file: Not a directory

The file exists on both servers (this is distributed-replicate with 2 x 2 
bricks), and I can read it here. Files added after the rebalance can be
accessed without any problem.

I tried stopping glusterfsd, removing all extended attribute on a bick
and restarting it. glusterfs was able to reconstruct attributes:
trusted.gfid              
trusted.glusterfs.test
trusted.afr.gfs1-client-0
trusted.afr.gfs1-client-1
trusted.afr.gfs1-client-2
trusted.afr.gfs1-client-3
trusted.glusterfs.dht

If I remove the desired file from a brick, afr will restre it from the
other brick, but it is still not accessible.

Removing extended atributes from the desired file case it to 
disapear from client view, and I get this in the logs:
  E [posix.c:510:posix_stat] 0-gfs1-posix: lstat on /old_file failed: 
  Attribute not found

Unmounting and remounting the volume lead me back to the original 
situation where reading the file gets ENODIR.

Any hint?

-- 
Emmanuel Dreyfus
manu at netbsd.org