[Gluster-devel] files exist but readdir do not see them

[Emmanuel Dreyfus]

Emmanuel Dreyfus <manu at netbsd.org> wrote:

> > Can you check if the 'acronyms' directory has the same gfid on all 4 servers
> > (from the backend)?
> Yes, it is the same on the 2 bricks (this is a Distributed-Replicate
> with 2x2 bricks)

I overlooked the client log. Here is it:
[2011-07-09 02:39:14.798565] I [afr-common.c:902:afr_fresh_lookup_cbk]
0-gfs-replicate-1: added root inode
[2011-07-09 02:39:14.798862] I [afr-common.c:801:afr_lookup_done]
0-gfs-replicate-1: background  meta-data entry self-heal triggered.
path: /
[2011-07-09 02:39:14.800398] I
[afr-self-heal-common.c:537:afr_sh_mark_sources] 0-gfs-replicate-1:
split-brain possible, no source detected
[2011-07-09 02:39:14.800552] E
[afr-self-heal-metadata.c:518:afr_sh_metadata_fix] 0-gfs-replicate-1:
Unable to self-heal permissions/ownership of '/' (possible split-brain).
Please fix the file on all backend volumes
[2011-07-09 02:39:14.808920] I
[afr-self-heal-metadata.c:81:afr_sh_metadata_done] 0-gfs-replicate-1:
split-brain detected, aborting selfheal of /
[2011-07-09 02:39:14.809079] E
[afr-self-heal-common.c:1554:afr_self_heal_completion_cbk]
0-gfs-replicate-1: background  meta-data entry self-heal failed on /

The permissions/ownerships seems okay, though:
$ ls -ld /export/wd3a/
drwxr-xr-x  3 root  wheel  512 Jul  8 09:53 /export/wd3a/
$ ls -ld /export/wd3a/
drwxr-xr-x  3 root  wheel  512 Jul  8 09:53 /export/wd3a/

But here is a difference. According to the sources, these are used to
detect split brain:
silo# getextattr -x system trusted.afr.gfs-client-3 /export/wd3a/
/export/wd3a/   00 00 00 00 00 00 00 02 00 00 00 00
silo# getextattr -x system trusted.afr.gfs-client-2 /export/wd3a/ 
/export/wd3a/   00 00 00 00 00 00 00 00 00 00 00 00
hangar# getextattr -x system trusted.afr.gfs-client-3 /export/wd3a/
/export/wd3a/   00 00 00 00 00 00 00 00 00 00 00 00
hangar# getextattr -x system trusted.afr.gfs-client-2 /export/wd3a/ 
/export/wd3a/   00 00 00 00 00 00 00 02 00 00 00 01

I tried removing these attributes and the errors are gone, but the
problem remain: none of the file from these two bricks are seen by
readdir.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu at netbsd.org