[Gluster-devel] AFR + Unify - SelfHeal Problems

[Jens Binnewies]

Hello World,


i run into strange Problems while using glusterfs:
(maybe this is a normal behavior?)


* Server & Client GlusterFS Version: >=1.3.12 (latest tested: mainline 3.0 - 01/12/2008)
* 2 Servers with AFR & Unify enabled
* 1 Linux Client
* Shared filesystems are ext3 with mopts: rw,noatime,nodiratime,user_xattr,acl,reservation
   ** tried also without "noatime,nodiratime,acl,reservation"
* Linux Kernel 2.6.28-rc6
   ** Extended Attributes compiled into kernel for ext3 and ext4
* Filesystem on top (root): ext4 (Namespace is mounted on it)
* Test run always with same glusterfs version on server & client side

I've read, gluster AFR Selfhealing sets and works with the xattrs
trusted.gluster.* - here starts the Problem: i do not see any attr on
glusterfs-shares or uploaded files, except:

Attribute "glusterfs.test" has a 8 byte value for data/
Attribute "glusterfs.afr.entry-pending" has a 8 byte value for data

for the root dir of the share, and:

Attribute "glusterfs.afr.data-pending" has a 8 byte value for data/test1

for the files.


The following scenario:

i upload file "test1" to the share.
The 2 Servers are replicating the file in-time 
while uploading and no selfhealing is triggered, which is ok.

A look into the share on both servers, shows me everything's fine. 
Namespace file is touched and the data and size in the share, seems to
be fine too on BOTH.

Now, i trigger a "ls" on the client and it -stucks-... i wonder why and
took a look into the serverlogs:

2008-12-01 12:18:22 W [afr-self-heal-common.c:955:afr_self_heal]
storage01-ds-afr: performing self heal on /test1 (metadata=0 data=1
entry=0)


I notice that the server is transfering this file again to the
other server and while this action runs, no dir-listing is done on
clientside and it stucks.


After this "initial" 'ls', everything seems to work.. but why does the
server selfheal and retransfer the whole file again to the another
server?!


Maybe a mistake in my Configfile or normal behavior?



Serverconfig:

http://gluster.pastebin.com/m68ddb7cc


Clientconfig:

http://gluster.pastebin.com/m3c95ec6a




"storage01.gluster.bin" points to the 2 Servers



Greets, Jens

-- 
ELF$((32+1))
  .who  "Jens Binnewies"
  .gpg  "0x4604A74D"
  .jab  "sigXcpu at jabber.ccc.de"
  .web  "http://fearnop.ath.cx"

static const char *loadingVal[4] = { "|", "/", "-", "\\" };
for (loadint = 0; FirstEOF != 1; loadint++)
{
     /* our ONE & ONLY hope-check before a BOF occurs :P */
     if (loadint >= 4)
         loadint = 0;
     p_consmsg("\rloading... %s", loadingVal[loadint]);
     (void) usleep(200000);
}