[Gluster-users] Need help in understanding volume heal-info behavior

[Chalcogen]

Hi everyone,

I have trouble understanding the following behavior:

Suppose I have a replica 2 volume 'testvol' on two servers, server1 and 
server2, composed of server1:/bricks/testvol/brick and 
server2:/bricks/testvol/brick. Also, suppose it contains a good number 
of files.

Now, assume I remove one of the two bricks, as:

root at server1~# gluster volume remove-brick testvol replica 1 
server1:/bricks/testvol/brick

Now, I unmount and delete the logical volume supporting the brick and 
then recreate it (with a different size), and mount it the same way as 
it was mounted before (at /brick/testvol/). Then, I re-add it as:

root at server1~# gluster volume add-brick testvol replica 2 
server1:/bricks/testvol/brick

I observe that the brick on server1 does not contain any of the data 
that was in the volume.

root at server1~# ls /bricks/testvol/brick
root at server1~#

This is all right by me, since glusterfs needs some time to discover and 
sync files that are absent on the brick of server1. In fact, if I leave 
the setup undisturbed for 15 minutes to half an hour, I find that all 
data appears within the brick of server1, just as you would expect. 
Also, if I wish to speed up the process, I simply do a ls -Ra on the 
directory where the volume is mounted, and all files sync onto server1's 
brick. This is also very much as expected.

However, during the period where data on server1's brick is not 
available, if you query the heal info for the volume, gluster cli 
reports that 'Number of entries' is '0', and that too all of 'info', 
'heal-failed', and 'split-brain'. This is what becomes a bit of a 
trouble for me. Fact is, we are attempting to automate the monitoring of 
our glusterfs volumes, and we depend upon heal info alone to decide 
whether data on server1 and server2 are in sync.

Could somebody, therefore, help me with the following questions?
a) Which files exactly show up in heal info?
b) What exactly should I look to monitor if we are to ascertain that 
data on our servers are in sync?

Thanks a lot for your responses!

Anirban

P.s. I am using glusterfs 3.4.2 over linux kernel version 2.6.34.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140428/d0e8acb4/attachment.html>