[Gluster-users] expanding volumes produce weird results

[Samuele Giovanni Tonon]

hi, i've found a strange problem while expanding volumes:

steps to do:

1) create a replica 2 volume
gluster volume create ddvol replica 2 transport tcp \
192.168.1.200:/public/dd0 192.168.1.201:/public/dd1 

gluster volume start ddvol

2) create a client vol files using:
glusterfs-volgen -n ddvol --num-replica=2  -t tcp
192.168.1.200:/public/dd0 192.168.1.201:/public/dd1 

3) mount it using 
mount -t glusterfs /root/ddvol-tcp.vol /mnt/

4) put some files in it and umount it:
ls -la /mnt/
total 114008
drwxr-xr-x  5 root root     8192 Nov  3 18:03 .
drwxr-xr-x 22 root root     4096 Nov  3 17:32 ..
-rw-r--r--  1 root root  2139951 Nov  3 18:02 03 - All Together Now.mp3
-rw-r--r--  1 root root 14133252 Nov  3 18:02 Cyborg 009_opening.mpg
-rw-r--r--  1 root root 31162400 Nov  3 18:03 Scrat2.mp4
drwxr-xr-x  2 root root     8192 Nov  3 18:03 sc2

umount /mnt

4) add 2 more bricks and rebalance
gluster volume add-brick ddvol 192.168.1.202:/public/dd2 \
192.168.1.203:/public/dd3

gluster volume rebalance ddvol start

5) after rebalance has finished create a new client vol file and mount
again:

gluster volume rebalance ddvol status
rebalance completed: rebalanced 94 files of size 50547334 (total files
scanned 230)


glusterfs-volgen -n ddvol --num-replica=2  -t tcp
192.168.1.200:/public/dd0 192.168.1.201:/public/dd1
192.168.1.202:/public/dd2 192.168.1.203:/public/dd3

6) mount and ls:
mount -t glusterfs /root/ddvol-tcp.vol /mnt/
 ls -la /mnt/
total 228100
drwxr-xr-x  5 root root    16384 Nov  3 18:05 .
drwxr-xr-x 22 root root     4096 Nov  3 17:32 ..
-rw-r--r--  1 root root  2139951 Nov  3 18:02 03 - All Together Now.mp3
-rw-r--r--  1 root root  2139951 Nov  3 18:02 03 - All Together Now.mp3
-rw-r--r--  1 root root 14133252 Nov  3 18:02 Cyborg 009_opening.mpg
-rw-r--r--  1 root root 14133252 Nov  3 18:02 Cyborg 009_opening.mpg
drwxr-xr-x  2 root root    16384 Nov  3 18:05 sc2
-rw-r--r--  1 root root 20977664 Nov  3 18:03 scryed.avi
-rw-r--r--  1 root root 20977664 Nov  3 18:03 scryed.avi


as you can see only files are doubled, however :
files inside sc2 are double too.

if you start from the beginning with 4 nodes it won't happen.
if you mount directly from one server you see normal files, this happen
only after an "add-brick" action.

you can delete one of those files but at the next mount you'll see them
again.


I'm mounting through client vol because i would like to implement a ha
system (and pointing directly to a server would be a single point of
failure) while being able to add more space "on the fly" if needed; i'm
aware i need to add brick in "couple" and it's perfectly fine since
until there aren't two servers down of the same couple i should be able
to see my replica, i didn't expect this kind of behaviour .

Is it well known ? i wasn't able to find any link to this issue.

btw all was tested on ubuntu 10.10 64bit with package 3.1.0
md5sum glusterfs_3.1.0-1_amd64.deb
9e91f8a335f54c59bd771a0aea8e6ae1  /root/glusterfs_3.1.0-1_amd64.deb

uname -a 
Linux ubuntu0 2.6.35-22-server #35-Ubuntu SMP Sat Oct 16 22:02:33 UTC
2010 x86_64 GNU/Linux

Best Regards
Samuele 

-- 
While various networks have become deeply rooted, and thoughts have been
sent out as light and electrons in a singular direction, this era has
yet to digitize/computerize to the degree necessary for individuals to
become a singular complex entity.
  KOUKAKU KIDOUTAI Stand Alone Complex