[Gluster-users] expanding volumes produce weird results

Wed Nov 3 18:05:39 UTC 2010

Samuele - 
You don't need to create a client vol file with 3.1. Please delete it from the clients and follow these instructions to mount - http://www.gluster.com/community/documentation/index.php/Gluster_3.1:_Manually_Mounting_Volumes 

Thanks, 
Craig 

--> 
Craig Carl 

Gluster, Inc. 
Cell - (408) 829-9953 (California, USA) 
Gtalk - craig.carl at gmail.com 

From: "Samuele Giovanni Tonon" <samu at linuxasylum.net> 
To: "Gluster General Discussion List" <gluster-users at gluster.org> 
Sent: Wednesday, November 3, 2010 10:20:14 AM 
Subject: [Gluster-users] expanding volumes produce weird results 

hi, i've found a strange problem while expanding volumes: 

steps to do: 

1) create a replica 2 volume 
gluster volume create ddvol replica 2 transport tcp \ 
192.168.1.200:/public/dd0 192.168.1.201:/public/dd1 

gluster volume start ddvol 

2) create a client vol files using: 
glusterfs-volgen -n ddvol --num-replica=2 -t tcp 
192.168.1.200:/public/dd0 192.168.1.201:/public/dd1 

3) mount it using 
mount -t glusterfs /root/ddvol-tcp.vol /mnt/ 

4) put some files in it and umount it: 
ls -la /mnt/ 
total 114008 
drwxr-xr-x 5 root root 8192 Nov 3 18:03 . 
drwxr-xr-x 22 root root 4096 Nov 3 17:32 .. 
-rw-r--r-- 1 root root 2139951 Nov 3 18:02 03 - All Together Now.mp3 
-rw-r--r-- 1 root root 14133252 Nov 3 18:02 Cyborg 009_opening.mpg 
-rw-r--r-- 1 root root 31162400 Nov 3 18:03 Scrat2.mp4 
drwxr-xr-x 2 root root 8192 Nov 3 18:03 sc2 

umount /mnt 

4) add 2 more bricks and rebalance 
gluster volume add-brick ddvol 192.168.1.202:/public/dd2 \ 
192.168.1.203:/public/dd3 

gluster volume rebalance ddvol start 

5) after rebalance has finished create a new client vol file and mount 
again: 

gluster volume rebalance ddvol status 
rebalance completed: rebalanced 94 files of size 50547334 (total files 
scanned 230) 

glusterfs-volgen -n ddvol --num-replica=2 -t tcp 
192.168.1.200:/public/dd0 192.168.1.201:/public/dd1 
192.168.1.202:/public/dd2 192.168.1.203:/public/dd3 

6) mount and ls: 
mount -t glusterfs /root/ddvol-tcp.vol /mnt/ 
ls -la /mnt/ 
total 228100 
drwxr-xr-x 5 root root 16384 Nov 3 18:05 . 
drwxr-xr-x 22 root root 4096 Nov 3 17:32 .. 
-rw-r--r-- 1 root root 2139951 Nov 3 18:02 03 - All Together Now.mp3 
-rw-r--r-- 1 root root 2139951 Nov 3 18:02 03 - All Together Now.mp3 
-rw-r--r-- 1 root root 14133252 Nov 3 18:02 Cyborg 009_opening.mpg 
-rw-r--r-- 1 root root 14133252 Nov 3 18:02 Cyborg 009_opening.mpg 
drwxr-xr-x 2 root root 16384 Nov 3 18:05 sc2 
-rw-r--r-- 1 root root 20977664 Nov 3 18:03 scryed.avi 
-rw-r--r-- 1 root root 20977664 Nov 3 18:03 scryed.avi 

as you can see only files are doubled, however : 
files inside sc2 are double too. 

if you start from the beginning with 4 nodes it won't happen. 
if you mount directly from one server you see normal files, this happen 
only after an "add-brick" action. 

you can delete one of those files but at the next mount you'll see them 
again. 

I'm mounting through client vol because i would like to implement a ha 
system (and pointing directly to a server would be a single point of 
failure) while being able to add more space "on the fly" if needed; i'm 
aware i need to add brick in "couple" and it's perfectly fine since 
until there aren't two servers down of the same couple i should be able 
to see my replica, i didn't expect this kind of behaviour . 

Is it well known ? i wasn't able to find any link to this issue. 

btw all was tested on ubuntu 10.10 64bit with package 3.1.0 
md5sum glusterfs_3.1.0-1_amd64.deb 
9e91f8a335f54c59bd771a0aea8e6ae1 /root/glusterfs_3.1.0-1_amd64.deb 

uname -a 
Linux ubuntu0 2.6.35-22-server #35-Ubuntu SMP Sat Oct 16 22:02:33 UTC 
2010 x86_64 GNU/Linux 

Best Regards 
Samuele 

-- 
While various networks have become deeply rooted, and thoughts have been 
sent out as light and electrons in a singular direction, this era has 
yet to digitize/computerize to the degree necessary for individuals to 
become a singular complex entity. 
KOUKAKU KIDOUTAI Stand Alone Complex 

_______________________________________________ 
Gluster-users mailing list 
Gluster-users at gluster.org 
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users 