[Gluster-users] How does volume rebalance work?

[Olivier Nicole]

Hi,

I have hard time understanding how volume rebalance works.

I had 2 bricks: gluster3:/data & gluster4:/data

It contains:

on at gluster3:/data$ ls -lrat
total 5117020
-rwx------  1 root root 108693504 2011-09-28 19:38 VMware-tools-linux-8.6.0-425873.iso
-rwx------  1 root root  93786112 2011-09-28 19:38 VMware-tools-linux-8.3.7-341836.iso
-rwx------  1 root root 266371072 2011-09-28 19:39 VMware-VMvisor-InstallerCD-3.5.0_Update_5-207095.i386.iso
-rwx------  1 root root 304859136 2011-09-28 19:39 VMware-VMvisor-Installer-5.0.0-469512.x86_64.iso
-rwx------  1 root root 412487680 2011-09-28 19:39 VMware-VMvisor-Installer-4.1.0.update1-348481.x86_64.iso
-rwx------  1 root root 611405824 2011-09-28 19:40 ubuntu-8.04.3-server-amd64.iso
-rwx------  1 root root 686305280 2011-09-28 19:41 ubuntu-9.10-server-amd64.iso
-rwx------  1 root root 631246848 2011-09-28 19:41 ubuntu-8.04.3-server-i386.iso
-rwx------  1 root root 671686656 2011-09-28 19:42 ubuntu-9.10-server-i386.iso
-rwx------  1 root root 706330624 2011-09-28 19:42 ubuntu-11.04-server-amd64.iso
-rwx------  1 root root  13121536 2011-09-28 19:42 ubuntu-10.04-minimal-i386.iso
-rwx------  1 root root 733419520 2011-09-28 19:42 ubuntu-10.04-desktop-i386.iso
drwxr-xr-x 22 root root      4096 2011-09-28 19:44 ..
drwxr-xr-x  2 root root      4096 2011-09-28 20:04 .


To my volume I add gluster1:/data & gluster2:/data


gluster> volume add-brick test-vol gluster1:/data gluster2:/data
Add Brick successful
gluster> volume info

Volume Name: test-vol
Type: Distributed-Replicate
Status: Started
Number of Bricks: 2 x 2 = 4
Transport-type: tcp
Bricks:
Brick1: gluster3:/data
Brick2: gluster4:/data
Brick3: gluster1:/data
Brick4: gluster2:/data
gluster> volume rebalance test-vol start
starting rebalance on volume test-vol has been successful
gluster> volume rebalance test-vol status
rebalance step 2: data migration in progress
gluster> volume rebalance test-vol status
rebalance completed
gluster> 

After the rebalance, if I look at gluster1:

on at gluster1:/data$ ls -la
total 1624168
drwxr-xr-x  2 root root      4096 2011-09-28 20:07 .
drwxr-xr-x 22 root root      4096 2011-09-28 20:04 ..
---------T  1 root root         0 2011-09-28 20:06 ubuntu-8.04.3-server-i386.iso
-rwx------  1 root root 631246848 2011-09-28 20:07 .ubuntu-8.04.3-server-i386.iso.gfs631246848
---------T  1 root root         0 2011-09-28 20:06 ubuntu-9.10-server-i386.iso
-rwx------  1 root root 671686656 2011-09-28 20:07 .ubuntu-9.10-server-i386.iso.gfs671686656
---------T  1 root root         0 2011-09-28 20:06 VMware-tools-linux-8.3.7-341836.iso
-rwx------  1 root root  93786112 2011-09-28 20:07 .VMware-tools-linux-8.3.7-341836.iso.gfs93786112
---------T  1 root root         0 2011-09-28 20:06 VMware-VMvisor-InstallerCD-3.5.0_Update_5-207095.i386.iso
-rwx------  1 root root 266371072 2011-09-28 20:06 .VMware-VMvisor-InstallerCD-3.5.0_Update_5-207095.i386.iso.gfs266371072

And bricks gluster3 and gluster4 still contain the exact same data.

Why on the new bricks the files come with a name prefixed with a dot,
why is there an empty file created with sticky bit? I would have
expected the temporary dof ile to be renamed into the final name
(sticky bit removed) and the original file on the old brick to be
removed.

Am I missing something?

Best regards,

Olivier