[Gluster-users] Removing bricks from a replicated setup completely brakes volume on Gluster 3.3

[Marc Seeger]

Initial setup: A replicated volume with 3 bricks
Goal: Remove one of the bricks from it.
Version: # glusterfs 3.3git built on Jun  7 2013 14:38:02 (branch release-3.3)

Initial setup: A replicated volume with 3 bricks
Goal: Remove one of the bricks from it.
Outcome: A completely broken volume


------------- Volume info -------------

root at fs-14.example:~# gluster volume info

Volume Name: test-fs-cluster-1
Type: Replicate
Volume ID: 752e7ffd-04bb-4234-8d16-d1f49ef510b7
Status: Started
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: fs-14.example.com:/mnt/brick21
Brick2: fs-15.example.com:/mnt/brick20
Brick3: fs-14.example.com:/mnt/brick33


------------- Trying to remove a brick -------------

fields-config-gluster.rb[5035]: Using commandline: gluster volume remove-brick test-fs-cluster-1 replica 2 fs-14.example.com:/mnt/brick33 start
fields-config-gluster.rb[5035]: Command returned exit code 255: gluster volume remove-brick test-fs-cluster-1 replica 2 fs-14.example.com:/mnt/brick33 start
stdout was:

stderr was:
Remove Brick start unsuccessful




------------- Volume turned Distributed-Replicate -------------
[12:23:37] root at fs-14.example:~# gluster volume info
 
Volume Name: test-fs-cluster-1
Type: Distributed-Replicate
Volume ID: 752e7ffd-04bb-4234-8d16-d1f49ef510b7
Status: Started
Number of Bricks: 1 x 2 = 3
Transport-type: tcp
Bricks:
Brick1: fs-14.example.com:/mnt/brick21
Brick2: fs-15.example.com:/mnt/brick20
Brick3: fs-14.example.com:/mnt/brick33


------------- Trying to remove brick again -------------

[12:26:20] root at fs-14.example:~# gluster volume remove-brick test-fs-cluster-1 replica 2 fs-14.example.com:/mnt/brick33 start
number of bricks provided (1) is not valid. need at least 2 (or 2xN)

------------- Trying to stop volume -------------

[12:28:34] root at fs-14.example:~# gluster volume stop test-fs-cluster-1
Stopping volume will make its data inaccessible. Do you want to continue? (y/n) y
Stopping volume test-fs-cluster-1 has been successful


------------- Trying to start volume again -------------
[12:29:03] root at fs-14.example:~# gluster volume start test-fs-cluster-1
Starting volume test-fs-cluster-1 has been unsuccessful

------------- Trying to stop volume again -------------

[12:29:49] root at fs-14.example:~# gluster volume stop test-fs-cluster-1
Stopping volume will make its data inaccessible. Do you want to continue? (y/n) y
Volume test-fs-cluster-1 is not in the started state

------------- Trying to delete volume -------------

[12:29:55] root at fs-14.example:~# gluster volume delete test-fs-cluster-1
Deleting volume will erase all information about the volume. Do you want to continue? (y/n) y
Volume test-fs-cluster-1 has been started.Volume needs to be stopped before deletion.

------------- Checking volume info -------------
# gluster volume info
 
Volume Name: test-fs-cluster-1
Type: Distributed-Replicate
Volume ID: 752e7ffd-04bb-4234-8d16-d1f49ef510b7
Status: Started
Number of Bricks: 1 x 2 = 3
Transport-type: tcp
Bricks:
Brick1: fs-14.example.com:/mnt/brick21
Brick2: fs-15.example.com:/mnt/brick20
Brick3: fs-14.example.com:/mnt/brick33

------------- Trying to stop volume again -------------
[12:30:50] root at fs-14.example:~# gluster volume stop test-fs-cluster-1
Stopping volume will make its data inaccessible. Do you want to continue? (y/n) y
Volume test-fs-cluster-1 is not in the started state



------------- Restarting glusterfs-server -------------

[12:38:05] root at fs-14.example:~# /etc/init.d/glusterfs-server restart
glusterfs-server start/running, process 6426

------------- Volume switched back to "Replicate" -------------
[12:38:33] root at fs-14.example:~# gluster volume info
 
Volume Name: test-fs-cluster-1
Type: Replicate
Volume ID: 752e7ffd-04bb-4234-8d16-d1f49ef510b7
Status: Started
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: fs-14.example.com:/mnt/brick21
Brick2: fs-15.example.com:/mnt/brick20
Brick3: fs-14.example.com:/mnt/brick33


------------- Trying to stop volume again -------------
[12:38:39] root at fs-14.example:~# gluster volume stop test-fs-cluster-1
Stopping volume will make its data inaccessible. Do you want to continue? (y/n) y
Volume test-fs-cluster-1 is not in the started state



Any idea what's up with that?

Cheers,
Marc