[Gluster-users] Unable to rebalance...status or stop after upgrade to 3.3

[Dan Bretherton]

Hello All-
I have noticed another problem after upgrading to version 3.3.  I am 
unable to do "gluster volume rebalance <VOLUME> fix-layout status" or 
"...fix-layout ... stop" after starting a rebalance operation with 
"gluster volume rebalance <VOLUME> fix-layout start".   The fix-layout 
operation seemed to be progressing normally on all the servers according 
to the log files, but all attempts to do "status" or "stop" result in 
the CLI usage message being returned.  The only reference to the 
rebalance commands in the log files were these, which all the servers 
seem to have one or more of.

[root at romulus glusterfs]# grep rebalance *.log
etc-glusterfs-glusterd.vol.log:[2012-08-08 12:49:04.870709] W 
[socket.c:1512:__socket_proto_state_machine] 0-management: reading from 
socket failed. Error (Transport endpoint is not connected), peer 
(/var/lib/glusterd/vols/tracks/rebalance/cb21050d-05c2-42b3-8660-230954bab324.sock)
tracks-rebalance.log:[2012-08-06 10:41:18.550241] I 
[graph.c:241:gf_add_cmdline_options] 0-tracks-dht: adding option 
'rebalance-cmd' for volume 'tracks-dht' with value '4'

The volume name is "tracks" by the way.  I wanted to stop the rebalance 
operation because it seemed to be causing a very high load on some of 
the servers had been running for several days.  I ended up having to 
manually kill the rebalance processes on all the servers followed by 
restarting glusterd.

After that I found that one of the servers had "rebalance_status=4" in 
file /var/lib/glusterd/vols/tracks/node_state.info, whereas all the 
others had "rebalance_status=0".  I manually changed the '4' to '0' and 
restarted glusterd.  I don't know if this was a consequence of the way I 
had killed the rebalance operation or the cause of the strange 
behaviour.  I don't really want to start another rebalance going to test 
because the last one was so disruptive.

Has anyone else experienced this problem since upgrading to 3.3?

Regards,
Dan.