[Gluster-users] How to replace a dead brick? (3.6.5)

[Lindsay Mathieson]

First up - one of the things that concerns me re gluster is the incoherent
state of documentation. The only docs linked on the main webpage are for
3.2 and there is almost nothing on how to handle failure modes such as dead
disks/bricks etc, which is one of glusters primary functions.

My problem - I have a replica 2 volume, 2 nodes, 2 bricks (zfs datasets).

As a test, I destroyed one brick (zfs destroy the dataset).


Can't start the datastore1:

  volume start: datastore1: failed: Failed to find brick directory
/glusterdata/datastore1 for volume datastore1. Reason : No such file or
directory

A bit disturbing, I was hoping it would work off the remaining brick.

Can't replace the brick:

  gluster volume replace-brick datastore1
vnb.proxmox.softlog:/glusterdata/datastore1
vnb.proxmox.softlog:/glusterdata/datastore1-2 commit force

because the store is not running.

After a lot of googling I found list messages referencing the remove brick
command:
gluster volume remove-brick datastore1 replica 2
vnb.proxmox.softlog:/glusterdata/datastore1c commit force

Fails with the unhelpful error:

wrong brick type: commit, use <HOSTNAME>:<export-dir-abs-path>
Usage: volume remove-brick <VOLNAME> [replica <COUNT>] <BRICK> ...
<start|stop|status|commit|force>

In the end I destroyed and recreated the volume so I could resume testing,
but I have no idea how I would handle a real failed brick in the future

-- 
Lindsay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20151007/c7e018b1/attachment.html>