[Gluster-users] Clarification on common tasks

[Anuradha Talur]

----- Original Message -----
> From: "Gandalf Corvotempesta" <gandalf.corvotempesta at gmail.com>
> To: "gluster-users" <Gluster-users at gluster.org>
> Sent: Thursday, August 11, 2016 2:43:34 PM
> Subject: [Gluster-users] Clarification on common tasks
> 
> I would like to make some clarification on common tasks needed by
> gluster administrators.
> 
> A) Let's assume a disk/brick is failed (or is going to fail) and I
> would like to replace.
> Which is the proper way to do so with no data loss or downtime ?
> 
> Looking on mailing list, seems to be the following:
> 
> 1) kill the brick process (how can I ensure which is the brick process
> to kill)? I have the following on a test cluster (with just one
> brick):
> # ps ax -o command | grep gluster
> /usr/sbin/glusterfsd -s 1.2.3.112 --volfile-id
> gv0.1.2.3.112.export-sdb1-brick -p
> /var/lib/glusterd/vols/gv0/run/1.2.3.112-export-sdb1-brick.pid -S
> /var/run/gluster/27555a68c738d9841879991c725e92e0.socket --brick-name
> /export/sdb1/brick -l /var/log/glusterfs/bricks/export-sdb1-brick.log
> --xlator-option
> *-posix.glusterd-uuid=c97606ac-f6b7-4fdc-a401-6c2d04dd73a8
> --brick-port 49152 --xlator-option gv0-server.listen-port=49152
> /usr/sbin/glusterd -p /var/run/glusterd.pid
> /usr/sbin/glusterfs -s localhost --volfile-id gluster/glustershd -p
> /var/lib/glusterd/glustershd/run/glustershd.pid -l
> /var/log/glusterfs/glustershd.log -S
> /var/run/gluster/5f3713389b19487b6c7d6efca6102987.socket
> --xlator-option
> *replicate*.node-uuid=c97606ac-f6b7-4fdc-a401-6c2d04dd73a8
> 
> which is the "brick process" ?
> 
As clarified by Lindsay, you can find the correct brick to kill
by mapping output of gluster v status with the brick that has failed.
> 2) unmount the brick, in example:
> unmount /dev/sdc
> 
> 3) remove the failed disk
> 
> 4) insert the new disk
> 5) create an XFS filesystem on the new disk
> 6) mount the new disk where the previous one was
> 7) add the new brick to the gluster. How ?
> 8) run "gluster v start force".

If this is a replicate volume then only these steps are not enough.

If you are okay with the mount of new and previous brick to be
different-

After you mount the new-brick, you will have to run
gluster v replace-brick <volname> old_brick new_brick commit force.

By doing this you would be adding new brick to the gluster cluster
and also letting the replicate translator know that
the brick has been replaced and that it needs to be healed.

Once this is done, self-heal-daemon will start the healing process
automatically.

If this step is done, you wouldn't have to run step 8 - gluster v start force.
As replace-brick command takes care of bringing the new brick up.

In case you want to mount the new brick to the same path as the previous one,
then after step 6, I'd suggest you:
a) Create a dummy-non-existent-dir under '/' of the volume's mount point.
b) create a dummy-non-existent-xattr on '/' of the volume's mount point.
The above steps are basically again letting the replicate translator know
that some healing has to be done on the brick that is down. replace-brick
command would do this for you but as it doesn't support same path for old
and new brick, this is a work-around. (Support for replacing bricks with
same path will be provided in upcoming releases. It is being worked on.)

Once this is done, run the replace-brick command mentioned above.
This should add some volume uuids to the brick, start the brick and then
trigger heal to new brick.
> 
> Why should I need the step 8? If the volume is already started and
> working (remember that I would like to change disk with no downtime,
> thus i can't stop the volume), why should I "start" it again ?
> 
> 
> 
> 
> B) let's assume I would like to add a bounch of new bricks on existing
> servers. Which is the proper procedure to do so?

Do you mean increase the capacity of the volume by adding new bricks?
You can use gluster v add-brick new-brick(s)

The options provided to add-brick are going to vary based on how you plan to
add these bricks (whether you want to increase replica-count or add a new
replica set etc).
> 
> 
> Ceph has a good documentation page where some common tasks are explained:
> http://docs.ceph.com/docs/master/rados/operations/add-or-rm-osds/
> i've not found anything similiar in gluster.

I found this for glusterFS :
https://gluster.readthedocs.io/en/latest/Administrator%20Guide/Managing%20Volumes/#replace-brick

Hope this helps.
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
> 

-- 
Thanks,
Anuradha.