[Gluster-users] Clarification about remove-brick
Mohammed Rafi K C
rkavunga at redhat.com
Mon Aug 1 05:07:58 UTC 2016
On 07/29/2016 11:08 PM, Richard Klein (RSI) wrote:
>
> Thank you very much for the information. I have read the
> documentation link you have listed below and just wanted confirmation
> about the remove-brick process. I did not see any documentation about
> the ability to use the remove-brick stop command so that is good to know.
>
I'm not sure, how much testing has been done in that area. But we do
have the remove stop command, and theoretically it should work. If you
do rebalance after remove-brick stop, then again the cluster will be
load balanced if files are already migrated as part of the remove brick
operation.
>
>
> I have a couple of related follow up questions as well:
>
>
>
> 1) Let’s say a brick fails (hard drive goes bad). I want to
> replace the bad drive with a new one and then get it back operational
> in the cluster. In this case I don’t have spare brick available so I
> can’t do the replace-brick. The only procedure I have found is a blog
> and bug report for 3.4 about this issue with a work around. Here is
> the link to the blog:
> https://joejulian.name/blog/replacing-a-brick-on-glusterfs-340/. Here
> is the link to the bug report:
> https://bugzilla.redhat.com/show_bug.cgi?id=991084. My apologies if
> this has been addressed before but I have searched and can’t find a
> solution where you just replace the bad drive with a good one and
> recover. Any instructions about this process would be appreciated.
>
If I understand you correctly, you want to replace a drive and create
the brick with same name. If that is the case, patch
http://review.gluster.org/#/c/12250/ will solve the. In fact the patch
is under review. CCing the replace brick experts to give you more info
regarding your replace queries.
> 2) Does the rebalance process lock data files in Gluster? We are
> using Gluster as primary storage in Cloudstack 4.7. If we shrink or
> expand the Gluster volume we do a rebalance of the layout and data.
> However, after this process we have several VMs which have disk
> volumes in a read only state like there were disk problems. Once we
> power cycle the VM all is well and no data loss occurred but there
> seems to be a correlation between the rebalance and the errors. I am
> wondering if the rebalance process locks the data somehow and makes it
> unavailable to the VM.
>
Rebalance process will take locks as part of the healing. But I'm not
sure whether it cause VM's to go into read-only state. The locks will be
took on directory for healing and for files during migration, but only
for a specific offset. So both should not cause an entire file to be
read-only for a long time.
Can you describe your problem more ? I will try to see if there is
anything else causing the issue.
Regards
Rafi KC
>
>
> Thanks again for the response.
>
>
>
> Richard Klein
>
> RSI
>
>
>
>
>
> *From:*Mohammed Rafi K C [mailto:rkavunga at redhat.com]
> *Sent:* Friday, July 29, 2016 1:38 AM
> *To:* Lenovo Lastname; Richard Klein (RSI); gluster-users at gluster.org
> *Subject:* Re: [Gluster-users] Clarification about remove-brick
>
>
>
> You can also see the documentation here
> https://gluster.readthedocs.io/en/latest/Administrator%20Guide/Managing%20Volumes/#shrinking-volumes
>
>
>
> Rafi KC
>
> On 07/29/2016 11:39 AM, Mohammed Rafi K C wrote:
>
>
>
> I will summarize the procedure for removing a brick with description.
>
>
>
> 1) start an add brick operation using gluster volume remov-brick
> command. This command will mark the mentioned brick as a
> decommissioned brick. Also, this will kick a process that will
> start migrating data from the decommissioned brick to the other
> bricks.
>
> 2) Once the migration is finished you can safely do a remove-brick
> commit.
>
> 3) Or if you wish to stop the process and reset the decommissioned
> brick, you can do remove-brick stop. This will not migrate the
> data back to the decommissioned brick. It will stay in the other
> bricks and the data will be still accessible, if you want to have
> proper load balancing after this, you can start rebalance process.
>
> 4) If you wish to do an instant remove brick you can use force
> option, which will not migrate data, hence your whole data in the
> removed brick will be lost from mount point.
>
> On 07/29/2016 01:25 AM, Lenovo Lastname wrote:
>
> I'm using 3.7.11, this command works with me,
>
>
>
> !remove-brick
> [root at node2 ~]# gluster volume remove-brick v1 replica 2
> 192.168.3.73:/gfs/b1/v1 force
> Removing brick(s) can result in data loss. Do you want to
> Continue? (y/n) y
> volume remove-brick commit force: success
>
>
>
> Don't know about the commit thingy...
>
>
>
> On Thursday, July 28, 2016 3:47 PM, Richard Klein (RSI)
> <rklein at rsitex.com> <mailto:rklein at rsitex.com> wrote:
>
>
>
> We are using Gluster 3.7.6 in a replica 2
> distributed-replicate configuration. I am wondering when we
> do a remove-brick with just one brick pair will the data be
> moved off the bricks once the status show complete and then
> you do the commit? Also, if you start a remove-brick
> process can you stop it? Is there an abort or stop command or
> do you just don’t do the commit?
>
>
>
> Any help would be appreciated.
>
>
>
> Richard Klein
>
> RSI
>
>
>
>
>
>
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org <mailto:Gluster-users at gluster.org>
> http://www.gluster.org/mailman/listinfo/gluster-users
>
>
>
>
> _______________________________________________
>
> Gluster-users mailing list
>
> Gluster-users at gluster.org <mailto:Gluster-users at gluster.org>
>
> http://www.gluster.org/mailman/listinfo/gluster-users
>
>
>
>
>
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160801/3ab83fea/attachment.html>
More information about the Gluster-users
mailing list