[Gluster-devel] Thoughts on enhancing replace-brick

[Joe Julian]

Meh, I should have read the bug first.

I didn't realize that AFR2 broke that, so with that patch:

1. ssh root at node-1
2. gluster volume stop example-vol node-1:/srv/brick-02/brick.0
3. # replace physical disk
4. remount /srv/brick-02
5. gluster volume replace-brick example-vol node-1:/srv/brick-02/brick.0 node-1:/srv/brick-02/brick.0


On 03/18/2016 08:34 AM, Joe Julian wrote:
> I completely agree that we should not have to kill (never -9 as this 
> will cause the clients to wait for ping-timeout) brick processes 
> directly. There should be a cli command to stop an individual brick.
>
> The "Today" method should be corrected as follows:
>   1. ssh root at node-1
>   2. kill $(gluster volume status example-vol | awk '/srv.brick-02.brick.0/{print $3}')
>   3. # replace physical disk
>   4. remount /srv/brick-02
>   5. gluster volume start example-vol force # no replace-brick necessary
>   6. gluster volume heal example-vol full
> I'd love to see step 2 replaced with
> gluster volume stop example-vol node-1:/srv/brick-02/brick.0
> This command should gracefully stop the brick, closing tcp 
> connections. It should fail if there are pending-heals with that brick 
> as the source unless you add the "force" keyword at the end.
>
> This would turn your hypothetical example into
>
>   1. ssh root at node-1
>   2. gluster volume stop example-vol node-1:/srv/brick-02/brick.0
>   3. # replace physical disk
>   4. remount /srv/brick-02
>   5. gluster volume start example-vol force
>   6. gluster volume heal example-vol full
>
>
>
> On 03/18/2016 08:11 AM, Peter Portante wrote:
>> Hi Folks,
>>
>> There is a review posted, http://review.gluster.org/#/c/12250, to 
>> which I tacked on a review comment for an update to the replace-brick 
>> command. The gist of it is at 
>> https://gist.github.com/portante/248407dbfb29c2515fc3
>>
> gist included for completeness:
>   > I like where this is going.  Perhaps we could consider the steps an
>   > admin has to take and orient the commands to address what they need
>   > to do.
>   >
>   > First, having to "kill -9" a process as an interface to stopping a
>   > brick seems dicey.
>   >
>   > So to that end, perhaps we could two sets of commands, the first to
>   > tell gluster to take the brick out of service, allowing the admin
>   > to then replace it, and a second command to brick the brick back
>   > into service.
>   >
>   > For example:
>   >
>   > gluster volume name: example-vol
>   > 6 node cluster, each member providing 12 bricks, three way replicated, distributed
>   > members are named node-0 ... node-5
>   > disks are named /srv/brick-00/brick.0 .. /srv/brick-11/brick.0
>   >
>   > Lets say the disk for /srv/brick-02/brick.0 on node-1 goes bad.
>   >
>   > Today I believe I have to:
>   >
>   > 1. ssh root at node-1
>   > 2. kill -9 $(gluster volume status | grep /srv/brick-02/brick | awk '{print $3}')
>   > 3. # replace physical disk
>   > 4. remount /srv/brick-02
>   > 5. mkdir /srv/brick-02/brick.1
>   > 6. # on a FUSE mnt, do the directory dance and the xattr dance
>   > 7. # ensure other node brick trusted IDs for healing are correct
>   > 8. gluster volume replace-brick example-vol node-1:/srv/brick-02/brick.0 node-1:/srv/brick-02/brick.1 commit force
>   >
>   > Perhaps we could do the following instead:
>   >
>   > 1. ssh root at node-1
>   > 2. gluster volume replace-brick example-vol node-1:/srv/brick-02/brick.0 offline
>   > 3. # replace physical disk
>   > 4. remount /srv/brick-02
>   > 5. mkdir /srv/brick-02/brick.0
>   > 6. gluster volume replace-brick example-vol node-1:/srv/brock-02/brick.0 online
>   >
>   > Then the new step #6 would take care of the old steps #6 & #7. We
>   > would lose the "kill -9" and replace with a declaration of
>   > intention command that tells gluster to "take this brick off line,
>   > stopping the brick process, and removing references to the mount
>   > point".
>   >
>   > What do you all think?
>   >
>   > Please excuse my ignorance of the ins-and-outs of gluster commands
>   > if I have the above steps off a bit.
>
>> What do folks think of such a proposal?
>>
>> Thanks!
>>
>> -peter
>>
>>
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-devel
>
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20160318/7478f040/attachment-0001.html>