<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p.MsoPlainText, li.MsoPlainText, div.MsoPlainText
{mso-style-priority:99;
mso-style-link:"Plain Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.PlainTextChar
{mso-style-name:"Plain Text Char";
mso-style-priority:99;
mso-style-link:"Plain Text";
font-family:"Calibri",sans-serif;}
span.EmailStyle20
{mso-style-type:personal;
font-family:"Calibri",sans-serif;}
span.EmailStyle21
{mso-style-type:personal;
font-family:"Calibri",sans-serif;}
span.EmailStyle23
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal">Can I please get some help in understanding the issue mentioned ?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Anand Malagi <br>
<b>Sent:</b> Monday, December 31, 2018 1:39 PM<br>
<b>To:</b> 'Anand Malagi' <amalagi@commvault.com>; gluster-users@gluster.org<br>
<b>Subject:</b> RE: replace-brick operation issue...<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Can someone please help here ??<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> <a href="mailto:gluster-users-bounces@gluster.org">
gluster-users-bounces@gluster.org</a> <<a href="mailto:gluster-users-bounces@gluster.org">gluster-users-bounces@gluster.org</a>>
<b>On Behalf Of </b>Anand Malagi<br>
<b>Sent:</b> Friday, December 21, 2018 3:44 PM<br>
<b>To:</b> <a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a><br>
<b>Subject:</b> [Gluster-users] replace-brick operation issue...<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Hi Friends,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Please note that, when replace-brick operation was tried for one of the bad brick present in distributed disperse EC volume, the command actually failed but the brick daemon of new replaced brick came online.<o:p></o:p></p>
<p class="MsoNormal">Please help to understand in what situations this issue may arise and proposed solution if possible ? :<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoPlainText"><b>glusterd.log :<o:p></o:p></b></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">[2018-12-11 11:04:43.774120] I [MSGID: 106503] [glusterd-replace-brick.c:147:__glusterd_handle_replace_brick] 0-management: Received replace-brick commit force request.<o:p></o:p></p>
<p class="MsoPlainText">[2018-12-11 11:04:44.784578] I [MSGID: 106504] [glusterd-utils.c:13079:rb_update_dstbrick_port] 0-glusterd: adding dst-brick port no 0<o:p></o:p></p>
<p class="MsoPlainText">…<o:p></o:p></p>
<p class="MsoPlainText">[2018-12-11 11:04:46.457537] E [MSGID: 106029] [glusterd-utils.c:7981:glusterd_brick_signal] 0-glusterd: Unable to open pidfile: /var/run/gluster/vols/AM6_HyperScale/am6sv0004sds.saipemnet.saipem.intranet-ws-disk3-ws_brick.pid [No such
file or directory]<o:p></o:p></p>
<p class="MsoPlainText">[2018-12-11 11:04:53.089810] I [glusterd-utils.c:5876:glusterd_brick_start] 0-management: starting a fresh brick process for brick /ws/disk15/ws_brick<o:p></o:p></p>
<p class="MsoPlainText">…<o:p></o:p></p>
<p class="MsoPlainText"><span style="background:yellow;mso-highlight:yellow">[2018-12-11 11:04:53.117935] W [socket.c:595:__socket_rwv] 0-socket.management: writev on 127.0.0.1:864 failed (Broken pipe)</span><o:p></o:p></p>
<p class="MsoPlainText">[2018-12-11 11:04:54.014023] I [socket.c:2465:socket_event_handler] 0-transport: EPOLLERR - disconnecting now<o:p></o:p></p>
<p class="MsoPlainText">[2018-12-11 11:04:54.273190] I [MSGID: 106005] [glusterd-handler.c:6120:__glusterd_brick_rpc_notify] 0-management: Brick am6sv0004sds.saipemnet.saipem.intranet:/ws/disk15/ws_brick has disconnected from glusterd.<o:p></o:p></p>
<p class="MsoPlainText">[2018-12-11 11:04:54.297603] E [MSGID: 106116] [glusterd-mgmt.c:135:gd_mgmt_v3_collate_errors] 0-management: Commit failed on am6sv0006sds.saipemnet.saipem.intranet. Please check log file for details.<o:p></o:p></p>
<p class="MsoPlainText">[2018-12-11 11:04:54.350666] I [MSGID: 106143] [glusterd-pmap.c:278:pmap_registry_bind] 0-pmap: adding brick /ws/disk15/ws_brick on port 49164<o:p></o:p></p>
<p class="MsoPlainText"><span style="background:yellow;mso-highlight:yellow">[2018-12-11 11:05:01.137449] E [MSGID: 106123] [glusterd-mgmt.c:1519:glusterd_mgmt_v3_commit] 0-management: Commit failed on peers<o:p></o:p></span></p>
<p class="MsoPlainText"><span style="background:yellow;mso-highlight:yellow">[2018-12-11 11:05:01.137496] E [MSGID: 106123] [glusterd-replace-brick.c:660:glusterd_mgmt_v3_initiate_replace_brick_cmd_phases] 0-management: Commit Op Failed</span><o:p></o:p></p>
<p class="MsoPlainText">[2018-12-11 11:06:12.275867] I [MSGID: 106499] [glusterd-handler.c:4370:__glusterd_handle_status_volume] 0-management: Received status volume req for volume AM6_HyperScale<o:p></o:p></p>
<p class="MsoPlainText">[2018-12-11 13:35:51.529365] I [MSGID: 106499] [glusterd-handler.c:4370:__glusterd_handle_status_volume] 0-management: Received status volume req for volume AM6_HyperScale<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="background:yellow;mso-highlight:yellow">gluster volume replace-brick AM6_HyperScale am6sv0004sds.saipemnet.saipem.intranet:/ws/disk3/ws_brick am6sv0004sds.saipemnet.saipem.intranet:/ws/disk15/ws_brick commit force<o:p></o:p></span></p>
<p class="MsoNormal">Replace brick failure, brick [/ws/disk3], volume [AM6_HyperScale]<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">“gluster volume status” now shows a new disk active /ws/disk15<o:p></o:p></p>
<p class="MsoPlainText">The replacement appears to be successful, looks like healing started<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><img border="0" width="632" height="28" style="width:6.5833in;height:.2916in" id="Picture_x0020_1" src="cid:image001.png@01D4A84C.A15F8680"><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks and Regards,<o:p></o:p></p>
<p class="MsoNormal">--Anand<o:p></o:p></p>
<div>
<div>
<div>
<div>
<p class="MsoNormal">********************************Legal Disclaimer********************************<o:p></o:p></p>
<div>
<p class="MsoNormal">"This communication may contain confidential and privileged material for the<o:p></o:p></p>
<div>
<p class="MsoNormal">sole use of the intended recipient. Any unauthorized review, use or distribution<o:p></o:p></p>
<div>
<p class="MsoNormal">by others is strictly prohibited. If you have received the message by mistake,<o:p></o:p></p>
<div>
<p class="MsoNormal">please advise the sender by reply email and delete the message. We may process<o:p></o:p></p>
<div>
<p class="MsoNormal">information in the email header of business emails sent and received by us<o:p></o:p></p>
<div>
<p class="MsoNormal">(including the names of recipient and sender, date and time of the email) for<o:p></o:p></p>
<div>
<p class="MsoNormal">the purposes of evaluating our existing or prospective business relationship.<o:p></o:p></p>
<div>
<p class="MsoNormal">The lawful basis we rely on for this processing is our legitimate interests. For<o:p></o:p></p>
<div>
<p class="MsoNormal">more information about how we use personal information please read our privacy<o:p></o:p></p>
<div>
<p class="MsoNormal">policy <a href="https://www.commvault.com/privacy-policy">https://www.commvault.com/privacy-policy</a>. Thank you."<o:p></o:p></p>
<div>
<p class="MsoNormal">********************************************************************************<o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<div><div><div><div>********************************Legal Disclaimer********************************<div>"This communication may contain confidential and privileged material for the<div>sole use of the intended recipient. Any unauthorized review, use or distribution<div>by others is strictly prohibited. If you have received the message by mistake,<div>please advise the sender by reply email and delete the message. We may process<div>information in the email header of business emails sent and received by us<div>(including the names of recipient and sender, date and time of the email) for<div>the purposes of evaluating our existing or prospective business relationship.<div>The lawful basis we rely on for this processing is our legitimate interests. For<div>more information about how we use personal information please read our privacy<div>policy https://www.commvault.com/privacy-policy. Thank you."<div>********************************************************************************</body>
</html>