<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body>
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Arial,Helvetica,sans-serif;" dir="ltr">
<p>Hi,</p>
<p><br>
</p>
<p>Attached are the logs for both the rebalance and the mount.</p>
<p><br>
</p>
<div id="Signature"><br>
<div class="ecxmoz-signature">-- <br>
<br>
<font color="#3366ff"><font color="#000000">Respectfully<b><br>
</b><b>Mahdi A. Mahdi</b></font></font><font color="#3366ff"><br>
<br>
</font><font color="#3366ff"></font></div>
</div>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Krutika Dhananjay <kdhananj@redhat.com><br>
<b>Sent:</b> Friday, May 26, 2017 1:12:28 PM<br>
<b>To:</b> Mahdi Adnan<br>
<b>Cc:</b> gluster-user; Gandalf Corvotempesta; Lindsay Mathieson; Kevin Lemonnier<br>
<b>Subject:</b> Re: Rebalance + VM corruption - current status and request for feedback</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div>Could you provide the rebalance and mount logs?<br>
<br>
</div>
-Krutika<br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, May 26, 2017 at 3:17 PM, Mahdi Adnan <span dir="ltr">
<<a href="mailto:mahdi.adnan@outlook.com" target="_blank">mahdi.adnan@outlook.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div id="m_4798580117692497506divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Arial,Helvetica,sans-serif" dir="ltr">
<p>Good morning,</p>
<p><br>
</p>
<p>So i have tested the new Gluster 3.10.2, and after starting rebalance two VMs were paused due to storage error and third one was not responding.</p>
<p>After rebalance completed i started the VMs and it did not boot, and throw an XFS wrong inode error into the screen.</p>
<p><br>
</p>
<p>My setup:</p>
<p>4 nodes running CentOS7.3 with Gluster 3.10.2</p>
<p>4 bricks in distributed replica with group set to virt.</p>
<p>I added the volume to ovirt and created three VMs, i ran a loop to create 5GB file inside the VMs.</p>
<p>Added new 4 bricks to the existing nodes. </p>
<p>Started rebalane "with force to bypass the warning message"</p>
<p>VMs started to fail after rebalancing.</p>
<span class="">
<p><br>
</p>
<p><br>
</p>
<div id="m_4798580117692497506Signature"><br>
<div class="m_4798580117692497506ecxmoz-signature">-- <br>
<br>
<font color="#3366ff"><font color="#000000">Respectfully<b><br>
</b><b>Mahdi A. Mahdi</b></font></font><font color="#3366ff"><br>
<br>
</font><font color="#3366ff"></font></div>
</div>
</span></div>
<hr style="display:inline-block;width:98%">
<div id="m_4798580117692497506divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" color="#000000" face="Calibri, sans-serif"><span class=""><b>From:</b> Krutika Dhananjay <<a href="mailto:kdhananj@redhat.com" target="_blank">kdhananj@redhat.com</a>><br>
<b>Sent:</b> Wednesday, May 17, 2017 6:59:20 AM<br>
<b>To:</b> gluster-user<br>
<b>Cc:</b> Gandalf Corvotempesta; Lindsay Mathieson; Kevin Lemonnier; Mahdi Adnan<br>
</span><b>Subject:</b> Rebalance + VM corruption - current status and request for feedback</font>
<div> </div>
</div>
<div>
<div class="h5">
<div>
<div dir="ltr">
<div>
<div>
<div>
<div>
<div>Hi,<br>
<br>
</div>
In the past couple of weeks, we've sent the following fixes concerning VM corruption upon doing rebalance -
<a href="https://review.gluster.org/#/q/status:merged+project:glusterfs+branch:master+topic:bug-1440051" target="_blank">
https://review.gluster.org/#/<wbr>q/status:merged+project:<wbr>glusterfs+branch:master+topic:<wbr>bug-1440051</a><br>
<br>
</div>
These fixes are very much part of the latest 3.10.2 release.<br>
<br>
</div>
Satheesaran within Red Hat also verified that they work and he's not seeing corruption issues anymore.<br>
<br>
</div>
I'd like to hear feedback from the users themselves on these fixes (on your test environments to begin with) before even changing the status of the bug to CLOSED.<br>
<br>
</div>
<div>Although 3.10.2 has a patch that prevents rebalance sub-commands from being executed on sharded volumes, you can override the check by using the 'force' option.<br>
<br>
</div>
<div>For example,<br>
<br>
</div>
<div># gluster volume rebalance myvol start force<br>
<br>
</div>
<div>Very much looking forward to hearing from you all.<br>
<br>
</div>
<div>Thanks,<br>
</div>
<div>Krutika<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</body>
</html>