<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Arial,Helvetica,sans-serif;" dir="ltr">
<p>Hi,</p>
<p><br>
</p>
<p>I have upgraded to Gluster 3.8.10 today and ran the add-brick procedure in a volume contains few VMs.</p>
<p>After the completion of rebalance, i have rebooted the VMs, some of ran just fine, and others just crashed.</p>
<p>Windows boot to recovery mode and Linux throw xfs errors and does not boot.</p>
<p>I ran the test again and it happened just as the first one, but i have noticed only VMs doing disk IOs are affected by this bug.</p>
<p>The VMs in power off mode started fine and even md5 of the disk file did not change after the rebalance.</p>
<p><br>
</p>
<p>anyone else can confirm this ?</p>
<p><br>
</p>
<p><br>
</p>
<p>Volume info:</p>
<p></p>
<div>Volume Name: vmware2</div>
<div>Type: Distributed-Replicate</div>
<div>Volume ID: 02328d46-a285-4533-aa3a-fb9bfeb688bf</div>
<div>Status: Started</div>
<div>Snapshot Count: 0</div>
<div>Number of Bricks: 22 x 2 = 44</div>
<div>Transport-type: tcp</div>
<div>Bricks:</div>
<div>Brick1: gluster01:/mnt/disk1/vmware2</div>
<div>Brick2: gluster03:/mnt/disk1/vmware2</div>
<div>Brick3: gluster02:/mnt/disk1/vmware2</div>
<div>Brick4: gluster04:/mnt/disk1/vmware2</div>
<div>Brick5: gluster01:/mnt/disk2/vmware2</div>
<div>Brick6: gluster03:/mnt/disk2/vmware2</div>
<div>Brick7: gluster02:/mnt/disk2/vmware2</div>
<div>Brick8: gluster04:/mnt/disk2/vmware2</div>
<div>Brick9: gluster01:/mnt/disk3/vmware2</div>
<div>Brick10: gluster03:/mnt/disk3/vmware2</div>
<div>Brick11: gluster02:/mnt/disk3/vmware2</div>
<div>Brick12: gluster04:/mnt/disk3/vmware2</div>
<div>Brick13: gluster01:/mnt/disk4/vmware2</div>
<div>Brick14: gluster03:/mnt/disk4/vmware2</div>
<div>Brick15: gluster02:/mnt/disk4/vmware2</div>
<div>Brick16: gluster04:/mnt/disk4/vmware2</div>
<div>Brick17: gluster01:/mnt/disk5/vmware2</div>
<div>Brick18: gluster03:/mnt/disk5/vmware2</div>
<div>Brick19: gluster02:/mnt/disk5/vmware2</div>
<div>Brick20: gluster04:/mnt/disk5/vmware2</div>
<div>Brick21: gluster01:/mnt/disk6/vmware2</div>
<div>Brick22: gluster03:/mnt/disk6/vmware2</div>
<div>Brick23: gluster02:/mnt/disk6/vmware2</div>
<div>Brick24: gluster04:/mnt/disk6/vmware2</div>
<div>Brick25: gluster01:/mnt/disk7/vmware2</div>
<div>Brick26: gluster03:/mnt/disk7/vmware2</div>
<div>Brick27: gluster02:/mnt/disk7/vmware2</div>
<div>Brick28: gluster04:/mnt/disk7/vmware2</div>
<div>Brick29: gluster01:/mnt/disk8/vmware2</div>
<div>Brick30: gluster03:/mnt/disk8/vmware2</div>
<div>Brick31: gluster02:/mnt/disk8/vmware2</div>
<div>Brick32: gluster04:/mnt/disk8/vmware2</div>
<div>Brick33: gluster01:/mnt/disk9/vmware2</div>
<div>Brick34: gluster03:/mnt/disk9/vmware2</div>
<div>Brick35: gluster02:/mnt/disk9/vmware2</div>
<div>Brick36: gluster04:/mnt/disk9/vmware2</div>
<div>Brick37: gluster01:/mnt/disk10/vmware2</div>
<div>Brick38: gluster03:/mnt/disk10/vmware2</div>
<div>Brick39: gluster02:/mnt/disk10/vmware2</div>
<div>Brick40: gluster04:/mnt/disk10/vmware2</div>
<div>Brick41: gluster01:/mnt/disk11/vmware2</div>
<div>Brick42: gluster03:/mnt/disk11/vmware2</div>
<div>Brick43: gluster02:/mnt/disk11/vmware2</div>
<div>Brick44: gluster04:/mnt/disk11/vmware2</div>
<div>Options Reconfigured:</div>
<div>cluster.server-quorum-type: server</div>
<div>nfs.disable: on</div>
<div>performance.readdir-ahead: on</div>
<div>transport.address-family: inet</div>
<div>performance.quick-read: off</div>
<div>performance.read-ahead: off</div>
<div>performance.io-cache: off</div>
<div>performance.stat-prefetch: off</div>
<div>cluster.eager-lock: enable</div>
<div>network.remote-dio: enable</div>
<div>features.shard: on</div>
<div>cluster.data-self-heal-algorithm: full</div>
<div>features.cache-invalidation: on</div>
<div>ganesha.enable: on</div>
<div>features.shard-block-size: 256MB</div>
<div>client.event-threads: 2</div>
<div>server.event-threads: 2</div>
<div>cluster.favorite-child-policy: size</div>
<div>storage.build-pgfid: off</div>
<div>network.ping-timeout: 5</div>
<div>cluster.enable-shared-storage: enable</div>
<div>nfs-ganesha: enable</div>
<div>cluster.server-quorum-ratio: 51%</div>
<div><br>
</div>
<div><br>
</div>
<div>Adding bricks:</div>
<div><span>gluster volume add-brick vmware2 replica 2 gluster01:/mnt/disk11/vmware2 gluster03:/mnt/disk11/vmware2 gluster02:/mnt/disk11/vmware2 gluster04:/mnt/disk11/vmware2</span><br>
</div>
<div><span><br>
</span></div>
<div><span><br>
</span></div>
<div><span>starting fix layout:</span></div>
<div><span><span>gluster volume rebalance vmware2 fix-layout start</span><br>
</span></div>
<div><span><span><br>
</span></span></div>
<div>Starting rebalance:</div>
<div><span>gluster volume rebalance vmware2 start</span><br>
</div>
<div><span><br>
</span></div>
<div><span><br>
</span></div>
<div><br>
</div>
<div><br>
</div>
<br>
<p></p>
<p><br>
</p>
<div id="Signature"><br>
<div class="ecxmoz-signature">-- <br>
<br>
<font color="#3366ff"><font color="#000000">Respectfully<b><br>
</b><b>Mahdi A. Mahdi</b></font></font><font color="#3366ff"><br>
<br>
</font><font color="#3366ff"></font></div>
</div>
</div>
</body>
</html>