<html dir="ltr">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="text-align:left; direction:ltr;">
<div>Hi,</div>
<div><br>
</div>
<div>We have an old Gluster cluster setup, running a replica 2 across two datacenters, and currently on version 4.1.5</div>
<div><br>
</div>
<div>I need to add an arbiter to this setup, but I'm concerned about the performance impact of this on the volumes.</div>
<div><br>
</div>
<div>I recently set up a new cluster, for a different purpose, and decided to test adding an arbiter to the volume after adding in some data.</div>
<div>Had a volume with ~435,000 files totaling about 12TB.</div>
<div>Adding the arbiter initiated a heal-operation that took almost 3 hours.</div>
<div><br>
</div>
<div>The older cluster, one of the volumes is about 14TB, but ~45,5 million files.</div>
<div><br>
</div>
<div>Since arbiter is only concerned about metadata and checksums, I'm concerned about the fact that we have 100 times the amount of files, i.e. 100 times the amount of I/O operations to execute during healing, and possibly 100 times the time which would mean
about 12,5 days.</div>
<div><br>
</div>
<div>Another "issue" is that the 'gluster volume heal <vol-name> info summary' command seems to "count" all the files, so the command can take a very long time to complete.</div>
<div>The metrics-scraping script I created for us, with a timeout of 110seconds, fails to complete when a volume has over ~800-900 files unsynced (which happens regularily when taking one cluster-node down for patching).</div>
<div><br>
</div>
<div><br>
</div>
<div>Does anyone have any experience with adding arbiter afterwards, performance impact, time to heal, etc.</div>
<div>Also other ways to get the status on healing.</div>
<div><br>
</div>
<div>Any advice would be appreciated.</div>
<div><span><br>
<br>
Best regards<br>
-- <br>
<b>THORGEIR MARTHINUSSEN</b><br>
<div>Senior Systems Consultant</div>
<b>BASEFARM</b></span></div>
</body>
</html>