<html xmlns="http://www.w3.org/1999/xhtml" xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office"><head><!--[if gte mso 9]><xml><o:OfficeDocumentSettings><o:AllowPNG/><o:PixelsPerInch>96</o:PixelsPerInch></o:OfficeDocumentSettings></xml><![endif]--></head><body>
As nobody chimed in, let me reply inline.<div><br><br>Best Regards,</div><div>Strahil Nikolov<br><p class="yahoo-quoted-begin" style="font-size: 15px; color: #715FFA; padding-top: 15px; margin-top: 0">On Sunday, April 23, 2023, 2:35 AM, Peter P <peter.a.pfeiffer@gmail.com> wrote:</p><blockquote class="iosymail"><div id="yiv6381549598">
<div>
<div class="yiv6381549598WordSection1">
Good afternoon,<br>
<br>
I am looking for additional information about glusterfs, arbiters
and thin-arbiters. The current stable release is gluster 11, so I
am considering that version for deployment. My planned setup is:
4 storage servers in distributed + replicated mode.<br>
<br>
Server1, server2 : replica 2, arbiter 1<br>
Server3, server4 : replica 2, arbiter 1<br>
<br>
Since having replica 2 is not recommended due to split-brains I
will have an arbiter.
<br>
<br>
Generic questions:<br>
<ul>
<li>
Is arbiter or thin-arbiter recommended in a production, large
volume storage environment?</li></ul><div>- Both were introduced a long time ago. Most users prefer full arbiter, so healing is far more optimal (only changed files will be healed).</div><ul>
<li>Is thin arbiter code stable and deployment ready?</li></ul><div>- I know that it’s in use but full arbiter has been introduced earlier and has a wider adoption.</div><div><br></div><ul>
<li>
Arbiter is file based and stores metadata for each file. In
this scenario I would at least double the default inode count
on the arbiter server. Thin-arbiter on the other hand is brick
based but I have not found enough information if its inode
usage is as inode hungry as the arbiter configuration. I am
thinking, it should not be as it is brick based. So do I need
to increase the inode count when using thin-arbiters? If yes,
what is recommended?</li></ul><div>- Full arbiter is sensitive to network latency and disk speed (a lot of small I/O for those inodes). Increase macpct (XFS) on arbiter bricks and prefer using a SSD/NVME. As full arbiter doesn’t store any data , you can set the maxpct to around 75%</div><div>- Thin arbiter doesn’t have a brick and when you create it, you just specify the replica id file ( see <a href="https://docs.gluster.org/en/main/Administrator-Guide/Thin-Arbiter-Volumes/" target="_blank" rel="noreferrer noopener">https://docs.gluster.org/en/main/Administrator-Guide/Thin-Arbiter-Volumes/</a> )<div id="temp-enhancr-placeholder" class="enhancr-placeholder-medium" data-size="medium"></div></div><ul>
<li>
I've read old bug reports reporting that thin arbiter was not
ready to serve multiple trusted pools. Is this still the
case? I may configure multiple trusted pools in the future.</li></ul><div>- I saw Kadalu uses their own thin arbiter and I never saw issues. I doubt I was the only one using it, so it should be fine.</div><ul>
<li>I have many linux boxes running different linux
distributions and releases. Ideally the assortment of boxes
would mount the same gluster pool/volume. I looked for
information about older versions of gluster clients running on
a range of older distributions mounting the most recent
gluster 11 pool/volume? Does that work? Can gluster client
(version 10, 9, 8, 7, etc.) mount gluster 11 volume and run
without significant issues? I understand that older versions
of client will not have the most recent features. Most recent
features aside, is such configuration supported/stable?</li>
</ul><div>- For that purpose gluster has 2 settings:</div><div>cluster.max-op-version -> the max compatibility version you can set your cluster based of the oldest client’s version</div><div>cluster.op-version -> the cluster’s compatibility version</div><div>As long you keep the cluster.op-version compatible with your client - it should work.</div>
<br>
<b>Thin-arbiter approach:</b> If I go with the thin-arbiter
configuration I will use a 5<sup>th</sup> server as this server
can be outside of the trusted pool and can be shared among
multiple trusted pools<br>
<br>
Server1, server2: replica 2, thin-arbiter server5<br>
Server3, server4: replica 2, thin-arbiter server5<br>
<br>
<br>
<b>Old arbiter approach:</b> If I go with the older arbiter
configuration, I am considering using 2 of the storage servers to
act as both replica and an arbiter. Is that configuration
possible/supported and reasonable?
<br>
<br>
<i>Server1</i>, server2: replica 2, arbiter <b>server3</b> <br>
<b>Server3</b>, server4: replica 2, arbiter <i>server1</i><br>
<br>- Yes, as long as you have a dedicated brick (in this example server3 should have a data brick and arbiter brick)</div><div class="yiv6381549598WordSection1"><br>
In this configuration, I am considering using server3 to be
arbiter for server{1,2} replica 2, and using server1 to be
arbiter for server{3,4} replica 2.
<br>
<br>
Questions:<br>
<ul>
<li>Is this a reasonable/recommended configuration?<span style="font-family:Symbol;"><span style=""></span></span></li></ul><div>- It’s used quite often</div><ul>
<li><span style="font-family:Symbol;"><span style=""><span style="font:7.0pt New;"> </span></span></span>Should
the arbiter metadata folder be inside or outside of the
volume?
</li>
<ul>
<li>In detail. Say server{1,2} replica has 1 brick each
<b>/gluster/brick1 </b>with<b> /gfs1vol1 </b>as the volume</li>
<li>
Should the arbiter metadata folder location be:
/gluster/arbiter/gfs1vol1 (outside of the volume path)
or
<b>/gfs1vol1/</b>arbiter1/ (inside the volume path)</li>
</ul>
</ul>
- Always keep bricks as separate mount points. For example:</div><div class="yiv6381549598WordSection1">/dev/vg/lv mounted on /bricks/databricks with directory vol1/brick1</div><div class="yiv6381549598WordSection1">/dev/vg/lv2 mounted on /bricks/arbiterbricks with directory vol1/arbiterbrick1</div><div class="yiv6381549598WordSection1"><br></div><div class="yiv6381549598WordSection1">The idea is that if the device is not mounted, the brick directory will be missing and the mess will be far less.</div><div class="yiv6381549598WordSection1"><br>
<span style="font-size:12.0pt;color:#333333;background:#FEFEFE;">Thank
you for your thoughts,</span><br>
<span style="font-size:12.0pt;color:#333333;background:#FEFEFE;">Peter</span><br>
</div>
<div id="yiv6381549598DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2"><br><table style="border-top:1px solid #D3D4DE;"><tbody><tr><td style="width:55px;padding-top:13px;"><a href="http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient" target="_blank" rel="noreferrer noopener"><img src="https://s-install.avcdn.net/ipm/preview/icons/icon-envelope-tick-green-avg-v1.png" alt="" width="46" height="29" style="width:46px;min-height:29px;"></a></td><td style="width:470px;padding-top:12px;color:#41424e;font-size:13px;font-family:Arial, Helvetica, sans-serif;line-height:18px;">Virus-free.<a href="http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient" style="color:#4453ea;" target="_blank" rel="noreferrer noopener">www.avg.com</a></td></tr></tbody></table><a href="#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2" target="_blank" rel="noreferrer noopener"> </a></div></div>
</div>________<br><br><br><br>Community Meeting Calendar:<br><br>Schedule -<br>Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC<br>Bridge: <a href="https://meet.google.com/cpu-eiue-hvk" target="_blank" rel="noreferrer noopener">https://meet.google.com/cpu-eiue-hvk</a><br>Gluster-users mailing list<br><a ymailto="mailto:Gluster-users@gluster.org" href="mailto:Gluster-users@gluster.org" target="_blank" rel="noreferrer noopener">Gluster-users@gluster.org</a><br><a href="https://lists.gluster.org/mailman/listinfo/gluster-users" target="_blank" rel="noreferrer noopener">https://lists.gluster.org/mailman/listinfo/gluster-users</a><br><blockquote></blockquote></blockquote></div>
</body></html>