<html><head></head><body><div class="yahoo-style-wrap" style="font-family:courier new, courier, monaco, monospace, sans-serif;font-size:16px;"><div style="font-family:'Helvetica Neue', Helvetica, Arial, sans-serif;font-size:13px;color:#26282a;">Hi Eduardo,<br><div><div dir="ltr"><br></div><div dir="ltr">> I am looking into a new gluster deployment to replace an ancient one.<br></div><div dir="ltr"><br></div><div dir="ltr"> > For this deployment I will be using some repurposed servers I<br></div><div dir="ltr">>already have in stock. The disk specs are 12 * 3 TB SATA disks. No HW<br></div><div dir="ltr">>RAID controller. They also have some SSD which would be nice to leverage<br></div><div dir="ltr">>as cache or similar to improve performance, since it is already there.<br></div><div dir="ltr">>Advice on how to leverage the SSDs would be greatly appreciated.<br></div><div dir="ltr"><br></div><div dir="ltr">Gluster Tiering was dropped in favour of the LVM cache.keep in mind that in RHEL/CentOS 7 you should be careful for migration_threshold value sometimes is smaller than the chunk size.</div><div dir="ltr">For details check: <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1668163" rel="nofollow" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=1668163</a></div><div dir="ltr"><br></div><div dir="ltr">> One of the design choices I have to make is using 3 nodes for a<br></div><div dir="ltr">>replica-3 with JBOD, or using 2 nodes with a replica-2 and using SW RAID<br></div><div dir="ltr">>6 for the disks, maybe adding a 3rd node with a smaller amount of disk<br></div><div dir="ltr">>as metadata node for the replica set. I would love to hear advice on the<br></div><div dir="ltr">>pros and cons of each setup from the gluster experts.<br></div><div dir="ltr"><br></div><div dir="ltr">If you go with replica3 - your reads will be from 3 servers - thus higher speeds</div><div dir="ltr">If you chose replica2 - you will eventually enter a split brain (Not a good one)</div><div dir="ltr">If you choose replica2 arbiter1 (old replica 3 arbiter1) - you will read from only 2 servers , but save bandwidth. keep in mind that you need high-bandwidth NICs (as bonding/teaming is balancing based on MAC, IP and Port which in your case will all be the same)</div><div dir="ltr">Another option is to use GlusterD2 with replica2 and remote arbiter (for example in the cloud or somewhere away). This setup does not require the arbiter to responce in a timely manner and is used only if 1 data brick is down.<br></div><div dir="ltr"><br></div><div dir="ltr"> > The data will be accessed from 4 to 6 systems with native gluster,<br></div><div dir="ltr">>not sure if that makes any difference.<br></div><div dir="ltr"><br></div><div dir="ltr"> > The amount of data I have to store there is currently 20 TB, with<br></div><div dir="ltr">>moderate growth. iops are quite low so high performance is not an issue.<br></div><div dir="ltr">>The data will fit in any of the two setups.</div><div dir="ltr"><br></div><div dir="ltr">I would go with replica3 if NICs are 10gbit/s or bigger and replica2 arbiter1 if NICs are smaller.GlusterD2 is still new and might be too risky for production (Gluster Devs can correct me here).<br></div><div dir="ltr"><br></div><div dir="ltr">My current setup is with Gluster v6.1 on Ovirt in a replica2 arbiter1 with 6 NICs x 1gbit/s ports (consumer grade) and in order to overcome the load-balancing issue , I'm using multiple thin LVs ontop a single NVMe - each LV is a gluster brick . Each gluster volume has a separate tcp port and thus the teaming device is load-balancing traffic on another NIC.<br></div><div dir="ltr">This allows me to stripe my data on VM level , but this setup is only OK for labs .</div><div dir="ltr"><br></div><div dir="ltr"><br></div><div dir="ltr"><br></div><div dir="ltr"> Best Regards,</div><div dir="ltr">Strahil Nikolov<br></div></div>
</div><div id="ydpa022c06yahoo_quoted_0130822274" class="ydpa022c06yahoo_quoted">
</div></div></body></html>