[Gluster-users] Advice for setup: SW RAID 6 vs JBOD

[Strahil Nikolov]

Hi Eduardo,

>    I am looking into a new gluster deployment to replace an ancient one.

  >  For this deployment I will be using some repurposed servers I
>already have in stock. The disk specs are 12 * 3 TB SATA disks. No HW
>RAID controller. They also have some SSD which would be nice to leverage
>as cache or similar to improve performance, since it is already there.
>Advice on how to leverage the SSDs would be greatly appreciated.

Gluster Tiering was dropped in favour of the LVM cache.keep in mind that in RHEL/CentOS 7 you should be careful for migration_threshold value sometimes is smaller than the chunk size.For details check: https://bugzilla.redhat.com/show_bug.cgi?id=1668163
>    One of the design choices I have to make is using 3 nodes for a
>replica-3 with JBOD, or using 2 nodes with a replica-2 and using SW RAID
>6 for the disks, maybe adding a 3rd node with a smaller amount of disk
>as metadata node for the replica set. I would love to hear advice on the
>pros and cons of each setup from the gluster experts.

If you go with replica3 - your reads will be from 3 servers - thus higher speedsIf you chose replica2 - you will eventually enter a split brain (Not a good one)If you choose replica2 arbiter1 (old replica 3 arbiter1) - you will read from only 2 servers , but save bandwidth. keep in mind that you need high-bandwidth NICs (as bonding/teaming is balancing based on MAC, IP and Port which in your case will all be the same)Another option is to use GlusterD2 with replica2 and remote arbiter (for example in the cloud or somewhere away). This setup does not require the arbiter to responce in a timely manner and is used only if 1 data brick is down.

 >   The data will be accessed from 4 to 6 systems with native gluster,
>not sure if that makes any difference.

  >  The amount of data I have to store there is currently 20 TB, with
>moderate growth. iops are quite low so high performance is not an issue.
>The data will fit in any of the two setups.
I would go with replica3 if NICs are 10gbit/s or bigger and replica2 arbiter1 if NICs are smaller.GlusterD2 is still new and might be too risky for production (Gluster Devs can correct me here).

My current setup is with Gluster v6.1 on Ovirt in a replica2 arbiter1 with 6 NICs x 1gbit/s ports (consumer grade) and in order to overcome the load-balancing issue , I'm using multiple thin LVs ontop a single NVMe - each LV is a gluster brick . Each gluster  volume has a separate tcp port and thus the teaming device is load-balancing traffic on another NIC.
This allows me to stripe my data on VM level , but this setup is only OK for labs .


 Best Regards,Strahil Nikolov
  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20190605/f5060c02/attachment.html>