[Gluster-users] Gluster setup for virtualization cluster

[Strahil Nikolov]

On February 17, 2020 4:59:17 AM GMT+02:00, Markus Kern <gluster at military.de> wrote:
>Greetings!
>
>I am currently evaluating our options to replace our old mixture of IBM
>
>SAN storage boxes. This will be a strategic decision for the next
>years.
>One of the solutions I am reviewing is a GlusterFS installation.
>
>Planned usage:
>- Central NFS server for around 25 systems providing around 400 docker 
>containers
>- Central storage for a small VMWare vCenter cluster and a RedHat 
>virtualization cluster. In total maybe around 15 machines
>
>The following requirements ensue from this:
>- Fast storage
>- High availability
>
>
>After reading all kind of tutorials and documentation, I came to the 
>conclusion that for the expected traffic a "Distributed Replicate 
>Volume" is the proper setup.
>
>Nothing has been purchased but I think about following small setup for 
>the beginning (call it PoC):
>
>4 x server, each with 8 x 1.8TB 10k SAS disks in a RAID60
>Two 10 GBit interfaces per server: One for communication betweens the 4
>
>systems only (separate VLAN), the other one for regular traffic between
>
>clients and servers.
>
>
>Does this all make sense?
>Generally speaking: Is such a setup capable of providing fast enough 
>storage for a virtualization cluster?
>Do you have any hints?
>
>Thanks
>
>Markus
>
>
>
>________
>
>Community Meeting Calendar:
>
>APAC Schedule -
>Every 2nd and 4th Tuesday at 11:30 AM IST
>Bridge: https://bluejeans.com/441850968
>
>NA/EMEA Schedule -
>Every 1st and 3rd Tuesday at 01:00 PM EDT
>Bridge: https://bluejeans.com/441850968
>
>Gluster-users mailing list
>Gluster-users at gluster.org
>https://lists.gluster.org/mailman/listinfo/gluster-users

Hi Markus,

It all depends on the IOPS requirements and the capabilities of the disks. Are these  spinning or SSD/NVME ?

For me a Raid60  is an overkill, as  you need 'replica  2 arbiter  1' or pure 'replica 3' when you care about your data.

I'd recommend you to add a separate node (either physical or Virtual)  as an Arbiter. 

Most probably a raid5/50  will be enough, but you can adapt in the POC stage.

If  you plan to use  NFS - consider NFS Ganesha in a HA cluster,  as  the built-in NFS is deprecated (and in some  distributions you need to rebuild from source).Also consider it as a 'gateway' which means that the NFS Ganesha should have more NICs.

If NFS Ganesha is used - you won't need the second group of NICs on the Gluster nodes.  Ganesha directly speeks with all nodes in the storage pool.

So, you can try with 3  Gluster nodes  and a node for NFS Ganesha and later scale-out.


Best Regards,
Strahil Nikolov