[Gluster-users] advice needed on configuring large gluster cluster

[Serkan Çoban]

Please find my comments inline.

> Hi
>
> we have a new gluster cluster we are planning on deploying.  We will have 24
> nodes each with JBOD, 39 8TB drives and 6, 900GB SSDs, and FDR IB
>
> We will not be using all of this as one volume , but I thought initially of
> using a distributed disperse volume.
>
> Never having attempted anything on this scale I have a couple of questions
> regarding EC and distibuted disperse volumes.
>
> Does a distributed dispersed volume have to start life as distributed
> dispersed, or can I  take a disperse volume and make it distributed by
> adding bricks?
Yes you can start with one subvolume and later you can increase the subvolumes.
But be careful about planning, if you start with m+n EC configuration,
you can add
another m+n subvolume to it.
>
> Does an EC scheme of 24+4 seem reasonable?  One requirement we will have is
> the need to tolerate two nodes down at once, as the nodes share a chassis.
> I assume that  distributed disperse volumes can be expanded in a similar
> fashion to distributed replicate volumes by adding additional disperse brick
> sets?
It is recommended in m+n configuration m should be power of two.
You can do 16+4 or 8+2. Higher m will cause slower healing but
parallel self heal
of EC volumes in 3.9+ will help. 8+2 configuration with one brick from
every node will
tolerate loss of two nodes.

>
> I would also like to consider adding a hot-tier using the SSDs,  I confess I
> have not done much reading on tiering, but am hoping I can use a different
> volume form for the hot tier.  Can I use create a disperse, or a distributed
> replicated?   If I am smoking rainbows then I can consider setting up a SSD
> only distributed disperse volume.
EC performance is quite good for our workload,I did not try any tier
in front of it
Test your workload without tier, if it works then KISS
>
> I'd also appreciate any feedback on likely performance issues and tuning
> tips?
You can find kernel performance tuning here:
https://gluster.readthedocs.io/en/latest/Administrator%20Guide/Linux%20Kernel%20Tuning/
You may also change client.event-threads, server.event-threads and
heal related parameters
but do not forget to test your workload after and before changing those values.
>
> Many Thanks
>
> -Alastair
>
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users