[Gluster-users] advice needed on configuring large gluster cluster
cobanserkan at gmail.com
Thu Mar 16 04:39:33 UTC 2017
> Thanks Serkan, I appreciate the feedback. Do you know if there are any
> associated performance issues with odd values of parity disks? e.g any
> reason to not go with 16+3?, also are there any things to watch for in terms
> of the number of distributed disperse sets?
AFAIK, there is no performance concern for n for m+n configuration, n affects
how many bricks/nodes will be tolerated in case of failure. Number of
are related with your workload and capacity. EC is not recommended for
small files and random
I/O so as I said before do your test with your prod workload.
> On 15 March 2017 at 13:32, Serkan Çoban <cobanserkan at gmail.com> wrote:
>> Please find my comments inline.
>> > Hi
>> > we have a new gluster cluster we are planning on deploying. We will
>> > have 24
>> > nodes each with JBOD, 39 8TB drives and 6, 900GB SSDs, and FDR IB
>> > We will not be using all of this as one volume , but I thought initially
>> > of
>> > using a distributed disperse volume.
>> > Never having attempted anything on this scale I have a couple of
>> > questions
>> > regarding EC and distibuted disperse volumes.
>> > Does a distributed dispersed volume have to start life as distributed
>> > dispersed, or can I take a disperse volume and make it distributed by
>> > adding bricks?
>> Yes you can start with one subvolume and later you can increase the
>> But be careful about planning, if you start with m+n EC configuration,
>> you can add
>> another m+n subvolume to it.
>> > Does an EC scheme of 24+4 seem reasonable? One requirement we will have
>> > is
>> > the need to tolerate two nodes down at once, as the nodes share a
>> > chassis.
>> > I assume that distributed disperse volumes can be expanded in a similar
>> > fashion to distributed replicate volumes by adding additional disperse
>> > brick
>> > sets?
>> It is recommended in m+n configuration m should be power of two.
>> You can do 16+4 or 8+2. Higher m will cause slower healing but
>> parallel self heal
>> of EC volumes in 3.9+ will help. 8+2 configuration with one brick from
>> every node will
>> tolerate loss of two nodes.
>> > I would also like to consider adding a hot-tier using the SSDs, I
>> > confess I
>> > have not done much reading on tiering, but am hoping I can use a
>> > different
>> > volume form for the hot tier. Can I use create a disperse, or a
>> > distributed
>> > replicated? If I am smoking rainbows then I can consider setting up a
>> > SSD
>> > only distributed disperse volume.
>> EC performance is quite good for our workload,I did not try any tier
>> in front of it
>> Test your workload without tier, if it works then KISS
>> > I'd also appreciate any feedback on likely performance issues and tuning
>> > tips?
>> You can find kernel performance tuning here:
>> You may also change client.event-threads, server.event-threads and
>> heal related parameters
>> but do not forget to test your workload after and before changing those
>> > Many Thanks
>> > -Alastair
>> > _______________________________________________
>> > Gluster-users mailing list
>> > Gluster-users at gluster.org
>> > http://lists.gluster.org/mailman/listinfo/gluster-users
More information about the Gluster-users