[Gluster-users] Low cost, extendable, failure tolerant home cloud storage question

[Diego Zuccato]

Il 04/10/20 21:29, Strahil Nikolov ha scritto:

> In order to be safe you need 'replica 3' or a disperse volumes.
At work I'm using "replica 3 arbiter 1" to balance storage overhead and
data security.

> In both cases extending by 1 brick (brick is not equal to node) is not possible in most cases. For example in 'replica 3' you need to add 3 more bricks (brick is a combination of 'server + directory' and it is recommended to be on separate systems or it's a potential single point of failure). Dispersed volumes also need to be extended in numbers of <disperse count>, so if you have 4+2 (4 bricks , 2 are the maximum you can loose without dataloss ) - you need to add another 6 bricks to extend.
To extend a replica 3 arbiter 1 you only have to add two disks. And have
enough inodes available on the third server. Don't understimate inodes
use, especially if you're using a single partition for all the arbiters!

>> - cheap nodes (with 1-2GB of RAM) able to handle the task (like Rpi, >Odroid XU4 or even HP T610)
> You need a little bit more ram for daily usage and most probably more cores as healing of data in replica is demanding (dispersed volumes are like raid's parity and require some cpu).
>From my experience 8GB is the minimum during healing. Less than that and
you'll get OOM kills and many problems. I'd recommend not less than 16G
for an "arbiter only" server, and 32G for a replica server. These
figures are for a volume with 26 10TB disks (two physical servers w/ 26
disks each plus the arbiter-only in a VM).

> The idea with the ITX boards is not so bad. You can get 2 small systems and create your erasure coding.
Isn't EC a tad overkill with only 2 systems?
BTW I noticed that too small systems are not practical: you have a lot
of (nearly) fixed costs (motherboard, enclosure, power suppy) that only
manages 2-3 disks. Then, most depends on how much  you think you'll
expand your storage. Old theorem is that "the time required to fill a
disk is constant" :)

> Yet, I would prefer the 'replica 3 arbiter 1' approach as it doesn't take so much space and extending will require only 2 data disks .
And you won't have split-brain issues that are a mess to fix!

-- 
Diego Zuccato
DIFA - Dip. di Fisica e Astronomia
Servizi Informatici
Alma Mater Studiorum - Università di Bologna
V.le Berti-Pichat 6/2 - 40127 Bologna - Italy
tel.: +39 051 20 95786