[Gluster-users] Volume to store vm

[Dave Sherohman]

On Fri, Sep 06, 2019 at 05:11:57PM +0200, Cristian Del Carlo wrote:
> I have 4 nodes and I don't want to lose space by dedicating one to the
> function of arbiter.

You're chasing a false efficiency here.

To avoid split-brain, you *absolutely must* have more than two bricks
aware of each file.  If you just have two replicas and no arbiter, and
one node is no longer able to see the other one, then how can it know
whether it has the "working" brick or the "malfunctioning" brick?  In
order to resolve this, you need a third reference point so that the node
can either say "there are two of us, we're working" or "I've lost both
of the others; I must be the one that failed".

That third reference point can be either another replica or an arbiter.

Another replica contains a full copy of all your data, so it will be
very large.  For a 10T subvolume, each replica needs a full 10T of
storage dedicated to it.

An arbiter contains only the file metatdata (directory entries, etc.)
for each file, which is much, much smaller, especially for something
like VM images, where you have a small number of very large files,
because the metadata size is solely a function of how many files you
have, and not the size of the files.  The gluster volume where I store
my VM images has a total capacity of 23T (6T used); its arbiter bricks
have a total of 1.8G - approximately 0.3% of the space that would be
needed for an additional replica of the data.

As you can see, using an arbiter gives you (nearly) as much data
security as an additional replica, while consuming a tiny, tiny fraction
of the space that would be "lost" to an additional full replica.  If
you're trying to maximize usable capacity in your volume then an arbiter
configuration is absolutely the way to go.

-- 
Dave Sherohman