[Gluster-users] design of gluster cluster

[Thing]

Hi,

I dont have any more hosts available.

I am a bit lost here, why a replica 3 and arbiter 1?  ie not replica2
arbiter1?   also no distributed part?  is the distributed flag
automatically assumed?    with a replica3 then there is a quorum (2 of 3)
so no arbiter is needed?   I have this running already like this so I am
assuming its robust?

I am still struggling to undersatnd the syntax, I wish the docs / examples
were better.

So on each gluster node I have an-unused 120gb data1 partition which is
left over from the OS install so the arbiter volume could go here?

in which case?

gluster volume create my-volume replica 2 arbiter 1 host1:/path/to/brick
host2:/path/to/brick (arb-)host3:/path/to/brick2   host4:/path/to/brick
host5:/path/to/brick (arb-)host6:/path/to/brick2 host3:/path/to/brick
host6:/path/to/brick (arb-)host1:/path/to/brick2

is this a sane command?

Otherwise maybe I am beginning to think I am better off doing 3 x 2TB
separate volumes.  rather interesting trying to understand this stuff...!




On 12 June 2018 at 23:10, Dave Sherohman <dave at sherohman.org> wrote:

> On Tue, Jun 12, 2018 at 03:04:14PM +1200, Thing wrote:
> > What I would like to do I think is a,
> >
> > *Distributed-Replicated volume*
> >
> > a) have 1 and 2 as raid1
> > b) have 4 and 5 as raid1
> > c) have 3 and 6 as a raid1
> > d) join this as concatenation 2+2+2tb
>
> You probably don't actually want to do that because quorum is handled
> separately for each subvolume (bricks 1/2, 4/5, or 3/6), not a single
> quorum for the volume as a whole.  (Consider if bricks 1 and 2 both went
> down.  You'd still have 4 of 6 bricks running, so whole-volume quorum
> would still be met, but the volume can't continue to run normally since
> the first subvolume is completely missing.)
>
> In the specific case of replica 2, gluster treats the first brick in
> each subvolume as "slightly more than one", so you'd be able to continue
> normally if brick 2, 5, or 6 went down, but, if brick 1, 4, or 3 went
> down, all files on that subvolume would become read-only.
>
> > I tried to do this and failed as it kept asking for an arbiter, which the
> > docs simply dont mention how to do.
>
> https://gluster.readthedocs.io/en/latest/Administrator%
> 20Guide/arbiter-volumes-and-quorum/
>
> > So say we have,
> >
> > a) have 1 and 2 as raid1 with 3 as the arbiter?
> > b) have 4 and 5 as raid 1 with 6 as the arbiter
> > c) 3 and 6 as a raid 1 with 5 as the arbiter
> > d) join this as concatenation 2+2+2tb
>
> I would recommend finding one or more other servers with small amounts
> of unused space and allocating the arbiter bricks there, or carving a
> gig or two out of your current bricks for that purpose.  Arbiters only
> need about 4k of disk space per file in the subvolume, regardless of the
> actual file size (the arbiter only stores metadata), so TB-sized
> arbiters would be a huge waste of space, especially if you're only
> putting a few very large files (such as VM disk images) on the volume.
>
> As a real-world data point, I'm using basically the setup you're aiming
> for - six data bricks plus three arbiters, used to store VM disk images.
> My data bricks are 11T each, while my arbiters are 98G.  Disk usage for
> the volume is currently at 19%, but all arbiters are under 1% usage (the
> largest has 370M used).  Assuming my usage patterns don't change, I
> could completely fill my 11T subvolumes and only need about 1.5G in the
> corresponding arbiters.
>
> > if so what is the command used to build this?
>
> # gluster volume create my-volume replica 3 arbiter 1 host1:/path/to/brick
> host2:/path/to/brick arb-host1:/path/to/brick host4:/path/to/brick
> host5:/path/to/brick arb-host2:/path/to/brick host3:/path/to/brick
> host6:/path/to/brick arb-host3:/path/to/brick
>
> --
> Dave Sherohman
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180613/a2222ff6/attachment.html>