[Gluster-users] Experimenting with thin-arbiter
Amar Tumballi
amar at kadalu.io
Thu Feb 17 11:26:58 UTC 2022
One of the thing with thin-arbiter is, the process is not expected to be
managed by the glusterd itself, ie, it is designed to be outside the
storage pool, in a cloud or a high latency backup/more available setup.
We (kadalu project) make use of it in our 'Replica 2' volume types. For
testing see if using tie-breaker.kadalu.io:/mnt as the brick for
thin-arbiter works for you.
-Amar
On Wed, Feb 16, 2022 at 6:22 PM Diego Zuccato <diego.zuccato at unibo.it>
wrote:
> Not there. It's not one of the defined services :(
> Maybe Debian does not support it?
>
> Il 16/02/2022 13:26, Strahil Nikolov ha scritto:
> > My bad, it should be /gluster-ta-volume.service/
> >
> > On Wed, Feb 16, 2022 at 7:45, Diego Zuccato
> > <diego.zuccato at unibo.it> wrote:
> > No such process is defined. Just the standard glusterd.service and
> > glustereventsd.service . Using Debian stable.
> >
> > Il 15/02/2022 15:41, Strahil Nikolov ha scritto:
> > > Any errors in gluster-ta.service on the arbiter node ?
> > >
> > > Best Regards,
> > > Strahil Nikolov
> > >
> > > On Tue, Feb 15, 2022 at 14:28, Diego Zuccato
> > > <diego.zuccato at unibo.it <mailto:diego.zuccato at unibo.it>>
> wrote:
> > > Hello all.
> > >
> > > I'm experimenting with thin-arbiter and getting disappointing
> > results.
> > >
> > > I have 3 hosts in the trusted pool:
> > > root at nas1 <mailto:root at nas1> <mailto:root at nas1
> > <mailto:root at nas1>>:~# gluster --version
> > > glusterfs 9.2
> > > [...]
> > > root at nas1 <mailto:root at nas1> <mailto:root at nas1
> > <mailto:root at nas1>>:~# gluster pool list
> > > UUID Hostname State
> > > d4791fed-3e6d-4f8f-bdb6-4e0043610ead nas3
> Connected
> > > bff398f0-9d1d-4bd0-8a47-0bf481d1d593 nas2
> Connected
> > > 4607034c-919d-4675-b5fc-14e1cad90214 localhost
> Connected
> > >
> > > When I try to create a new volume, the first initialization
> > succeeds:
> > > root at nas1 <mailto:root at nas1> <mailto:root at nas1
> > <mailto:root at nas1>>:~# gluster v create Bck replica 2
> > > thin-arbiter 1
> > > nas{1,3}:/bricks/00/Bck nas2:/bricks/arbiter/Bck
> > > volume create: Bck: success: please start the volume to access
> > data
> > >
> > > But adding a second brick segfaults the daemon:
> > > root at nas1 <mailto:root at nas1> <mailto:root at nas1
> > <mailto:root at nas1>>:~# gluster v add-brick Bck
> > > nas{1,3}:/bricks/01/Bck
> > > Connection failed. Please check if gluster daemon is
> operational.
> > >
> > > After erroring out, systemctl status glusterd reports daemon in
> > > "restarting" state and it eventually restarts. But the new
> > brick is not
> > > added to the volume, even if trying to re-add it yelds a
> "brick is
> > > already part of a volume" error. Seems glusterd crashes
> > between marking
> > > brick dir as used and recording its data in the config.
> > >
> > > If I try to add all the bricks during the creation, glusterd
> > does not
> > > die but the volume doesn't get created:
> > > root at nas1 <mailto:root at nas1> <mailto:root at nas1
> > <mailto:root at nas1>>:~# rm -rf /bricks/{00..07}/Bck && mkdir
> > > /bricks/{00..07}/Bck
> > > root at nas1 <mailto:root at nas1> <mailto:root at nas1
> > <mailto:root at nas1>>:~# gluster v create Bck replica 2
> > > thin-arbiter 1
> > > nas{1,3}:/bricks/00/Bck nas{1,3}:/bricks/01/Bck
> > nas{1,3}:/bricks/02/Bck
> > > nas{1,3}:/bricks/03/Bck nas{1,3}:/bricks/04/Bck
> > nas{1,3}:/bricks/05/Bck
> > > nas{1,3}:/bricks/06/Bck nas{1,3}:/bricks/07/Bck
> > nas2:/bricks/arbiter/Bck
> > > volume create: Bck: failed: Commit failed on localhost. Please
> > check
> > > the
> > > log file for more details.
> > >
> > > Couldn't find anything useful in the logs :(
> > >
> > > If I create a "replica 3 arbiter 1" over the same brick
> > directories
> > > (just adding some directories to keep arbiters separated), it
> > succeeds:
> > > root at nas1 <mailto:root at nas1> <mailto:root at nas1
> > <mailto:root at nas1>>:~# gluster v create Bck replica 3
> > > arbiter 1
> > > nas{1,3}:/bricks/00/Bck nas2:/bricks/arbiter/Bck/00
> > > volume create: Bck: success: please start the volume to access
> > data
> > > root at nas1 <mailto:root at nas1> <mailto:root at nas1
> > <mailto:root at nas1>>:~# for T in {01..07}; do gluster v
> > > add-brick Bck
> > > nas{1,3}:/bricks/$T/Bck nas2:/bricks/arbiter/Bck/$T ; done
> > > volume add-brick: success
> > > volume add-brick: success
> > > volume add-brick: success
> > > volume add-brick: success
> > > volume add-brick: success
> > > volume add-brick: success
> > > volume add-brick: success
> > > root at nas1 <mailto:root at nas1> <mailto:root at nas1
> > <mailto:root at nas1>>:~# gluster v start Bck
> > > volume start: Bck: success
> > > root at nas1 <mailto:root at nas1> <mailto:root at nas1
> > <mailto:root at nas1>>:~# gluster v info Bck
> > >
> > > Volume Name: Bck
> > > Type: Distributed-Replicate
> > > Volume ID: 4786e747-8203-42bf-abe8-107a50b238ee
> > > Status: Started
> > > Snapshot Count: 0
> > > Number of Bricks: 8 x (2 + 1) = 24
> > > Transport-type: tcp
> > > Bricks:
> > > Brick1: nas1:/bricks/00/Bck
> > > Brick2: nas3:/bricks/00/Bck
> > > Brick3: nas2:/bricks/arbiter/Bck/00 (arbiter)
> > > Brick4: nas1:/bricks/01/Bck
> > > Brick5: nas3:/bricks/01/Bck
> > > Brick6: nas2:/bricks/arbiter/Bck/01 (arbiter)
> > > Brick7: nas1:/bricks/02/Bck
> > > Brick8: nas3:/bricks/02/Bck
> > > Brick9: nas2:/bricks/arbiter/Bck/02 (arbiter)
> > > Brick10: nas1:/bricks/03/Bck
> > > Brick11: nas3:/bricks/03/Bck
> > > Brick12: nas2:/bricks/arbiter/Bck/03 (arbiter)
> > > Brick13: nas1:/bricks/04/Bck
> > > Brick14: nas3:/bricks/04/Bck
> > > Brick15: nas2:/bricks/arbiter/Bck/04 (arbiter)
> > > Brick16: nas1:/bricks/05/Bck
> > > Brick17: nas3:/bricks/05/Bck
> > > Brick18: nas2:/bricks/arbiter/Bck/05 (arbiter)
> > > Brick19: nas1:/bricks/06/Bck
> > > Brick20: nas3:/bricks/06/Bck
> > > Brick21: nas2:/bricks/arbiter/Bck/06 (arbiter)
> > > Brick22: nas1:/bricks/07/Bck
> > > Brick23: nas3:/bricks/07/Bck
> > > Brick24: nas2:/bricks/arbiter/Bck/07 (arbiter)
> > > Options Reconfigured:
> > > cluster.granular-entry-heal: on
> > > storage.fips-mode-rchecksum: on
> > > transport.address-family: inet
> > > nfs.disable: on
> > > performance.client-io-threads: off
> > >
> > > Does thin arbiter support just one replica of bricks?
> > >
> > > --
> > > Diego Zuccato
> > > DIFA - Dip. di Fisica e Astronomia
> > > Servizi Informatici
> > > Alma Mater Studiorum - Università di Bologna
> > > V.le Berti-Pichat 6/2 - 40127 Bologna - Italy
> > > tel.: +39 051 20 95786
> > > ________
> > >
> > >
> > >
> > > Community Meeting Calendar:
> > >
> > > Schedule -
> > > Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
> > > Bridge: https://meet.google.com/cpu-eiue-hvk
> > <https://meet.google.com/cpu-eiue-hvk>
> > > <https://meet.google.com/cpu-eiue-hvk
> > <https://meet.google.com/cpu-eiue-hvk>>
> > > Gluster-users mailing list
> > > Gluster-users at gluster.org <mailto:Gluster-users at gluster.org>
> > <mailto:Gluster-users at gluster.org <mailto:Gluster-users at gluster.org
> >>
> > > https://lists.gluster.org/mailman/listinfo/gluster-users
> > <https://lists.gluster.org/mailman/listinfo/gluster-users>
> > > <https://lists.gluster.org/mailman/listinfo/gluster-users
> > <https://lists.gluster.org/mailman/listinfo/gluster-users>>
> >
> > >
> >
> > --
> > Diego Zuccato
> > DIFA - Dip. di Fisica e Astronomia
> > Servizi Informatici
> > Alma Mater Studiorum - Università di Bologna
> > V.le Berti-Pichat 6/2 - 40127 Bologna - Italy
> > tel.: +39 051 20 95786
> >
>
> --
> Diego Zuccato
> DIFA - Dip. di Fisica e Astronomia
> Servizi Informatici
> Alma Mater Studiorum - Università di Bologna
> V.le Berti-Pichat 6/2 - 40127 Bologna - Italy
> tel.: +39 051 20 95786
> ________
>
>
>
> Community Meeting Calendar:
>
> Schedule -
> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
> Bridge: https://meet.google.com/cpu-eiue-hvk
> Gluster-users mailing list
> Gluster-users at gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-users
>
--
--
https://kadalu.io
Container Storage made easy!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20220217/bec73de0/attachment.html>
More information about the Gluster-users
mailing list