<div dir="ltr">One of the thing with thin-arbiter is, the process is not expected to be managed by the glusterd itself, ie, it is designed to be outside the storage pool, in a cloud or a high latency backup/more available setup.<div><br></div><div>We (kadalu project) make use of it in our 'Replica 2' volume types. For testing see if using tie-breaker.kadalu.io:/mnt as the brick for thin-arbiter works for you.</div><div><br></div><div>-Amar</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Feb 16, 2022 at 6:22 PM Diego Zuccato <<a href="mailto:diego.zuccato@unibo.it">diego.zuccato@unibo.it</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Not there. It's not one of the defined services :(<br>
Maybe Debian does not support it?<br>
<br>
Il 16/02/2022 13:26, Strahil Nikolov ha scritto:<br>
> My bad, it should be /gluster-ta-volume.service/<br>
> <br>
> On Wed, Feb 16, 2022 at 7:45, Diego Zuccato<br>
> <<a href="mailto:diego.zuccato@unibo.it" target="_blank">diego.zuccato@unibo.it</a>> wrote:<br>
> No such process is defined. Just the standard glusterd.service and<br>
> glustereventsd.service . Using Debian stable.<br>
> <br>
> Il 15/02/2022 15:41, Strahil Nikolov ha scritto:<br>
> > Any errors in gluster-ta.service on the arbiter node ?<br>
> ><br>
> > Best Regards,<br>
> > Strahil Nikolov<br>
> ><br>
> > On Tue, Feb 15, 2022 at 14:28, Diego Zuccato<br>
> > <<a href="mailto:diego.zuccato@unibo.it" target="_blank">diego.zuccato@unibo.it</a> <mailto:<a href="mailto:diego.zuccato@unibo.it" target="_blank">diego.zuccato@unibo.it</a>>> wrote:<br>
> > Hello all.<br>
> ><br>
> > I'm experimenting with thin-arbiter and getting disappointing<br>
> results.<br>
> ><br>
> > I have 3 hosts in the trusted pool:<br>
> > root@nas1 <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a>> <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a><br>
> <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a>>>:~# gluster --version<br>
> > glusterfs 9.2<br>
> > [...]<br>
> > root@nas1 <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a>> <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a><br>
> <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a>>>:~# gluster pool list<br>
> > UUID Hostname State<br>
> > d4791fed-3e6d-4f8f-bdb6-4e0043610ead nas3 Connected<br>
> > bff398f0-9d1d-4bd0-8a47-0bf481d1d593 nas2 Connected<br>
> > 4607034c-919d-4675-b5fc-14e1cad90214 localhost Connected<br>
> ><br>
> > When I try to create a new volume, the first initialization<br>
> succeeds:<br>
> > root@nas1 <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a>> <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a><br>
> <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a>>>:~# gluster v create Bck replica 2<br>
> > thin-arbiter 1<br>
> > nas{1,3}:/bricks/00/Bck nas2:/bricks/arbiter/Bck<br>
> > volume create: Bck: success: please start the volume to access<br>
> data<br>
> ><br>
> > But adding a second brick segfaults the daemon:<br>
> > root@nas1 <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a>> <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a><br>
> <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a>>>:~# gluster v add-brick Bck<br>
> > nas{1,3}:/bricks/01/Bck<br>
> > Connection failed. Please check if gluster daemon is operational.<br>
> ><br>
> > After erroring out, systemctl status glusterd reports daemon in<br>
> > "restarting" state and it eventually restarts. But the new<br>
> brick is not<br>
> > added to the volume, even if trying to re-add it yelds a "brick is<br>
> > already part of a volume" error. Seems glusterd crashes<br>
> between marking<br>
> > brick dir as used and recording its data in the config.<br>
> ><br>
> > If I try to add all the bricks during the creation, glusterd<br>
> does not<br>
> > die but the volume doesn't get created:<br>
> > root@nas1 <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a>> <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a><br>
> <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a>>>:~# rm -rf /bricks/{00..07}/Bck && mkdir<br>
> > /bricks/{00..07}/Bck<br>
> > root@nas1 <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a>> <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a><br>
> <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a>>>:~# gluster v create Bck replica 2<br>
> > thin-arbiter 1<br>
> > nas{1,3}:/bricks/00/Bck nas{1,3}:/bricks/01/Bck<br>
> nas{1,3}:/bricks/02/Bck<br>
> > nas{1,3}:/bricks/03/Bck nas{1,3}:/bricks/04/Bck<br>
> nas{1,3}:/bricks/05/Bck<br>
> > nas{1,3}:/bricks/06/Bck nas{1,3}:/bricks/07/Bck<br>
> nas2:/bricks/arbiter/Bck<br>
> > volume create: Bck: failed: Commit failed on localhost. Please<br>
> check<br>
> > the<br>
> > log file for more details.<br>
> ><br>
> > Couldn't find anything useful in the logs :(<br>
> ><br>
> > If I create a "replica 3 arbiter 1" over the same brick<br>
> directories<br>
> > (just adding some directories to keep arbiters separated), it<br>
> succeeds:<br>
> > root@nas1 <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a>> <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a><br>
> <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a>>>:~# gluster v create Bck replica 3<br>
> > arbiter 1<br>
> > nas{1,3}:/bricks/00/Bck nas2:/bricks/arbiter/Bck/00<br>
> > volume create: Bck: success: please start the volume to access<br>
> data<br>
> > root@nas1 <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a>> <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a><br>
> <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a>>>:~# for T in {01..07}; do gluster v<br>
> > add-brick Bck<br>
> > nas{1,3}:/bricks/$T/Bck nas2:/bricks/arbiter/Bck/$T ; done<br>
> > volume add-brick: success<br>
> > volume add-brick: success<br>
> > volume add-brick: success<br>
> > volume add-brick: success<br>
> > volume add-brick: success<br>
> > volume add-brick: success<br>
> > volume add-brick: success<br>
> > root@nas1 <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a>> <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a><br>
> <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a>>>:~# gluster v start Bck<br>
> > volume start: Bck: success<br>
> > root@nas1 <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a>> <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a><br>
> <mailto:<a href="mailto:root@nas1" target="_blank">root@nas1</a>>>:~# gluster v info Bck<br>
> ><br>
> > Volume Name: Bck<br>
> > Type: Distributed-Replicate<br>
> > Volume ID: 4786e747-8203-42bf-abe8-107a50b238ee<br>
> > Status: Started<br>
> > Snapshot Count: 0<br>
> > Number of Bricks: 8 x (2 + 1) = 24<br>
> > Transport-type: tcp<br>
> > Bricks:<br>
> > Brick1: nas1:/bricks/00/Bck<br>
> > Brick2: nas3:/bricks/00/Bck<br>
> > Brick3: nas2:/bricks/arbiter/Bck/00 (arbiter)<br>
> > Brick4: nas1:/bricks/01/Bck<br>
> > Brick5: nas3:/bricks/01/Bck<br>
> > Brick6: nas2:/bricks/arbiter/Bck/01 (arbiter)<br>
> > Brick7: nas1:/bricks/02/Bck<br>
> > Brick8: nas3:/bricks/02/Bck<br>
> > Brick9: nas2:/bricks/arbiter/Bck/02 (arbiter)<br>
> > Brick10: nas1:/bricks/03/Bck<br>
> > Brick11: nas3:/bricks/03/Bck<br>
> > Brick12: nas2:/bricks/arbiter/Bck/03 (arbiter)<br>
> > Brick13: nas1:/bricks/04/Bck<br>
> > Brick14: nas3:/bricks/04/Bck<br>
> > Brick15: nas2:/bricks/arbiter/Bck/04 (arbiter)<br>
> > Brick16: nas1:/bricks/05/Bck<br>
> > Brick17: nas3:/bricks/05/Bck<br>
> > Brick18: nas2:/bricks/arbiter/Bck/05 (arbiter)<br>
> > Brick19: nas1:/bricks/06/Bck<br>
> > Brick20: nas3:/bricks/06/Bck<br>
> > Brick21: nas2:/bricks/arbiter/Bck/06 (arbiter)<br>
> > Brick22: nas1:/bricks/07/Bck<br>
> > Brick23: nas3:/bricks/07/Bck<br>
> > Brick24: nas2:/bricks/arbiter/Bck/07 (arbiter)<br>
> > Options Reconfigured:<br>
> > cluster.granular-entry-heal: on<br>
> > storage.fips-mode-rchecksum: on<br>
> > transport.address-family: inet<br>
> > nfs.disable: on<br>
> > performance.client-io-threads: off<br>
> ><br>
> > Does thin arbiter support just one replica of bricks?<br>
> ><br>
> > --<br>
> > Diego Zuccato<br>
> > DIFA - Dip. di Fisica e Astronomia<br>
> > Servizi Informatici<br>
> > Alma Mater Studiorum - Università di Bologna<br>
> > V.le Berti-Pichat 6/2 - 40127 Bologna - Italy<br>
> > tel.: +39 051 20 95786<br>
> > ________<br>
> ><br>
> ><br>
> ><br>
> > Community Meeting Calendar:<br>
> ><br>
> > Schedule -<br>
> > Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC<br>
> > Bridge: <a href="https://meet.google.com/cpu-eiue-hvk" rel="noreferrer" target="_blank">https://meet.google.com/cpu-eiue-hvk</a><br>
> <<a href="https://meet.google.com/cpu-eiue-hvk" rel="noreferrer" target="_blank">https://meet.google.com/cpu-eiue-hvk</a>><br>
> > <<a href="https://meet.google.com/cpu-eiue-hvk" rel="noreferrer" target="_blank">https://meet.google.com/cpu-eiue-hvk</a><br>
> <<a href="https://meet.google.com/cpu-eiue-hvk" rel="noreferrer" target="_blank">https://meet.google.com/cpu-eiue-hvk</a>>><br>
> > Gluster-users mailing list<br>
> > <a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a> <mailto:<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a>><br>
> <mailto:<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a> <mailto:<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a>>><br>
> > <a href="https://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-users</a><br>
> <<a href="https://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-users</a>><br>
> > <<a href="https://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-users</a><br>
> <<a href="https://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-users</a>>><br>
> <br>
> ><br>
> <br>
> -- <br>
> Diego Zuccato<br>
> DIFA - Dip. di Fisica e Astronomia<br>
> Servizi Informatici<br>
> Alma Mater Studiorum - Università di Bologna<br>
> V.le Berti-Pichat 6/2 - 40127 Bologna - Italy<br>
> tel.: +39 051 20 95786<br>
> <br>
<br>
-- <br>
Diego Zuccato<br>
DIFA - Dip. di Fisica e Astronomia<br>
Servizi Informatici<br>
Alma Mater Studiorum - Università di Bologna<br>
V.le Berti-Pichat 6/2 - 40127 Bologna - Italy<br>
tel.: +39 051 20 95786<br>
________<br>
<br>
<br>
<br>
Community Meeting Calendar:<br>
<br>
Schedule -<br>
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC<br>
Bridge: <a href="https://meet.google.com/cpu-eiue-hvk" rel="noreferrer" target="_blank">https://meet.google.com/cpu-eiue-hvk</a><br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="https://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-users</a><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr">--<div><a href="https://kadalu.io" target="_blank">https://kadalu.io</a></div><div>Container Storage made easy!</div><div><br></div></div></div>