[Gluster-users] gluster, arbiter, thin-arbiter questions

[Strahil Nikolov]

As nobody chimed in, let me reply inline.

Best Regards,Strahil Nikolov

On Sunday, April 23, 2023, 2:35 AM, Peter P <peter.a.pfeiffer at gmail.com> wrote:

   Good afternoon,
 
 I am looking for additional information about glusterfs, arbiters and thin-arbiters. The current stable release is gluster 11, so I am considering that version for deployment. My planned setup is:  4 storage servers in distributed + replicated mode.
 
 Server1, server2 : replica 2, arbiter 1
 Server3, server4 : replica 2, arbiter 1
 
 Since having replica 2 is not recommended due to split-brains I will have an arbiter. 
 
 Generic questions:
    
   -  Is arbiter or thin-arbiter recommended in a production, large volume storage environment?
- Both were introduced a long time ago. Most users prefer full arbiter, so healing is far more optimal (only changed files will be healed).   
   - Is thin arbiter code stable and deployment ready?
- I know that it’s in use but full arbiter has been introduced earlier and has a wider adoption.
   
   -  Arbiter is file based and stores metadata for each file. In this scenario I would at least double the default inode count on the arbiter server. Thin-arbiter on the other hand is brick based but I have not found enough information if its inode usage is as inode hungry as the arbiter configuration. I am thinking, it should not be as it is brick based. So do I need to increase the inode count when using thin-arbiters? If yes, what is recommended?
- Full arbiter is sensitive to network latency and disk speed (a lot of small I/O for those inodes). Increase macpct (XFS) on arbiter bricks and prefer using a SSD/NVME. As full arbiter doesn’t store any data , you can set the maxpct to around 75%- Thin arbiter doesn’t have a brick and when you create it, you just specify the replica id file ( see https://docs.gluster.org/en/main/Administrator-Guide/Thin-Arbiter-Volumes/  )   
   -  I've read old bug reports reporting that thin arbiter was not ready to serve multiple trusted pools. Is this still the case?  I may configure multiple trusted pools in the future.
- I saw Kadalu uses their own thin arbiter and I never saw issues. I doubt I was the only one using it, so it should be fine.   
   - I have many linux boxes running different linux distributions and releases. Ideally the assortment of boxes would mount the same gluster pool/volume. I looked for information about older versions of gluster clients running on a range of older distributions mounting the most recent gluster 11 pool/volume? Does that work?  Can gluster client (version 10, 9, 8, 7, etc.) mount gluster 11 volume and run without significant issues?  I understand that older versions of client will not have the most recent features. Most recent features aside, is such configuration supported/stable?
- For that purpose gluster has 2 settings:cluster.max-op-version -> the max compatibility version you can set your cluster based of the oldest client’s versioncluster.op-version -> the cluster’s compatibility versionAs long you keep the cluster.op-version compatible with your client - it should work. 
 Thin-arbiter approach:  If I go with the thin-arbiter configuration I will use a 5th server as this server can be outside of the trusted pool and can be shared among multiple trusted pools
 
 Server1, server2: replica 2, thin-arbiter server5
 Server3, server4: replica 2, thin-arbiter server5
 
 
 Old arbiter approach:  If I go with the older arbiter configuration, I am considering using 2 of the storage servers to act as both replica and an arbiter. Is that configuration possible/supported and reasonable? 
 
 Server1, server2: replica 2, arbiter server3  
 Server3, server4: replica 2, arbiter server1
 
- Yes, as long as you have a dedicated brick (in this example server3 should have a data brick and arbiter brick)
 In this configuration, I am considering using server3 to be arbiter for server{1,2} replica 2,  and using server1 to be arbiter for server{3,4} replica 2.  
 
 Questions:
    
   - Is this a reasonable/recommended configuration?
- It’s used quite often   
   -  Should the arbiter metadata folder be inside or outside of the volume? 
   
   - In detail. Say server{1,2} replica has 1 brick each /gluster/brick1 with /gfs1vol1 as the volume
   -  Should the arbiter metadata folder location be:   /gluster/arbiter/gfs1vol1   (outside of the  volume path)  or   /gfs1vol1/arbiter1/  (inside the volume path)
  - Always keep bricks as separate mount points. For example:/dev/vg/lv mounted on /bricks/databricks with directory vol1/brick1/dev/vg/lv2 mounted on /bricks/arbiterbricks with directory vol1/arbiterbrick1
The idea is that if the device is not mounted, the brick directory will be missing and the mess will be far less.
 Thank you for your thoughts,
 Peter
  

|  | Virus-free.www.avg.com |

 ________



Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://meet.google.com/cpu-eiue-hvk
Gluster-users mailing list
Gluster-users at gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20230514/dd5142d2/attachment.html>