[Gluster-users] GlusterFS on FreeBSD

[Kyle Johnson]

BTW, here's the bug: https://bugzilla.redhat.com/show_bug.cgi?id=1356076

On 07/14/2016 08:28 PM, Angelo Rivera wrote:
> Hello,
>
> Thank you very much for your response this really is a big help on my 
> side, since I'm only starting to study glusterfs on freebsd. I'm 
> running 4 FreeBSD 10.2 server with 50GB space each.
>
> On 14/07/2016 1:33 PM, Kyle Johnson wrote:
>> Installed, yes.  Properly?  Kind of.
>>
>> I'm running gluster 3.7.6 with a two-node distributed volume between 
>> a centos and a freebsd peer.
>>
>> There is currently a bug where gluster on freebsd is not able to 
>> correctly determine file system information.  That is, on my freebsd 
>> peer with ~40T of storage, gluster is reporting 12.6PB of storage.  
>> As gluster believes this server has (far) more available storage, it 
>> sends most of the data on a volume to the brick(s) on that server.
>>
>> This causes the disk space on the freebsd server to fill up well 
>> before the other server does.
>>
>> $ sudo gluster volume status ftp detail
>> Status of volume: ftp
>> ------------------------------------------------------------------------------ 
>>
>> Brick                : Brick 192.168.110.1:/tank/bricks/ftp
>> TCP Port             : 49159
>> RDMA Port            : 0
>> Online               : Y
>> Pid                  : 69547
>> Disk Space Free      : 2.5PB
>> Total Disk Space     : 12.6PB
>> Inode Count          : 21097221251
>> Free Inodes          : 21087351972
>> ------------------------------------------------------------------------------ 
>>
>> Brick                : Brick 192.168.110.2:/ftp/bricks/ftp
>> TCP Port             : 49152
>> RDMA Port            : 0
>> Online               : Y
>> Pid                  : 1087
>> Disk Space Free      : 39.3TB
>> Total Disk Space     : 46.3TB
>> Inode Count          : 84298676694
>> Free Inodes          : 84297712404
>>
>> $ zfs list -r tank
>> NAME          USED  AVAIL  REFER  MOUNTPOINT
>> tank         40.5T  9.82T   153K  /tank
>> tank/bricks  40.5T  9.82T  40.5T  /tank/bricks
>>
>>
>> My temporary workaround is to disable weighted-rebalance:
>>
>> [kyle at colossus ~]$ sudo gluster volume set ftp 
>> cluster.weighted-rebalance off
>>
>> I haven't tested this on any other freebsd boxes, nor have I tested 
>> other types of volumes (e.g. replicated).
>>
>> In addition, I wasn't able to get the gluster fuse mount to work on 
>> freebsd, though I did not try very hard.
>>
>> As for the install:  $ sudo pkg install glusterfs
>>
>> Hope this helps, 
>