[Gluster-users] How to use system.affinity/distributed.migrate-data on distributed/replicated volume?
nl at fischer-ka.de
Sun Oct 28 22:02:22 UTC 2018
has noone an idea on system.affinity/distributed.migrate-data ?
Or how to correctly enable nufa?
BTW: the used gluster version is 4.1.5
Thank you for your help on this!
Am 24.10.18 um 12:54 schrieb Ingo Fischer:
> I have setup a glusterfs volume gv0 as distributed/replicated:
> root at pm1:~# gluster volume info gv0
> Volume Name: gv0
> Type: Distributed-Replicate
> Volume ID: 64651501-6df2-4106-b330-fdb3e1fbcdf4
> Status: Started
> Snapshot Count: 0
> Number of Bricks: 3 x 2 = 6
> Transport-type: tcp
> Brick1: 192.168.178.50:/gluster/brick1/gv0
> Brick2: 192.168.178.76:/gluster/brick1/gv0
> Brick3: 192.168.178.50:/gluster/brick2/gv0
> Brick4: 192.168.178.81:/gluster/brick1/gv0
> Brick5: 192.168.178.50:/gluster/brick3/gv0
> Brick6: 192.168.178.82:/gluster/brick1/gv0
> Options Reconfigured:
> performance.client-io-threads: off
> nfs.disable: on
> transport.address-family: inet
> root at pm1:~# gluster volume status
> Status of volume: gv0
> Gluster process TCP Port RDMA Port Online Pid
> Brick 192.168.178.50:/gluster/brick1/gv0 49152 0 Y
> Brick 192.168.178.76:/gluster/brick1/gv0 49152 0 Y
> Brick 192.168.178.50:/gluster/brick2/gv0 49153 0 Y
> Brick 192.168.178.81:/gluster/brick1/gv0 49152 0 Y
> Brick 192.168.178.50:/gluster/brick3/gv0 49154 0 Y
> Brick 192.168.178.82:/gluster/brick1/gv0 49152 0 Y
> Self-heal Daemon on localhost N/A N/A Y
> Self-heal Daemon on 192.168.178.81 N/A N/A Y
> Self-heal Daemon on 192.168.178.82 N/A N/A Y
> Self-heal Daemon on pm2.fritz.box N/A N/A Y
> I use the fs to store VM files, so not many, but big files.
> The distribution now put 4 big files on one brick set and only one file
> on an other. This means that the one brick set it "overcommited" now as
> soon as all VMs using max space. SO I would like to manually
> redistribute the files a bit better.
> After log googling I found that the following should work:
> setfattr -n 'system.affinity' -v $location $filepath
> setfattr -n 'distribute.migrate-data' -v 'force' $filepath
> But I have problems with it because it gives errors or doing nothing at all.
> The mounting looks like:
> 192.168.178.50:gv0 on /mnt/pve/glusterfs type fuse.glusterfs
> Here is what I tried for the first xattr:
> root at pm1:~# setfattr -n 'system.affinity' -v 'gv0-client-5'
> setfattr: /mnt/pve/glusterfs/201/imagesvm201.qcow2: Operation not supported
> So I found on google to use trusted.affinity instead and yes this works.
> I'm only not sure if the location "gv0-client-5" is correct to move the
> file to "Brick 5" from "gluster volume info gv0" ... or how this
> location is build?
> Commit Message from http://review.gluster.org/#/c/glusterfs/+/5233/ says
>> The value is the internal client or AFR brick name where you want the
> file to be.
> So what do I need to set there? maybe I do need the "afr" because
> replicated? But where to get that name from?
> I also tried to enter other client or replicate names like
> "gv0-replicate-0" or such which seems to be more fitting for a
> replicated volume, but result the same.
> For the second command I get:
> root at pm1:~# setfattr -n 'distribute.migrate-data' -v 'force'
> setfattr: /mnt/pve/glusterfs/images/201/vm-201-disk-0.qcow2: Operation
> not supported
> root at pm1:~# setfattr -n 'trusted.distribute.migrate-data' -v 'force'
> setfattr: /mnt/pve/glusterfs/images/201/vm-201-disk-0.qcow2: File exists
> I also experimented with other "names" then "gv0-client-5" above but
> always the same.
> I saw that instead of the second command I could start a rebalance with
> force, but this also did nothing. Ended after max1 second and moved nothing.
> Can someone please advice how to do it right?
> An other idea was to enable nufa and kind of "re-copy" the files on the
> glusterfs, but here it seems that the documentation is wrong.
> gluster volume set gv0 cluster.nufa enable on
> gluster volume set gv0 cluster.nufa 1
> Thank you very much!
More information about the Gluster-users