[Gluster-users] "No space left on device" during rebalance with failed brick on Gluster 4.1.7

[Alan Orth]

Dear list,

We are using a Distributed-Replicate volume with replica 2 on Gluster 4.1.7
on CentOS 7. One of our nodes died recently and we will add new nodes and
bricks to replace it soon. In preparation for the maintenance I wanted to
rebalance the volume to make the disk thrashing less intense when we
add/remove bricks, but after eight hours of scanning I see millions of
"failures" in the rebalance status. The volume rebalance log shows many
errors like:

[2019-05-07 06:06:02.310843] E [MSGID: 109023]
[dht-rebalance.c:2907:gf_defrag_migrate_single_file] 0-data-dht:
migrate-data failed for
/ilri/miseq/MiSeq2/MiSeq2Output2018/180912_M03021_0002_000000000-BVM95/Thumbnail_Images/L001/C174.1/s_1_2103_c.jpg
[No space left on device]

The bricks on the healthy nodes all have 1.5TB of free space so I'm not
sure what this error means. Could it be because one of the replicas is
unavailable? I saw a similar bug report¹ about that. I've started a simple
fix-layout without data migration and it is working fine.

Thank you,

¹ https://access.redhat.com/solutions/456333
--
Alan Orth
alan.orth at gmail.com
https://picturingjordan.com
https://englishbulgaria.net
https://mjanja.ch
"In heaven all the interesting people are missing." ―Friedrich Nietzsche
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20190507/636ec3e6/attachment.html>