[Gluster-users] Bug with hardlink limitation in 3.12.13 ?

[Reiner Keller]

Hello,

I got yesterday unexpected error "No space left on device" on my new
gluster volume caused by too many hardlinks.
This happened while I done "rsync --aAHXxv ..." replication from old
gluster to new gluster servers - each running latest version 3.12.13
(for changing volume schema from 2x2 to 3x1 with quorum and a fresh
Debian Stretch setup instead Jessie).

When I deduplicated it around half a year ago with "rdfind" hardlinking
was working fine (I think that was glusterfs around version 3.12.8 -
3.12.10 ?)

My search for documentation found only the parameter
"storage.max-hardlinks" with default of 100 for version 4.0.
I checked it in my gluster 3.12.13 but here the parameter is not yet
implemented.

I tested/proofed it by running my small test on underlaying ext4
filesystem brick directly and on gluster volume using same ext4
filesystem of the brick:

Testline for it:
            mkdir test; cd test; echo "hello" > test; for I in $(seq 1
100); do ln test test-$I ; done

* on ext4 fs (old brick: xfs) I could do 100 hardlinks without problems
(from documentation I found ext has 65.000 hardlinks compiled in )
* on actual GlusterFS (same on my old and new gluster volumes) I could
do only up to 45 hardlinks now

But from deduplication around 6 months ago I could find e.g. a file with
240 hardlinks setup and there is no problem using these referenced files
(caused by multiple languages / multiple uploads per language ,
production/staging system cloned... ).

My actual workaround has to be using duplicated content but it would be
great if this could be fixed in next versions ;)

(Saltstack didn't support yet successful setup of glusterfs 4.0
peers/volumes; something in output of "gluster --xml --mode=script" call
must be weird but I haven't seen any differences so far)

Bests


Reiner