[Bugs] [Bug 1741899] the volume of occupied space in the bricks of gluster volume (3 nodes replica) differs on nodes and the healing does not fix it

[bugzilla at redhat.com]

https://bugzilla.redhat.com/show_bug.cgi?id=1741899



--- Comment #12 from Sergey Pleshkov <s.pleshkov at hostco.ru> ---
Hello

Once again, I executed the command to replace the brick for lsy-gl-03 (as a
simple way to repair a identity of files on lsy-gl-03):

(gluster volume replace-brick TST lsy-gl-03:/diskForData/tst
lsy-gl-03:/diskForTestData/tst-fix commit force)

it must synchronize all files from live nodes (lsy-gl-01, lsy-gl-03), as i
know.

But as a result, I again got a discrepancy between the actual sizes on the disk
(df -h)

[root at LSY-GL-02 host]# df -h
Filesystem                         Size  Used Avail Use% Mounted on
LSY-GL-02:/TST                     500G  115G  385G  23% /mnt/tst
/dev/sdc1                          500G  110G  390G  22% /diskForTestData

[root at LSY-GL-03 host]# df -h
Filesystem                         Size  Used Avail Use% Mounted on
/dev/sdc1                          500G  107G  394G  22% /diskForTestData
LSY-GL-03:/TST                     500G  115G  385G  23% /mnt/tst

I finded diff files by exec a diff command (/diskForTestData/tst is symlink to
/diskForTestData/tst-fix):
[root at LSY-GL-02 ~]# diff <(ls -Ra  /diskForTestData/tst/lsy-tst/) <(ssh
host at lsy-gl-03 sudo ls -Ra  /diskForTestData/tst/lsy-tst)
1c1
< /diskForTestData/tst/lsy-tst/:
---
> /diskForTestData/tst/lsy-tst:
357638a357639,357643
> 00b0d046-1e1c-4088-bb67-527513bd432d.1
> 00b0d046-1e1c-4088-bb67-527513bd432d.2
> 00b0d046-1e1c-4088-bb67-527513bd432d.3
> 00b0d046-1e1c-4088-bb67-527513bd432d.4
> 00b0d046-1e1c-4088-bb67-527513bd432d.5
357644a357650,357652
> 0339fa08-fb52-4f9f-bbc1-998a88bad3a9.1
> 0339fa08-fb52-4f9f-bbc1-998a88bad3a9.2
> 0339fa08-fb52-4f9f-bbc1-998a88bad3a9.3
357652a357661,357663
.....

Also finded a reason, what arequal-checksum command shows a lot more regular
files on lsy-gl-03 - it is folder /diskForTestData/tst/lsy-tst/.shard and files
in it.

But on lsy-gl-03 it have size like 70gb, but on lsy-lg-01,02 - 58gb

[root at LSY-GL-03 .shard]# du -sh /diskForTestData/tst/lsy-tst/.shard/
70G     /diskForTestData/tst/lsy-tst/.shard/
[root at LSY-GL-02 host]# du -sh /diskForTestData/tst/lsy-tst/.shard/
58G     /diskForTestData/tst/lsy-tst/.shard/

Also I have folder /diskForTestData/tst/.shard with identical files (hardlinks,
i think)

What should I do with this situation ?
Copy .shard files from lsy-l-03 on lsy-gl-02,01 ?

Heal status count zero

[root at LSY-GL-03 tst]# gluster volume heal TST info
Brick lsy-gl-01:/diskForTestData/tst
Status: Connected
Number of entries: 0

Brick lsy-gl-02:/diskForTestData/tst
Status: Connected
Number of entries: 0

Brick lsy-gl-03:/diskForTestData/tst-fix
Status: Connected
Number of entries: 0

-- 
You are receiving this mail because:
You are on the CC list for the bug.
You are the assignee for the bug.