[Gluster-users] Mounted size reported incorrectly after replacing bricks

[Patrick Dijkgraaf]

Hi all, I hope this message finds you well.

I sent some messages earlier, but I found that they bounced a lot due
to DMARC/SPF, so I am sending this question again from another mail
account. Please accept my apologies for spamming.

I've been running a Gluster volume (32 bricks in distributed,
replicated mode) on my 2 home servers for about 1,5 years now. I'm
generally very happy with it!
It's running on Arch Linux and the current version of glusterfs is
10.1.

Because some disks were about to fail, I started replacing multiple
bricks. And taking advantage of this, I replaced them with a larger
disk (4TB -> 8TB). Healing took care of copying all data to the new
brick and it finished succesfully. However, I see an incorrect size on
the mounted Gluster volume.

Some things I have found:

 * shared-brick-count in /var/lib/glusterd/vols/data/* is higher than 1
   on some local bricks, even though they are actually on separate file
   systems
 * I have duplicate brick-fsid numbers in
   /var/lib/glusterd/vols/data/bricks/*, even though they are actually
   on separate file systems
 * I have restarted glusterd and still have the duplicate brick-fsid's

So I am wondering where the duplicate FSIDs come from, and how to
(forcefully?) resolve them. Can I safely alter them in
/var/lib/glusterd/vols/data/bricks/* and restart glusterd maybe?

I *may* at some point have accidentally replaced a brick to a wrong
location, being either the parent file system or another brick. But I
have corrected this by replacing it again to the correct location. Each
time I used the "gluster volume replace-brick" command.

I have attached what I believe would be all relevant information to
diagnose the issue.
Please let me know if I can provide more information to get this issue
resolved.

-- 
groet / cheers,
Patrick Dijkgraaf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20220419/40df9490/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Gluster issue.zip
Type: application/zip
Size: 211469 bytes
Desc: not available
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20220419/40df9490/attachment.zip>