[Bugs] [Bug 1741899] New: the volume of occupied space in the bricks of gluster volume (3 nodes replica) differs on nodes and the healing does not fix it
bugzilla at redhat.com
bugzilla at redhat.com
Fri Aug 16 11:49:02 UTC 2019
https://bugzilla.redhat.com/show_bug.cgi?id=1741899
Bug ID: 1741899
Summary: the volume of occupied space in the bricks of gluster
volume (3 nodes replica) differs on nodes and the
healing does not fix it
Product: GlusterFS
Version: 5
Hardware: x86_64
OS: Linux
Status: NEW
Component: glusterd
Severity: medium
Assignee: bugs at gluster.org
Reporter: s.pleshkov at hostco.ru
CC: bugs at gluster.org
Target Milestone: ---
Classification: Community
Description of problem:
I have a gluster volume on 3 nodes (replicate) with following configuration
[root at LSY-GL-0(1,2,3) /]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.6 (Maipo)
[root at LSY-GL-02 host]# gluster volume info TST
Volume Name: TST
Type: Replicate
Volume ID: a96c7b8c-61ec-4a4d-b47e-b445faf6c39b
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: lsy-gl-01:/diskForTestData/tst
Brick2: lsy-gl-02:/diskForTestData/tst
Brick3: lsy-gl-03:/diskForTestData/tst
Options Reconfigured:
cluster.favorite-child-policy: size
features.shard-block-size: 64MB
features.shard: on
performance.io-thread-count: 24
client.event-threads: 24
server.event-threads: 24
server.allow-insecure: on
network.ping-timeout: 5
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.heal-timeout: 120
Recently this volume been moved to other disk by command
gluster volume replace-brick TST lsy-gl-0(1,2,3):/diskForData/tst
lsy-gl-0(1,2,3):/diskForTestData/tst commit force
sequentialy, started with lsy-gl-03 node, all nodes been online
And now i have this state
[root at LSY-GL-02 host]# gluster volume status TST detail
Status of volume: TST
------------------------------------------------------------------------------
Brick : Brick lsy-gl-01:/diskForTestData/tst
TCP Port : 49154
RDMA Port : 0
Online : Y
Pid : 7555
File System : xfs
Device : /dev/sdc1
Mount Options : rw,seclabel,relatime,attr2,inode64,noquota
Inode Size : 512
Disk Space Free : 399.9GB
Total Disk Space : 499.8GB
Inode Count : 262143488
Free Inodes : 261684925
------------------------------------------------------------------------------
Brick : Brick lsy-gl-02:/diskForTestData/tst
TCP Port : 49154
RDMA Port : 0
Online : Y
Pid : 25732
File System : xfs
Device : /dev/sdc1
Mount Options : rw,seclabel,relatime,attr2,inode64,noquota
Inode Size : 512
Disk Space Free : 399.9GB
Total Disk Space : 499.8GB
Inode Count : 262143488
Free Inodes : 261684925
------------------------------------------------------------------------------
Brick : Brick lsy-gl-03:/diskForTestData/tst
TCP Port : 49154
RDMA Port : 0
Online : Y
Pid : 25243
File System : xfs
Device : /dev/sdc1
Mount Options : rw,seclabel,relatime,attr2,inode64,noquota
Inode Size : 512
Disk Space Free : 357.6GB
Total Disk Space : 499.8GB
Inode Count : 262143488
Free Inodes : 261684112
[root at LSY-GL-02 host]# gluster volume heal TST full
Launching heal operation to perform full self heal on volume TST has been
successful
Use heal info commands to check status.
[root at LSY-GL-02 host]# gluster volume heal TST info
Brick lsy-gl-01:/diskForTestData/tst
Status: Connected
Number of entries: 0
Brick lsy-gl-02:/diskForTestData/tst
Status: Connected
Number of entries: 0
Brick lsy-gl-03:/diskForTestData/tst
Status: Connected
Number of entries: 0
[root at LSY-GL-01 /]# df -Th
Filesystem Type Size Used Avail Use% Mounted
on
LSY-GL-01:/TST fuse.glusterfs 500G 148G 353G 30%
/mnt/tst
/dev/sdc1 xfs 500G 100G 400G 20%
/diskForTestData
[root at LSY-GL-02 host]# df -Th
Filesystem Type Size Used Avail Use% Mounted
on
LSY-GL-02:/TST fuse.glusterfs 500G 148G 353G 30%
/mnt/tst
/dev/sdc1 xfs 500G 100G 400G 20%
/diskForTestData
[root at LSY-GL-03 host]# df -Th
Filesystem Type Size Used Avail Use% Mounted
on
/dev/sdc1 xfs 500G 143G 358G 29%
/diskForTestData
LSY-GL-03:/TST fuse.glusterfs 500G 148G 353G 30%
/mnt/tst
Version-Release number of selected component (if applicable):
[root at LSY-GL-0(1,2,3) /]# rpm -qa | grep gluster*
glusterfs-libs-5.5-1.el7.x86_64
glusterfs-fuse-5.5-1.el7.x86_64
glusterfs-client-xlators-5.5-1.el7.x86_64
centos-release-gluster5-1.0-1.el7.centos.noarch
glusterfs-api-5.5-1.el7.x86_64
glusterfs-cli-5.5-1.el7.x86_64
nfs-ganesha-gluster-2.7.1-1.el7.x86_64
glusterfs-5.5-1.el7.x86_64
glusterfs-server-5.5-1.el7.x86_64
How reproducible:
Umm, I will test it again soon and do comment
Steps to Reproduce:
1.
2.
3.
Actual results:
Size of brick on lsy-gl-01, and lsy-gl-02 differ from size brick on lsy-gl-03.
Healing full not fixed this situation
Expected results:
What things I should do to fix it ?
Additional info:
--
You are receiving this mail because:
You are on the CC list for the bug.
You are the assignee for the bug.
More information about the Bugs
mailing list