[Gluster-users] cluster.min-free-disk is not working in distributed disperse volume
Mohamed Pakkeer
mdfakkeer at gmail.com
Wed Aug 26 08:38:02 UTC 2015
Hi Susant,
I am monitoring the re-balance daemon data movement. I have few queries
about this data movement.I can see the data movement on the following cases.
1. Source sub-volume is* 90%* occupied and destination sub-volume is *98%*
occupied.
* log info*
[2015-08-26 06:53:26.219674] I [MSGID: 109022]
[dht-rebalance.c:1282:dht_migrate_file] 0-qubevaultdr-dht: completed
migration of
/Packages/Features/MPEG/B/BrokenHorses_FTR_S_EN-en_INT-XX_51_HD_RIM_IOP_OV/BrokenHorses_FTR_S_EN-en_INT-XX_51_HD_20150401_RIM_IOP_OV/BROKEN-HORSES_EN-en_INT_REV_MPEG-reel-3-mpeg2.mxf
from subvolume *qubevaultdr-disperse-7 to qubevaultdr-disperse-10*
2. Source sub-volume is 98% occupied and destination sub-volume is 97%
occupied
*log info*
[2015-08-26 06:52:18.071802] I [dht-rebalance.c:1002:dht_migrate_file]
0-glustertest-dht:
/Packages/Features/MPEG/B/Broker2_FTR_S_TE-XX_IN-A_51_HD_RIH_IOP_OV/Broker2_FTR-V1_
S_TE-XX_IN-A_51_HD_20140526_RIH_IOP_OV/BROKER-2_R3_MPEG_AUDIO_250514.mxf:
attempting to move from glustertest-disperse-10 to glustertest-disperse-8
3. source sub-volume is 94% occupied and destination sub-volume is 15%
occupied
*log info *
[2015-08-26 06:53:26.348804] I [MSGID: 109022]
[dht-rebalance.c:1282:dht_migrate_file] 0-glustertest-dht: completed
migration of
/Packages/Features/MPEG/B/Broker2_FTR_S_TE-XX_IN-A_51_HD_RIH_IOP_OV/Broker2_FTR-V1_S_TE-XX_IN-A_51_HD_20140526_RIH_IOP_OV/ASSETMAP
from sub-volume glustertest-disperse-26 to glustertest-disperse-56
I can see the *case 2 and 3 are valid* cases because source has more data
than destination. But Case 1 is* invalid,* because source sub-volume has
less data than destination sub-volume. Also i have one more query about
case 2. Why does the case 2 choose the destination as 94% occupied
sub-volume instead of so many under utilized sub-volumes( 13% to 15 %
sub-volumes- *glustertest-disperse-36 to glustertest-disperse-71*) ?
*Rebalance Log*
[2015-08-26 06:52:18.071802] I [dht-rebalance.c:1002:dht_migrate_file]
0-glustertest-dht:
/Packages/Features/MPEG/B/Broker2_FTR_S_TE-XX_IN-A_51_HD_RIH_IOP_OV/Broker2_FTR-V1_
S_TE-XX_IN-A_51_HD_20140526_RIH_IOP_OV/BROKER-2_R3_MPEG_AUDIO_250514.mxf:
attempting to move from glustertest-disperse-10 to glustertest-disperse-8
[2015-08-26 06:52:22.633777] I [MSGID: 109022]
[dht-rebalance.c:1282:dht_migrate_file] 0-glustertest-dht: completed
migration of /Packages/Features/MPEG/B/Broker2_FTR_S_TE-X
X_IN-A_51_HD_RIH_IOP_OV/Broker2_FTR-V1_S_TE-XX_IN-A_51_HD_20140526_RIH_IOP_OV/BROKER_SMK_MPEG_260514-reel-1-mpeg2.mxf
from subvolume glustertest-disperse-9 to glustertest-disperse-34
[2015-08-26 06:52:22.638955] I [dht-rebalance.c:1002:dht_migrate_file]
0-glustertest-dht:
/Packages/Features/MPEG/B/Broker2_FTR_S_TE-XX_IN-A_51_HD_RIH_IOP_OV/Broker2_FTR-V1_
S_TE-XX_IN-A_51_HD_20140526_RIH_IOP_OV/BROKER-2_R1-2-3-4-5-6-7-8_MPEG_240514-reel-3-mpeg2.mxf:
attempting to move from glustertest-disperse-14 to glustertest-disperse-39
[2015-08-26 06:52:50.171304] I [MSGID: 109022]
[dht-rebalance.c:1282:dht_migrate_file] 0-glustertest-dht: completed
migration of /Packages/Features/MPEG/B/Broker2_FTR_S_TE-X
X_IN-A_51_HD_RIH_IOP_OV/Broker2_FTR-V1_S_TE-XX_IN-A_51_HD_20140526_RIH_IOP_OV/BROKER-2_R3_MPEG_AUDIO_250514.mxf
from sub-volume glustertest-disperse-10 to glustertest-disperse-8
[2015-08-26 06:52:50.179238] I [dht-rebalance.c:1002:dht_migrate_file]
0-glustertest-dht:
/Packages/Features/MPEG/B/Broker2_FTR_S_TE-XX_IN-A_51_HD_RIH_IOP_OV/Broker2_FTR-V1_
S_TE-XX_IN-A_51_HD_20140526_RIH_IOP_OV/BROKER-2_R1-2-3-4-5-6-7-8_MPEG_240514-reel-4-mpeg2.mxf:
attempting to move from glustertest-disperse-16 to glustertest-disperse-5
[2015-08-26 06:53:08.579988] I [MSGID: 109022]
[dht-rebalance.c:1282:dht_migrate_file] 0-glustertest-dht: completed
migration of /Packages/Features/MPEG/B/BrokenHorses_FTR_S
_EN-en_INT-XX_51_HD_RIM_IOP_OV/BrokenHorses_FTR_S_EN-en_INT-XX_51_HD_20150401_RIM_IOP_OV/BROKEN-HORSES_EN-en_INT_REV_MPEG-reel-4-mpeg2.mxf
from subvolume glustertest-disperse-2 to glustertest-disperse-23
[2015-08-26 06:53:08.583021] I [dht-rebalance.c:1002:dht_migrate_file]
0-glustertest-dht:
/Packages/Features/MPEG/B/Broker2_FTR_S_TE-XX_IN-A_51_HD_RIH_IOP_OV/Broker2_FTR-V1_
S_TE-XX_IN-A_51_HD_20140526_RIH_IOP_OV/IntegrityCheckResults.log:
attempting to move from glustertest-disperse-18 to glustertest-disperse-0
[2015-08-26 06:53:08.698014] I [MSGID: 109022]
[dht-rebalance.c:1282:dht_migrate_file] 0-glustertest-dht: completed
migration of /Packages/Features/MPEG/B/Broker2_FTR_S_TE-X
X_IN-A_51_HD_RIH_IOP_OV/Broker2_FTR-V1_S_TE-XX_IN-A_51_HD_20140526_RIH_IOP_OV/IntegrityCheckResults.log
from sub-volume glustertest-disperse-18 to glustertest-disperse-0
[2015-08-26 06:53:08.703567] I [dht-rebalance.c:1002:dht_migrate_file]
0-glustertest-dht:
/Packages/Features/MPEG/B/Broker2_FTR_S_TE-XX_IN-A_51_HD_RIH_IOP_OV/Broker2_FTR-V1_
S_TE-XX_IN-A_51_HD_20140526_RIH_IOP_OV/BROKER-2_R1-2-3-4-5-6-7-8_MPEG_240514-reel-5-mpeg2.mxf:
attempting to move from glustertest-disperse-21 to glustertest-disperse-40
[2015-08-26 06:53:26.219674] I [MSGID: 109022]
[dht-rebalance.c:1282:dht_migrate_file] 0-glustertest-dht: completed
migration of
/Packages/Features/MPEG/B/BrokenHorses_FTR_S_EN-en_INT-XX_51_HD_RIM_IOP_OV/BrokenHorses_FTR_S_EN-en_INT-XX_51_HD_20150401_RIM_IOP_OV/BROKEN-HORSES_EN-en_INT_REV_MPEG-reel-3-mpeg2.mxf
from subvolume glustertest-disperse-7 to glustertest-disperse-10
[2015-08-26 06:53:26.226307] I [dht-rebalance.c:1002:dht_migrate_file]
0-glustertest-dht:
/Packages/Features/MPEG/B/Broker2_FTR_S_TE-XX_IN-A_51_HD_RIH_IOP_OV/Broker2_FTR-V1_S_TE-XX_IN-A_51_HD_20140526_RIH_IOP_OV/ASSETMAP:
attempting to move from glustertest-disperse-26 to glustertest-disperse-56
[2015-08-26 06:53:26.348804] I [MSGID: 109022]
[dht-rebalance.c:1282:dht_migrate_file] 0-glustertest-dht: completed
migration of
/Packages/Features/MPEG/B/Broker2_FTR_S_TE-XX_IN-A_51_HD_RIH_IOP_OV/Broker2_FTR-V1_S_TE-XX_IN-A_51_HD_20140526_RIH_IOP_OV/ASSETMAP
from subvolume glustertest-disperse-26 to glustertest-disperse-56
[2015-08-26 06:53:26.354215] I [dht-rebalance.c:1002:dht_migrate_file]
0-glustertest-dht:
/Packages/Features/MPEG/B/Broker2_FTR_S_TE-XX_IN-A_51_HD_RIH_IOP_OV/Broker2_FTR-V1_S_TE-XX_IN-A_51_HD_20140526_RIH_IOP_OV/BROKER-2_R8_MPEG_AUDIO_250514.mxf:
attempting to move from glustertest-disperse-29 to glustertest-disperse-16
*Subvolume0 to subvolume35*
root at glustertestdn001:# df -h
Filesystem Size Used Avail Use% Mounted on Subvolume Details
========== ==== ==== ===== ==== =========== ==================
/dev/sdb1 3.7T 3.6T 51G 99% /media/disk1 - glustertest-disperse-0
/dev/sdc1 3.7T 3.3T 384G 90% /media/disk2 - glustertest-disperse-1
/dev/sdd1 3.7T 3.6T 76G 98% /media/disk3 - glustertest-disperse-2
/dev/sde1 3.7T 3.5T 214G 95% /media/disk4 - glustertest-disperse-3
/dev/sdf1 3.7T 3.3T 365G 91% /media/disk5 - glustertest-disperse-4
/dev/sdg1 3.7T 3.5T 173G 96% /media/disk6 - glustertest-disperse-5
/dev/sdh1 3.7T 3.4T 284G 93% /media/disk7 - glustertest-disperse-6
/dev/sdi1 3.7T 3.3T 373G 90% /media/disk8 - glustertest-disperse-7
/dev/sdj1 3.7T 3.6T 122G 97% /media/disk9 - glustertest-disperse-8
/dev/sdk1 3.7T 3.4T 247G 94% /media/disk10 - glustertest-disperse-9
/dev/sdl1 3.7T 3.6T 108G 98% /media/disk11 -
glustertest-disperse-10
/dev/sdm1 3.7T 3.4T 252G 94% /media/disk12 -
glustertest-disperse-11
/dev/sdn1 3.7T 3.5T 241G 94% /media/disk13 -
glustertest-disperse-12
/dev/sdo1 3.7T 3.6T 136G 97% /media/disk14 -
glustertest-disperse-13
/dev/sdp1 3.7T 3.4T 247G 94% /media/disk15 -
glustertest-disperse-14
/dev/sdq1 3.7T 3.5T 170G 96% /media/disk16 -
glustertest-disperse-15
/dev/sdr1 3.7T 3.4T 257G 94% /media/disk17 -
glustertest-disperse-16
/dev/sds1 3.7T 3.4T 258G 94% /media/disk18 -
glustertest-disperse-17
/dev/sdt1 3.7T 3.5T 171G 96% /media/disk19 -
glustertest-disperse-18
/dev/sdu1 3.7T 3.5T 184G 96% /media/disk20 -
glustertest-disperse-19
/dev/sdv1 3.7T 3.4T 281G 93% /media/disk21 -
glustertest-disperse-20
/dev/sdw1 3.7T 3.3T 347G 91% /media/disk22 -
glustertest-disperse-21
/dev/sdx1 3.7T 3.5T 191G 95% /media/disk23 -
glustertest-disperse-22
/dev/sdy1 3.7T 3.5T 217G 95% /media/disk24 -
glustertest-disperse-23
/dev/sdz1 3.7T 3.4T 263G 93% /media/disk25 -
glustertest-disperse-24
/dev/sdaa1 3.7T 3.6T 66G 99% /media/disk26 -
glustertest-disperse-25
/dev/sdab1 3.7T 3.5T 225G 94% /media/disk27 -
glustertest-disperse-26
/dev/sdac1 3.7T 3.4T 310G 92% /media/disk28 -
glustertest-disperse-27
/dev/sdad1 3.7T 3.5T 159G 96% /media/disk29 -
glustertest-disperse-28
/dev/sdae1 3.7T 3.3T 373G 91% /media/disk30 -
glustertest-disperse-29
/dev/sdaf1 3.7T 3.4T 305G 92% /media/disk31 -
glustertest-disperse-30
/dev/sdag1 3.7T 3.5T 184G 96% /media/disk32 -
glustertest-disperse-31
/dev/sdah1 3.7T 3.5T 186G 96% /media/disk33 -
glustertest-disperse-32
/dev/sdai1 3.7T 3.3T 373G 91% /media/disk34 -
glustertest-disperse-33
/dev/sdaj1 3.7T 3.3T 361G 91% /media/disk35 -
glustertest-disperse-34
/dev/sdak1 3.7T 3.3T 375G 90% /media/disk36 -
glustertest-disperse-35
*Subvolume36 to subvolume71*
root at glustertestdn011:/var/log/glusterfs# df -h
Filesystem Size Used Avail Use% Mounted on SubVolume Detail
========== ==== ==== ===== ==== ========== ================
/dev/sdc1 3.7T 475G 3.2T 13% /media/disk1 -
glustertest-disperse-36
/dev/sdd1 3.7T 467G 3.2T 13% /media/disk2 -
glustertest-disperse-37
/dev/sde1 3.7T 492G 3.2T 14% /media/disk3 -
glustertest-disperse-38
/dev/sdf1 3.7T 470G 3.2T 13% /media/disk4 -
glustertest-disperse-39
/dev/sdg1 3.7T 498G 3.2T 14% /media/disk5 -
glustertest-disperse-40
/dev/sdh1 3.7T 461G 3.2T 13% /media/disk6 -
glustertest-disperse-41
/dev/sdi1 3.7T 451G 3.2T 13% /media/disk7 -
glustertest-disperse-42
/dev/sdj1 3.7T 464G 3.2T 13% /media/disk8 -
glustertest-disperse-43
/dev/sdk1 3.7T 426G 3.3T 12% /media/disk9 -
glustertest-disperse-44
/dev/sdl1 3.7T 425G 3.3T 12% /media/disk10 -
glustertest-disperse-45
/dev/sdm1 3.7T 460G 3.2T 13% /media/disk11 -
glustertest-disperse-46
/dev/sdn1 3.7T 478G 3.2T 13% /media/disk12 -
glustertest-disperse-47
/dev/sdo1 3.7T 475G 3.2T 13% /media/disk13 -
glustertest-disperse-48
/dev/sdp1 3.7T 520G 3.2T 14% /media/disk14 -
glustertest-disperse-49
/dev/sdq1 3.7T 456G 3.2T 13% /media/disk15 -
glustertest-disperse-50
/dev/sdr1 3.7T 442G 3.3T 12% /media/disk16 -
glustertest-disperse-51
/dev/sds1 3.7T 494G 3.2T 14% /media/disk17 -
glustertest-disperse-52
/dev/sdt1 3.7T 495G 3.2T 14% /media/disk18 -
glustertest-disperse-53
/dev/sdu1 3.7T 426G 3.3T 12% /media/disk19 -
glustertest-disperse-54
/dev/sdv1 3.7T 471G 3.2T 13% /media/disk20 -
glustertest-disperse-55
/dev/sdw1 3.7T 531G 3.2T 15% /media/disk21 -
glustertest-disperse-56
/dev/sdx1 3.7T 576G 3.1T 16% /media/disk22 -
glustertest-disperse-57
/dev/sdy1 3.7T 411G 3.3T 12% /media/disk23 -
glustertest-disperse-58
/dev/sdz1 3.7T 455G 3.2T 13% /media/disk24 -
glustertest-disperse-59
/dev/sdaa1 3.7T 427G 3.3T 12% /media/disk25 -
glustertest-disperse-60
/dev/sdab1 3.7T 442G 3.3T 12% /media/disk26 -
glustertest-disperse-61
/dev/sdac1 3.7T 449G 3.2T 13% /media/disk27 -
glustertest-disperse-62
/dev/sdad1 3.7T 446G 3.3T 12% /media/disk28 -
glustertest-disperse-63
/dev/sdae1 3.7T 412G 3.3T 12% /media/disk29 -
glustertest-disperse-64
/dev/sdaf1 3.7T 480G 3.2T 13% /media/disk30 -
glustertest-disperse-65
/dev/sdag1 3.7T 485G 3.2T 14% /media/disk31 -
glustertest-disperse-66
/dev/sdah1 3.7T 453G 3.2T 13% /media/disk32 -
glustertest-disperse-67
/dev/sdai1 3.7T 410G 3.3T 12% /media/disk33 -
glustertest-disperse-68
/dev/sdaj1 3.7T 457G 3.2T 13% /media/disk34 -
glustertest-disperse-69
/dev/sdak1 3.7T 444G 3.3T 12% /media/disk35 -
glustertest-disperse-70
/dev/sdal1 3.7T 484G 3.2T 13% /media/disk36 -
glustertest-disperse-71
Is this the normal behavior of DHT in distributed disperse volume ?
Thanks
Backer
On Tue, Aug 25, 2015 at 12:36 PM, Susant Palai <spalai at redhat.com> wrote:
> Mohamed,
> Will investigate in to weighted rebalance behavior.
>
> Susant
>
> ----- Original Message -----
> From: "Mohamed Pakkeer" <mdfakkeer at gmail.com>
> To: "Susant Palai" <spalai at redhat.com>
> Cc: "Mathieu Chateau" <mathieu.chateau at lotp.fr>, "gluster-users" <
> gluster-users at gluster.org>, "Gluster Devel" <gluster-devel at gluster.org>
> Sent: Tuesday, 25 August, 2015 9:40:01 AM
> Subject: Re: [Gluster-users] cluster.min-free-disk is not working in
> distributed disperse volume
>
>
> Hi Sasant,
> We have created the disperse volume across nodes. We stopped all the
> upload operations and started the rebalance last night.After overnight
> re-balance, some harddisk is occupied 100% and some disks have 13% disk
> space.
>
>
> disk1 belongs to disperse-set-0 ..... disk36 belongs to disperse-set-35
>
>
> df -h result of one data node
>
>
>
> /dev/sdb1 3.7T 3.7T 545M 100% /media/disk1
> /dev/sdc1 3.7T 3.2T 496G 87% /media/disk2
> /dev/sdd1 3.7T 3.7T 30G 100% /media/disk3
> /dev/sde1 3.7T 3.5T 173G 96% /media/disk4
> /dev/sdf1 3.7T 3.2T 458G 88% /media/disk5
> /dev/sdg1 3.7T 3.5T 143G 97% /media/disk6
> /dev/sdh1 3.7T 3.5T 220G 95% /media/disk7
> /dev/sdi1 3.7T 3.3T 415G 89% /media/disk8
> /dev/sdj1 3.7T 3.6T 72G 99% /media/disk9
> /dev/sdk1 3.7T 3.5T 186G 96% /media/disk10
> /dev/sdl1 3.7T 3.6T 65G 99% /media/disk11
> /dev/sdm1 3.7T 3.5T 195G 95% /media/disk12
> /dev/sdn1 3.7T 3.5T 199G 95% /media/disk13
> /dev/sdo1 3.7T 3.6T 78G 98% /media/disk14
> /dev/sdp1 3.7T 3.5T 200G 95% /media/disk15
> /dev/sdq1 3.7T 3.6T 119G 97% /media/disk16
> /dev/sdr1 3.7T 3.5T 206G 95% /media/disk17
> /dev/sds1 3.7T 3.5T 193G 95% /media/disk18
> /dev/sdt1 3.7T 3.6T 131G 97% /media/disk19
> /dev/sdu1 3.7T 3.5T 141G 97% /media/disk20
> /dev/sdv1 3.7T 3.5T 243G 94% /media/disk21
> /dev/sdw1 3.7T 3.4T 299G 92% /media/disk22
> /dev/sdx1 3.7T 3.5T 163G 96% /media/disk23
> /dev/sdy1 3.7T 3.5T 168G 96% /media/disk24
> /dev/sdz1 3.7T 3.5T 219G 95% /media/disk25
> /dev/sdaa1 3.7T 3.7T 37G 100% /media/disk26
> /dev/sdab1 3.7T 3.5T 172G 96% /media/disk27
> /dev/sdac1 3.7T 3.4T 276G 93% /media/disk28
> /dev/sdad1 3.7T 3.6T 108G 98% /media/disk29
> /dev/sdae1 3.7T 3.3T 399G 90% /media/disk30
> /dev/sdaf1 3.7T 3.5T 240G 94% /media/disk31
> /dev/sdag1 3.7T 3.6T 122G 97% /media/disk32
> /dev/sdah1 3.7T 3.5T 147G 97% /media/disk33
> /dev/sdai1 3.7T 3.4T 342G 91% /media/disk34
> /dev/sdaj1 3.7T 3.4T 288G 93% /media/disk35
> /dev/sdak1 3.7T 3.4T 342G 91% /media/disk36
>
>
> disk1 belongs to disperse-set-0. Rebalancer logs shows, still rebalancer
> is trying to fill the disperse-set-0 after filling to 100%
>
>
>
>
> [2015-08-24 19:52:53.036622] E [MSGID: 109023]
> [dht-rebalance.c:672:__dht_check_free_space] 0-glustertest-dht: data
> movement attempted from node (glustertest-disperse-7) to node
> (glustertest-disperse-0) which does not have required free space for
> (/Packages/Features/MPEG/A/AMEO-N-CHALLANGE_FTR_S_BEN-XX_IN-UA_51_HD_RIC_OV/AMEO-N-CHALLANGE_FTR_S_BEN-XX_IN-UA_51_HD_20110521_RIC_OV/AMI-NEBO-C_R3_AUDIO_190511.mxf)
>
>
> [2015-08-24 19:52:53.042026] I [dht-rebalance.c:1002:dht_migrate_file]
> 0-glustertest-dht:
> /Packages/Features/MPEG/A/AMEO-N-CHALLANGE_FTR_S_BEN-XX_IN-UA_51_HD_RIC_OV/AMEO-N-CHALLANGE_FTR_S_BEN-XX_IN-UA_51_HD_20110521_RIC_OV/AMINEBO-CHALLANGE_BEN_R1-2-3-4-5-6_MPEG_200511-reel-5-mpeg2.mxf:
> attempting to move from glustertest-disperse-13 to glustertest-disperse-0
>
>
> I think, cluster.weighted-rebalance and cluster.min-free-dis k have bugs
> for re-balancing the data, based on weight and disk free space.
>
>
> Thanks
> Backer
>
>
>
>
> On Mon, Aug 24, 2015 at 4:28 PM, Mohamed Pakkeer < mdfakkeer at gmail.com >
> wrote:
>
>
>
> Hi Susant,
>
>
> Thanks for your quick reply. We are not updating any files. Actually we
> are archiving video files on this cluster. I think there is a bug in
> cluster.min-free-disk.
>
>
> Also i would like to know about rebalance the cluster. Currently we have
> 20 nodes and 10 nodes hard disks are almost full . So we need to rebalance
> the data. If i run the rebalancer, it starts on first node(node1) and
> starts the migration process. The first node cpu usage is always high
> during rebalance compare with rest of the cluster nodes.To reduce the cpu
> usage of rebalancer datanode( node1), i peer a new node( without disk) for
> rebalance and start the rebalancer. It starts again the rebalancer on same
> node1. How can we run a rebalancer on a dedicated node?
>
>
> Also we are facing memory leaks in fixlayout and heal full operations.
>
>
> Regards
> Backer
>
>
>
>
> On Mon, Aug 24, 2015 at 2:57 PM, Susant Palai < spalai at redhat.com > wrote:
>
>
> Hi,
> Cluster.min-free-disk controls new file creation on the bricks. If you
> happen to write to the existing files on the brick and that is leading to
> brick getting full, then most probably you should run a rebalance.
>
> Regards,
> Susant
>
>
>
> ----- Original Message -----
> From: "Mathieu Chateau" < mathieu.chateau at lotp.fr >
> To: "Mohamed Pakkeer" < mdfakkeer at gmail.com >
> Cc: "gluster-users" < gluster-users at gluster.org >, "Gluster Devel" <
> gluster-devel at gluster.org >
> Sent: Monday, 24 August, 2015 2:47:00 PM
> Subject: Re: [Gluster-users] cluster.min-free-disk is not working in
> distributed disperse volume
>
>
>
>
> 720 brick! Respect !
> Le 24 août 2015 09:48, "Mohamed Pakkeer" < mdfakkeer at gmail.com > a écrit :
>
>
>
> Hi,
>
>
> I have a cluster of 720 bricks, all bricks are 4TB in size. I have change
> the cluster.min-free-disk default value 10% to 3%. So all the disks should
> have 3% minimum disk space free. But some cluster disks are getting full
> now. Is there any additional configuration for keeping some percentage of
> disk space kept free?
>
>
>
>
>
> Volume Name: glustertest
> Type: Distributed-Disperse
> Volume ID: 2b575b5c-df2e-449c-abb9-c56cec27e609
> Status: Started
> Number of Bricks: 72 x (8 + 2) = 720
> Transport-type: tcp
>
>
>
>
>
> Options Reconfigured:
> features.default-soft-limit: 95%
> cluster.min-free-disk: 3%
> performance.readdir-ahead: on
>
>
> df -h of one node
>
>
>
> /dev/sdb1 3.7T 3.6T 132G 97% /media/disk1
> /dev/sdc1 3.7T 3.2T 479G 88% /media/disk2
> /dev/sdd1 3.7T 3.6T 109G 98% /media/disk3
>
>
> Any help will be greatly appreciated.
>
>
>
>
> Regards
> Backer
>
>
>
>
>
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>
>
>
>
>
>
>
--
Thanks & Regards
K.Mohamed Pakkeer
Mobile- 0091-8754410114
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150826/eb3b3411/attachment.html>
More information about the Gluster-users
mailing list