<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><br class=""></div><div class="">Hi Ashish,</div><div class=""><br class=""></div><div class="">in attachment you can find the rebalance log file and the last updated brick log file (the other files in /var/log/glusterfs/bricks directory seem to be too old).</div><div class="">I just stopped the running rebalance (as you can see at the bottom of the rebalance log file).</div><div class="">So, if exists a safe procedure to correct the problem I would like execute it.</div><div class=""><br class=""></div><div class="">I don’t know if I can ask you it, but, if it is possible, could you please describe me step by step the right procedure to remove the newly added bricks without losing the data that have been already rebalanced?</div><div class=""><br class=""></div><div class="">The following outputs show the result of “df -h” command executed on one of the first 3 nodes (s01, s02, s03) already existing and on one of the last 3 nodes (s04, s05, s06) added recently.</div><div class=""><br class=""></div><div class=""><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">[root@s06 bricks]# df -h</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">File system Dim. Usati Dispon. Uso% Montato su</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/cl_s06-root 100G 2,1G 98G 3% /</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">devtmpfs 32G 0 32G 0% /dev</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">tmpfs 32G 4,0K 32G 1% /dev/shm</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">tmpfs 32G 26M 32G 1% /run</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">tmpfs 32G 0 32G 0% /sys/fs/cgroup</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/cl_s06-var 100G 2,0G 99G 2% /var</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/cl_s06-gluster 100G 33M 100G 1% /gluster</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/sda1 1014M 152M 863M 15% /boot</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/gluster_vgd-gluster_lvd 9,0T 807G 8,3T 9% /gluster/mnt3</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/gluster_vgg-gluster_lvg 9,0T 807G 8,3T 9% /gluster/mnt6</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/gluster_vgc-gluster_lvc 9,0T 807G 8,3T 9% /gluster/mnt2</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/gluster_vge-gluster_lve 9,0T 807G 8,3T 9% /gluster/mnt4</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/gluster_vgj-gluster_lvj 9,0T 887G 8,2T 10% /gluster/mnt9</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/gluster_vgb-gluster_lvb 9,0T 807G 8,3T 9% /gluster/mnt1</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/gluster_vgh-gluster_lvh 9,0T 887G 8,2T 10% /gluster/mnt7</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/gluster_vgf-gluster_lvf 9,0T 807G 8,3T 9% /gluster/mnt5</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/gluster_vgi-gluster_lvi 9,0T 887G 8,2T 10% /gluster/mnt8</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/gluster_vgl-gluster_lvl 9,0T 887G 8,2T 10% /gluster/mnt11</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/gluster_vgk-gluster_lvk 9,0T 887G 8,2T 10% /gluster/mnt10</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/gluster_vgm-gluster_lvm 9,0T 887G 8,2T 10% /gluster/mnt12</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">tmpfs 6,3G 0 6,3G 0% /run/user/0</span></div></div><div class=""><br class=""></div><div class=""><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">[root@s01 ~]# df -h</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">File system Dim. Usati Dispon. Uso% Montato su</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/cl_s01-root 100G 5,3G 95G 6% /</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">devtmpfs 32G 0 32G 0% /dev</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">tmpfs 32G 39M 32G 1% /dev/shm</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">tmpfs 32G 26M 32G 1% /run</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">tmpfs 32G 0 32G 0% /sys/fs/cgroup</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/cl_s01-var 100G 11G 90G 11% /var</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/md127 1015M 151M 865M 15% /boot</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/cl_s01-gluster 100G 33M 100G 1% /gluster</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/gluster_vgi-gluster_lvi 9,0T 5,5T 3,6T 61% /gluster/mnt7</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/gluster_vgm-gluster_lvm 9,0T 5,4T 3,6T 61% /gluster/mnt11</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/gluster_vgf-gluster_lvf 9,0T 5,7T 3,4T 63% /gluster/mnt4</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/gluster_vgl-gluster_lvl 9,0T 5,8T 3,3T 64% /gluster/mnt10</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/gluster_vgj-gluster_lvj 9,0T 5,5T 3,6T 61% /gluster/mnt8</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/gluster_vgn-gluster_lvn 9,0T 5,4T 3,6T 61% /gluster/mnt12</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/gluster_vgk-gluster_lvk 9,0T 5,8T 3,3T 64% /gluster/mnt9</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/gluster_vgh-gluster_lvh 9,0T 5,6T 3,5T 63% /gluster/mnt6</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/gluster_vgg-gluster_lvg 9,0T 5,6T 3,5T 63% /gluster/mnt5</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/gluster_vge-gluster_lve 9,0T 5,7T 3,4T 63% /gluster/mnt3</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/gluster_vgc-gluster_lvc 9,0T 5,6T 3,5T 62% /gluster/mnt1</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">/dev/mapper/gluster_vgd-gluster_lvd 9,0T 5,6T 3,5T 62% /gluster/mnt2</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">tmpfs 6,3G 0 6,3G 0% /run/user/0</span></div><div style="margin: 0px; line-height: normal; font-family: 'Courier New'; color: rgb(184, 180, 59); background-color: rgb(0, 0, 0);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">s01-stg:tier2 420T 159T 262T 38% /tier2</span></div></div><div class=""><br class=""></div><div class="">As you can see, used space value of each brick of the last servers is about 800GB.</div><div class=""><br class=""></div><div class="">Thank you,</div><div class="">Mauro</div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""></div></body></html>