<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On 8 March 2017 at 23:34, Jarsulic, Michael [CRI] <span dir="ltr"><<a href="mailto:mjarsulic@bsd.uchicago.edu" target="_blank">mjarsulic@bsd.uchicago.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I am having issues with one of my systems that houses two bricks and want to bring it down for maintenance. I was able to remove the first brick successfully and committed the changes. The second brick is giving me a lot of problems with the rebalance when I try to remove it. It seems like it is stuck somewhere in that process:<br>
<br>
# gluster volume remove-brick hpcscratch cri16fs002-ib:/data/brick4/<wbr>scratch status<br>
Node Rebalanced-files size scanned failures skipped status run time in secs<br>
--------- ----------- ----------- ----------- ----------- ----------- ------------ --------------<br>
localhost 0 0Bytes 522 0 0 in progress 915.00<br>
<br>
<br>
The rebalance logs show the following error message.<br>
<br>
[2017-03-08 17:48:19.329934] I [MSGID: 109081] [dht-common.c:3810:dht_<wbr>setxattr] 0-hpcscratch-dht: fixing the layout of /userx/Ethiopian_imputation<br>
[2017-03-08 17:48:19.329960] I [MSGID: 109045] [dht-selfheal.c:1509:dht_fix_<wbr>layout_of_directory] 0-hpcscratch-dht: subvolume 0 (hpcscratch-client-0): 45778954 chunks<br>
[2017-03-08 17:48:19.329968] I [MSGID: 109045] [dht-selfheal.c:1509:dht_fix_<wbr>layout_of_directory] 0-hpcscratch-dht: subvolume 1 (hpcscratch-client-1): 45778954 chunks<br>
[2017-03-08 17:48:19.329974] I [MSGID: 109045] [dht-selfheal.c:1509:dht_fix_<wbr>layout_of_directory] 0-hpcscratch-dht: subvolume 2 (hpcscratch-client-4): 45778954 chunks<br>
[2017-03-08 17:48:19.329979] I [MSGID: 109045] [dht-selfheal.c:1509:dht_fix_<wbr>layout_of_directory] 0-hpcscratch-dht: subvolume 3 (hpcscratch-client-5): 45778954 chunks<br>
[2017-03-08 17:48:19.329983] I [MSGID: 109045] [dht-selfheal.c:1509:dht_fix_<wbr>layout_of_directory] 0-hpcscratch-dht: subvolume 4 (hpcscratch-client-7): 45778954 chunks<br>
[2017-03-08 17:48:19.400394] I [MSGID: 109036] [dht-common.c:7869:dht_log_<wbr>new_layout_for_dir_selfheal] 0-hpcscratch-dht: Setting layout of /userx/Ethiopian_imputation with [Subvol_name: hpcscratch-client-0, Err: -1 , Start: 1052915942 , Stop: 2105831883 , Hash: 1 ], [Subvol_name: hpcscratch-client-1, Err: -1 , Start: 3158747826 , Stop: 4294967295 , Hash: 1 ], [Subvol_name: hpcscratch-client-4, Err: -1 , Start: 0 , Stop: 1052915941 , Hash: 1 ], [Subvol_name: hpcscratch-client-5, Err: -1 , Start: 2105831884 , Stop: 3158747825 , Hash: 1 ], [Subvol_name: hpcscratch-client-7, Err: 22 , Start: 0 , Stop: 0 , Hash: 0 ],<br>
[2017-03-08 17:48:19.480882] I [dht-rebalance.c:2446:gf_<wbr>defrag_process_dir] 0-hpcscratch-dht: migrate data called on /userx/Ethiopian_imputation<br>
<br></blockquote><div><br></div><div><br></div><div>These are not error messages - these are info messages logged when the layout for a directory is being set and can be ignored. </div><div><br></div><div>The remove-brick operation is still in progress according to the status. What is it that makes you feel it is stuck? Is there no difference in the status output even after a considerable interval?</div><div><br></div><div>Regards,</div><div>Nithya</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Any suggestions on how I can get this brick out of play and preserve the data?<br>
<br>
--<br>
Mike Jarsulic<br>
Sr. HPC Administrator<br>
Center for Research Informatics | University of Chicago<br>
773.702.2066<br>
______________________________<wbr>_________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a href="http://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://lists.gluster.org/<wbr>mailman/listinfo/gluster-users</a><br>
</blockquote></div><br></div></div>