<div dir="ltr"><div class="gmail_extra"><br><div class="gmail_quote">On 7 July 2017 at 15:42, Szymon Miotk <span dir="ltr"><<a href="mailto:szymon.miotk@gmail.com" target="_blank">szymon.miotk@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hello everyone,<br>
<br>
<br>
I have problem rebalancing Gluster volume.<br>
Gluster version is 3.7.3.<br>
My 1x3 replicated volume become full, so I've added three more bricks<br>
to make it 2x3 and wanted to rebalance.<br>
But every time I start rebalancing, it fails immediately.<br>
Rebooting Gluster nodes doesn't help.<br>
<br>
# gluster volume rebalance gsae_artifactory_cluster_<wbr>storage start<br>
volume rebalance: gsae_artifactory_cluster_<wbr>storage: success: Rebalance<br>
on gsae_artifactory_cluster_<wbr>storage has been started successfully. Use<br>
rebalance status command to check status of the rebalance process.<br>
ID: b22572ff-7575-4557-8317-<wbr>765f7e52d445<br>
<br>
# gluster volume rebalance gsae_artifactory_cluster_<wbr>storage status<br>
Node Rebalanced-files<br>
size scanned failures skipped status<br>
run time in secs<br>
--------- -----------<br>
----------- ----------- ----------- -----------<br>
------------ --------------<br>
localhost 0<br>
0Bytes 0 0 0 failed<br>
0.00<br>
10.239.40.9 0<br>
0Bytes 0 0 0 failed<br>
0.00<br>
10.239.40.8 0<br>
0Bytes 0 0 0 failed<br>
0.00<br>
volume rebalance: gsae_artifactory_cluster_<wbr>storage: success:<br>
<br>
The messages in logfiles mention 'failed to get index':<br>
[2017-07-07 10:07:18.230202] E [MSGID: 106062]<br>
[glusterd-utils.c:7997:<wbr>glusterd_volume_rebalance_use_<wbr>rsp_dict]<br>
0-glusterd: failed to get index<br>
<br>
and then the rebalance process crashes:<br>
pending frames:<br>
frame : type(0) op(0)<br>
frame : type(0) op(0)<br>
frame : type(0) op(0)<br>
patchset: git://<a href="http://git.gluster.com/glusterfs.git" rel="noreferrer" target="_blank">git.gluster.com/<wbr>glusterfs.git</a><br>
signal received: 4<br>
time of crash:<br>
2017-07-07 10:07:23<br>
configuration details:<br>
argp 1<br>
backtrace 1<br>
dlfcn 1<br>
libpthread 1<br>
llistxattr 1<br>
setfsid 1<br>
spinlock 1<br>
epoll.h 1<br>
xattr.h 1<br>
st_atim.tv_nsec 1<br>
package-string: glusterfs 3.7.3<br>
/usr/lib/x86_64-linux-gnu/<wbr>libglusterfs.so.0(_gf_msg_<wbr>backtrace_nomem+0x92)[<wbr>0x7f24de214502]<br>
/usr/lib/x86_64-linux-gnu/<wbr>libglusterfs.so.0(gf_print_<wbr>trace+0x31d)[0x7f24de23059d]<br>
/lib/x86_64-linux-gnu/libc.so.<wbr>6(+0x36d40)[0x7f24dd612d40]<br>
/lib/x86_64-linux-gnu/<wbr>libpthread.so.0(+0x11f6b)[<wbr>0x7f24dd9b2f6b]<br>
/usr/lib/x86_64-linux-gnu/<wbr>libglusterfs.so.0(inode_ref+<wbr>0x19)[0x7f24de234e69]<br>
/usr/lib/x86_64-linux-gnu/<wbr>libglusterfs.so.0(loc_copy+<wbr>0x4a)[0x7f24de21291a]<br>
/usr/lib/x86_64-linux-gnu/<wbr>glusterfs/3.7.3/xlator/<wbr>cluster/distribute.so(dht_<wbr>local_init+0x4b)[<wbr>0x7f24d851f51b]<br>
/usr/lib/x86_64-linux-gnu/<wbr>glusterfs/3.7.3/xlator/<wbr>cluster/distribute.so(dht_<wbr>lookup+0x91)[0x7f24d8550521]<br>
/usr/lib/x86_64-linux-gnu/<wbr>libglusterfs.so.0(syncop_<wbr>lookup+0x1a2)[0x7f24de258fc2]<br>
/usr/lib/x86_64-linux-gnu/<wbr>glusterfs/3.7.3/xlator/<wbr>cluster/distribute.so(gf_<wbr>defrag_fix_layout+0x87)[<wbr>0x7f24d85289e7]<br>
/usr/lib/x86_64-linux-gnu/<wbr>glusterfs/3.7.3/xlator/<wbr>cluster/distribute.so(gf_<wbr>defrag_start_crawl+0x6d3)[<wbr>0x7f24d8529ce3]<br>
/usr/lib/x86_64-linux-gnu/<wbr>libglusterfs.so.0(synctask_<wbr>wrap+0x12)[0x7f24de255822]<br>
/lib/x86_64-linux-gnu/libc.so.<wbr>6(+0x498b0)[0x7f24dd6258b0]<br>
<br>
<br>
Anybody has a clue how to fix 'failed to get index' error?<br></blockquote><div><br></div><div>Do you have the coredump? If yes, please send that and the rebalance logs across.</div><div><br></div><div>Regards,</div><div>Nithya </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Thank you in advance!<br>
<span class="gmail-HOEnZb"><font color="#888888">Szymon Miotk<br>
______________________________<wbr>_________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a href="http://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://lists.gluster.org/<wbr>mailman/listinfo/gluster-users</a><br>
</font></span></blockquote></div><br></div></div>