<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On 13 July 2017 at 14:02, Szymon Miotk <span dir="ltr"><<a href="mailto:szymon.miotk@gmail.com" target="_blank">szymon.miotk@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi Nithya,<br>
I see index in context:<br>
<span class="gmail-">[2017-07-07 10:07:18.230202] E [MSGID: 106062]<br>
[glusterd-utils.c:7997:<wbr>glusterd_volume_rebalance_use_<wbr>rsp_dict]<br>
0-glusterd: failed to get index<br>
<br>
</span>I wonder if there is anything I can do to fix it.<br>
I was trying to strace gluster process but still have no clue what<br>
exactly is gluster index.<br>
<br></blockquote><div><br></div><div>This message looks like a glusterd log message (not rebalance) and glusterd_volume_rebalance_use_<wbr>rsp_dict possibly failed because the rebalance process had already crashed. The index here is simply the value of the number of nodes on which the rebalance process should be running - it is used to track the rebalance status on all nodes.</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Best regards,<br>
Szymon Miotk<br>
<div class="gmail-HOEnZb"><div class="gmail-h5"><br>
On Thu, Jul 13, 2017 at 10:12 AM, Nithya Balachandran<br>
<<a href="mailto:nbalacha@redhat.com">nbalacha@redhat.com</a>> wrote:<br>
> Hi Szymon,<br>
><br>
> I have received the files and will take a look and get back to you.<br>
><br>
> In what context are you seeing index?<br>
><br>
> Thanks,<br>
> Nithya<br>
><br>
> On 11 July 2017 at 01:15, Szymon Miotk <<a href="mailto:szymon.miotk@gmail.com">szymon.miotk@gmail.com</a>> wrote:<br>
>><br>
>> Hi Nithya,<br>
>><br>
>> the files were sent to priv to avoid spamming the list with large<br>
>> attachments.<br>
>> Could someone explain what is index in Gluster?<br>
>> Unfortunately index is popular word, so googling is not very helpful.<br>
>><br>
>> Best regards,<br>
>> Szymon Miotk<br>
>><br>
>> On Sun, Jul 9, 2017 at 6:37 PM, Nithya Balachandran <<a href="mailto:nbalacha@redhat.com">nbalacha@redhat.com</a>><br>
>> wrote:<br>
>> ><br>
>> > On 7 July 2017 at 15:42, Szymon Miotk <<a href="mailto:szymon.miotk@gmail.com">szymon.miotk@gmail.com</a>> wrote:<br>
>> >><br>
>> >> Hello everyone,<br>
>> >><br>
>> >><br>
>> >> I have problem rebalancing Gluster volume.<br>
>> >> Gluster version is 3.7.3.<br>
>> >> My 1x3 replicated volume become full, so I've added three more bricks<br>
>> >> to make it 2x3 and wanted to rebalance.<br>
>> >> But every time I start rebalancing, it fails immediately.<br>
>> >> Rebooting Gluster nodes doesn't help.<br>
>> >><br>
>> >> # gluster volume rebalance gsae_artifactory_cluster_<wbr>storage start<br>
>> >> volume rebalance: gsae_artifactory_cluster_<wbr>storage: success: Rebalance<br>
>> >> on gsae_artifactory_cluster_<wbr>storage has been started successfully. Use<br>
>> >> rebalance status command to check status of the rebalance process.<br>
>> >> ID: b22572ff-7575-4557-8317-<wbr>765f7e52d445<br>
>> >><br>
>> >> # gluster volume rebalance gsae_artifactory_cluster_<wbr>storage status<br>
>> >> Node Rebalanced-files<br>
>> >> size scanned failures skipped status<br>
>> >> run time in secs<br>
>> >> --------- -----------<br>
>> >> ----------- ----------- ----------- -----------<br>
>> >> ------------ --------------<br>
>> >> localhost 0<br>
>> >> 0Bytes 0 0 0 failed<br>
>> >> 0.00<br>
>> >> 10.239.40.9 0<br>
>> >> 0Bytes 0 0 0 failed<br>
>> >> 0.00<br>
>> >> 10.239.40.8 0<br>
>> >> 0Bytes 0 0 0 failed<br>
>> >> 0.00<br>
>> >> volume rebalance: gsae_artifactory_cluster_<wbr>storage: success:<br>
>> >><br>
>> >> The messages in logfiles mention 'failed to get index':<br>
>> >> [2017-07-07 10:07:18.230202] E [MSGID: 106062]<br>
>> >> [glusterd-utils.c:7997:<wbr>glusterd_volume_rebalance_use_<wbr>rsp_dict]<br>
>> >> 0-glusterd: failed to get index<br>
>> >><br>
>> >> and then the rebalance process crashes:<br>
>> >> pending frames:<br>
>> >> frame : type(0) op(0)<br>
>> >> frame : type(0) op(0)<br>
>> >> frame : type(0) op(0)<br>
>> >> patchset: git://<a href="http://git.gluster.com/glusterfs.git" rel="noreferrer" target="_blank">git.gluster.com/<wbr>glusterfs.git</a><br>
>> >> signal received: 4<br>
>> >> time of crash:<br>
>> >> 2017-07-07 10:07:23<br>
>> >> configuration details:<br>
>> >> argp 1<br>
>> >> backtrace 1<br>
>> >> dlfcn 1<br>
>> >> libpthread 1<br>
>> >> llistxattr 1<br>
>> >> setfsid 1<br>
>> >> spinlock 1<br>
>> >> epoll.h 1<br>
>> >> xattr.h 1<br>
>> >> st_atim.tv_nsec 1<br>
>> >> package-string: glusterfs 3.7.3<br>
>> >><br>
>> >><br>
>> >> /usr/lib/x86_64-linux-gnu/<wbr>libglusterfs.so.0(_gf_msg_<wbr>backtrace_nomem+0x92)[<wbr>0x7f24de214502]<br>
>> >><br>
>> >><br>
>> >> /usr/lib/x86_64-linux-gnu/<wbr>libglusterfs.so.0(gf_print_<wbr>trace+0x31d)[0x7f24de23059d]<br>
>> >> /lib/x86_64-linux-gnu/libc.so.<wbr>6(+0x36d40)[0x7f24dd612d40]<br>
>> >> /lib/x86_64-linux-gnu/<wbr>libpthread.so.0(+0x11f6b)[<wbr>0x7f24dd9b2f6b]<br>
>> >><br>
>> >><br>
>> >> /usr/lib/x86_64-linux-gnu/<wbr>libglusterfs.so.0(inode_ref+<wbr>0x19)[0x7f24de234e69]<br>
>> >><br>
>> >> /usr/lib/x86_64-linux-gnu/<wbr>libglusterfs.so.0(loc_copy+<wbr>0x4a)[0x7f24de21291a]<br>
>> >><br>
>> >><br>
>> >> /usr/lib/x86_64-linux-gnu/<wbr>glusterfs/3.7.3/xlator/<wbr>cluster/distribute.so(dht_<wbr>local_init+0x4b)[<wbr>0x7f24d851f51b]<br>
>> >><br>
>> >><br>
>> >> /usr/lib/x86_64-linux-gnu/<wbr>glusterfs/3.7.3/xlator/<wbr>cluster/distribute.so(dht_<wbr>lookup+0x91)[0x7f24d8550521]<br>
>> >><br>
>> >><br>
>> >> /usr/lib/x86_64-linux-gnu/<wbr>libglusterfs.so.0(syncop_<wbr>lookup+0x1a2)[0x7f24de258fc2]<br>
>> >><br>
>> >><br>
>> >> /usr/lib/x86_64-linux-gnu/<wbr>glusterfs/3.7.3/xlator/<wbr>cluster/distribute.so(gf_<wbr>defrag_fix_layout+0x87)[<wbr>0x7f24d85289e7]<br>
>> >><br>
>> >><br>
>> >> /usr/lib/x86_64-linux-gnu/<wbr>glusterfs/3.7.3/xlator/<wbr>cluster/distribute.so(gf_<wbr>defrag_start_crawl+0x6d3)[<wbr>0x7f24d8529ce3]<br>
>> >><br>
>> >><br>
>> >> /usr/lib/x86_64-linux-gnu/<wbr>libglusterfs.so.0(synctask_<wbr>wrap+0x12)[0x7f24de255822]<br>
>> >> /lib/x86_64-linux-gnu/libc.so.<wbr>6(+0x498b0)[0x7f24dd6258b0]<br>
>> >><br>
>> >><br>
>> >> Anybody has a clue how to fix 'failed to get index' error?<br>
>> ><br>
>> ><br>
>> > Do you have the coredump? If yes, please send that and the rebalance<br>
>> > logs<br>
>> > across.<br>
>> ><br>
>> > Regards,<br>
>> > Nithya<br>
>> >><br>
>> >><br>
>> >> Thank you in advance!<br>
>> >> Szymon Miotk<br>
>> >> ______________________________<wbr>_________________<br>
>> >> Gluster-users mailing list<br>
>> >> <a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
>> >> <a href="http://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://lists.gluster.org/<wbr>mailman/listinfo/gluster-users</a><br>
>> ><br>
>> ><br>
><br>
><br>
</div></div></blockquote></div><br></div></div>