[Gluster-users] Rebalance task fails

[Nithya Balachandran]

On 7 July 2017 at 15:42, Szymon Miotk <szymon.miotk at gmail.com> wrote:

> Hello everyone,
>
>
> I have problem rebalancing Gluster volume.
> Gluster version is 3.7.3.
> My 1x3 replicated volume become full, so I've added three more bricks
> to make it 2x3 and wanted to rebalance.
> But every time I start rebalancing, it fails immediately.
> Rebooting Gluster nodes doesn't help.
>
> # gluster volume rebalance  gsae_artifactory_cluster_storage start
> volume rebalance: gsae_artifactory_cluster_storage: success: Rebalance
> on gsae_artifactory_cluster_storage has been started successfully. Use
> rebalance status command to check status of the rebalance process.
> ID: b22572ff-7575-4557-8317-765f7e52d445
>
> # gluster volume rebalance  gsae_artifactory_cluster_storage status
>                                     Node Rebalanced-files
> size       scanned      failures       skipped               status
> run time in secs
>                                ---------      -----------
> -----------   -----------   -----------   -----------
> ------------     --------------
>                                localhost                0
> 0Bytes             0             0             0               failed
>              0.00
>                              10.239.40.9                0
> 0Bytes             0             0             0               failed
>              0.00
>                              10.239.40.8                0
> 0Bytes             0             0             0               failed
>              0.00
> volume rebalance: gsae_artifactory_cluster_storage: success:
>
> The messages in logfiles mention 'failed to get index':
> [2017-07-07 10:07:18.230202] E [MSGID: 106062]
> [glusterd-utils.c:7997:glusterd_volume_rebalance_use_rsp_dict]
> 0-glusterd: failed to get index
>
> and then the rebalance process crashes:
> pending frames:
> frame : type(0) op(0)
> frame : type(0) op(0)
> frame : type(0) op(0)
> patchset: git://git.gluster.com/glusterfs.git
> signal received: 4
> time of crash:
> 2017-07-07 10:07:23
> configuration details:
> argp 1
> backtrace 1
> dlfcn 1
> libpthread 1
> llistxattr 1
> setfsid 1
> spinlock 1
> epoll.h 1
> xattr.h 1
> st_atim.tv_nsec 1
> package-string: glusterfs 3.7.3
> /usr/lib/x86_64-linux-gnu/libglusterfs.so.0(_gf_msg_backtrace_nomem+0x92)[
> 0x7f24de214502]
> /usr/lib/x86_64-linux-gnu/libglusterfs.so.0(gf_print_
> trace+0x31d)[0x7f24de23059d]
> /lib/x86_64-linux-gnu/libc.so.6(+0x36d40)[0x7f24dd612d40]
> /lib/x86_64-linux-gnu/libpthread.so.0(+0x11f6b)[0x7f24dd9b2f6b]
> /usr/lib/x86_64-linux-gnu/libglusterfs.so.0(inode_ref+
> 0x19)[0x7f24de234e69]
> /usr/lib/x86_64-linux-gnu/libglusterfs.so.0(loc_copy+0x4a)[0x7f24de21291a]
> /usr/lib/x86_64-linux-gnu/glusterfs/3.7.3/xlator/
> cluster/distribute.so(dht_local_init+0x4b)[0x7f24d851f51b]
> /usr/lib/x86_64-linux-gnu/glusterfs/3.7.3/xlator/
> cluster/distribute.so(dht_lookup+0x91)[0x7f24d8550521]
> /usr/lib/x86_64-linux-gnu/libglusterfs.so.0(syncop_
> lookup+0x1a2)[0x7f24de258fc2]
> /usr/lib/x86_64-linux-gnu/glusterfs/3.7.3/xlator/cluster/distribute.so(gf_
> defrag_fix_layout+0x87)[0x7f24d85289e7]
> /usr/lib/x86_64-linux-gnu/glusterfs/3.7.3/xlator/cluster/distribute.so(gf_
> defrag_start_crawl+0x6d3)[0x7f24d8529ce3]
> /usr/lib/x86_64-linux-gnu/libglusterfs.so.0(synctask_
> wrap+0x12)[0x7f24de255822]
> /lib/x86_64-linux-gnu/libc.so.6(+0x498b0)[0x7f24dd6258b0]
>
>
> Anybody has a clue how to fix 'failed to get index' error?
>

Do you have the coredump? If yes, please send that and the rebalance logs
across.

Regards,
Nithya

>
> Thank you in advance!
> Szymon Miotk
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170709/455cd95d/attachment.html>