[Bugs] [Bug 1221534] rebalance failed after attaching the tier to the volume.

bugzilla at redhat.com bugzilla at redhat.com
Fri May 15 09:37:19 UTC 2015


https://bugzilla.redhat.com/show_bug.cgi?id=1221534



--- Comment #1 from Triveni Rao <trao at redhat.com> ---


This was reproducible on new downstream build also:

[root at rhsqa14-vm1 ~]# rpm -qa | grep gluster
glusterfs-3.7.0-2.el6rhs.x86_64
glusterfs-cli-3.7.0-2.el6rhs.x86_64
glusterfs-libs-3.7.0-2.el6rhs.x86_64
glusterfs-client-xlators-3.7.0-2.el6rhs.x86_64
glusterfs-api-3.7.0-2.el6rhs.x86_64
glusterfs-server-3.7.0-2.el6rhs.x86_64
glusterfs-fuse-3.7.0-2.el6rhs.x86_64
[root at rhsqa14-vm1 ~]# 


[root at rhsqa14-vm1 ~]# glusterfs --version
glusterfs 3.7.0 built on May 15 2015 01:31:10
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2013 Red Hat, Inc. <http://www.redhat.com/>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
It is licensed to you under your choice of the GNU Lesser
General Public License, version 3 or any later version (LGPLv3
or later), or the GNU General Public License, version 2 (GPLv2),
in all cases as published by the Free Software Foundation.
[root at rhsqa14-vm1 ~]# 


[root at rhsqa14-vm1 ~]# gluster v attach-tier vol1 replica 2
10.70.46.233:/rhs/brick3/m1 10.70.46.236:/rhs/brick3/m1
10.70.46.233:/rhs/brick5/m1 10.70.4
6.236:/rhs/brick5/m1  
Attach tier is recommended only for testing purposes in this release. Do you
want to continue? (y/n) y
volume attach-tier: success
volume rebalance: vol1: success: Rebalance on vol1 has been started
successfully. Use rebalance status command to check status of the rebalance
proces
s.
ID: b954b5e0-c4fa-4619-92ca-3c5e657269aa

You have new mail in /var/spool/mail/root
[root at rhsqa14-vm1 ~]# 
[root at rhsqa14-vm1 ~]# gluster v rebalance vol1  status
                                    Node Rebalanced-files          size      
scanned      failures       skipped               status   run time in s
ecs
                               ---------      -----------   -----------  
-----------   -----------   -----------         ------------     -----------
---
                               localhost                0        0Bytes        
    0             0             0               failed               0
.00
                            10.70.46.236                0        0Bytes        
    0             0             0               failed               0
.00
volume rebalance: vol1: success:
[root at rhsqa14-vm1 ~]# 
[root at rhsqa14-vm1 ~]# 
[root at rhsqa14-vm1 ~]# 



[root at rhsqa14-vm1 ~]# less /var/log/glusterfs/vol1-rebalance.log 
[2015-05-15 08:55:31.691592] I [MSGID: 100030] [glusterfsd.c:2294:main]
0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.0 (args:
/usr/sbin/glusterfs -s localhost --volfile-id rebalance/vol1 --xlator-option
*dht.use-readdirp=yes --xlator-option *dht.lookup-unhashed=yes --xlator-option
*dht.assert-no-child-down=yes --xlator-option *replicate*.data-self-heal=off
--xlator-option *replicate*.metadata-self-heal=off --xlator-option
*replicate*.entry-self-heal=off --xlator-option
*replicate*.readdir-failover=off --xlator-option *dht.readdir-optimize=on
--xlator-option *tier-dht.xattr-name=trusted.tier-gfid --xlator-option
*dht.rebalance-cmd=6 --xlator-option
*dht.node-uuid=87acbf29-e821-48bf-9aa8-bbda9321e609 --xlator-option
*dht.commit-hash=2863506458 --socket-file
/var/run/gluster/gluster-rebalance-37d0a9c0-21c1-46cf-ba95-419f9fbfbab0.sock
--pid-file
/var/lib/glusterd/vols/vol1/rebalance/87acbf29-e821-48bf-9aa8-bbda9321e609.pid
-l /var/log/glusterfs/vol1-rebalance.log)
[2015-05-15 08:55:31.730515] I [event-epoll.c:629:event_dispatch_epoll_worker]
0-epoll: Started thread with index 1
[2015-05-15 08:55:36.690982] I [graph.c:269:gf_add_cmdline_options] 0-tier-dht:
adding option 'commit-hash' for volume 'tier-dht' with value '2863506458'
[2015-05-15 08:55:36.691003] I [graph.c:269:gf_add_cmdline_options] 0-tier-dht:
adding option 'node-uuid' for volume 'tier-dht' with value
'87acbf29-e821-48bf-9aa8-bbda9321e609'
[2015-05-15 08:55:36.691014] I [graph.c:269:gf_add_cmdline_options] 0-tier-dht:
adding option 'rebalance-cmd' for volume 'tier-dht' with value '6'
[2015-05-15 08:55:36.691025] I [graph.c:269:gf_add_cmdline_options] 0-tier-dht:
adding option 'xattr-name' for volume 'tier-dht' with value 'trusted.tier-gfid'
[2015-05-15 08:55:36.691035] I [graph.c:269:gf_add_cmdline_options] 0-tier-dht:
adding option 'readdir-optimize' for volume 'tier-dht' with value 'on'
[2015-05-15 08:55:36.691046] I [graph.c:269:gf_add_cmdline_options] 0-tier-dht:
adding option 'assert-no-child-down' for volume 'tier-dht' with value 'yes'
[2015-05-15 08:55:36.691056] I [graph.c:269:gf_add_cmdline_options] 0-tier-dht:
adding option 'lookup-unhashed' for volume 'tier-dht' with value 'yes'
[2015-05-15 08:55:36.691069] I [graph.c:269:gf_add_cmdline_options] 0-tier-dht:
adding option 'use-readdirp' for volume 'tier-dht' with value 'yes'
[2015-05-15 08:55:36.691080] I [graph.c:269:gf_add_cmdline_options]
0-vol1-hot-dht: adding option 'commit-hash' for volume 'vol1-hot-dht' with
value '2863506458'
[2015-05-15 08:55:36.692412] I [graph.c:269:gf_add_cmdline_options]
0-vol1-hot-dht: adding option 'node-uuid' for volume 'vol1-hot-dht' with value
'87acbf29-e821-48bf-9aa8-bbda9321e609'
[2015-05-15 08:55:36.692425] I [graph.c:269:gf_add_cmdline_options]
0-vol1-hot-dht: adding option 'rebalance-cmd' for volume 'vol1-hot-dht' with
value '6'
[2015-05-15 08:55:36.692437] I [graph.c:269:gf_add_cmdline_options]
0-vol1-hot-dht: adding option 'readdir-optimize' for volume 'vol1-hot-dht' with
value 'on'
[2015-05-15 08:55:36.692449] I [graph.c:269:gf_add_cmdline_options]
0-vol1-hot-dht: adding option 'assert-no-child-down' for volume 'vol1-hot-dht'
with value 'yes'
[2015-05-15 08:55:36.692459] I [graph.c:269:gf_add_cmdline_options]
0-vol1-hot-dht: adding option 'lookup-unhashed' for volume 'vol1-hot-dht' with
val...skipping...
ith value 'yes'
[2015-05-15 08:57:58.911456] I [graph.c:269:gf_add_cmdline_options]
0-vol1-cold-dht: adding option 'lookup-unhashed' for volume 'vol1-cold-dht'
with value 'yes'
[2015-05-15 08:57:58.911466] I [graph.c:269:gf_add_cmdline_options]
0-vol1-cold-dht: adding option 'use-readdirp' for volume 'vol1-cold-dht' with
value 'yes'
[2015-05-15 08:57:58.911477] I [graph.c:269:gf_add_cmdline_options]
0-vol1-cold-replicate-1: adding option 'readdir-failover' for volume
'vol1-cold-replicate-1' with value 'off'
[2015-05-15 08:57:58.911486] I [graph.c:269:gf_add_cmdline_options]
0-vol1-cold-replicate-1: adding option 'entry-self-heal' for volume
'vol1-cold-replicate-1' with value 'off'
[2015-05-15 08:57:58.911496] I [graph.c:269:gf_add_cmdline_options]
0-vol1-cold-replicate-1: adding option 'metadata-self-heal' for volume
'vol1-cold-replicate-1' with value 'off'
[2015-05-15 08:57:58.911505] I [graph.c:269:gf_add_cmdline_options]
0-vol1-cold-replicate-1: adding option 'data-self-heal' for volume
'vol1-cold-replicate-1' with value 'off'
[2015-05-15 08:57:58.911517] I [graph.c:269:gf_add_cmdline_options]
0-vol1-cold-replicate-0: adding option 'readdir-failover' for volume
'vol1-cold-replicate-0' with value 'off'
[2015-05-15 08:57:58.911527] I [graph.c:269:gf_add_cmdline_options]
0-vol1-cold-replicate-0: adding option 'entry-self-heal' for volume
'vol1-cold-replicate-0' with value 'off'
[2015-05-15 08:57:58.911536] I [graph.c:269:gf_add_cmdline_options]
0-vol1-cold-replicate-0: adding option 'metadata-self-heal' for volume
'vol1-cold-replicate-0' with value 'off'
[2015-05-15 08:57:58.911545] I [graph.c:269:gf_add_cmdline_options]
0-vol1-cold-replicate-0: adding option 'data-self-heal' for volume
'vol1-cold-replicate-0' with value 'off'
[2015-05-15 08:57:58.912392] I [dht-shared.c:598:dht_init] 0-tier-dht: dht_init
using commit hash 2863507596
[2015-05-15 08:57:58.912452] I
[dht-shared.c:299:dht_parse_decommissioned_bricks] 0-tier-dht: decommissioning
subvolume vol1-hot-dht
[2015-05-15 08:57:58.913862] E [MSGID: 109037]
[tier.c:1007:tier_load_externals] 0-tier-dht: Error loading libgfdb.so
/usr/lib64/libgfdb.so: cannot open shared object file: No such file or
directory

[2015-05-15 08:57:58.913899] E [MSGID: 109037] [tier.c:1070:tier_init]
0-tier-dht: Could not load externals. Aborting
[2015-05-15 08:57:58.913946] E [xlator.c:426:xlator_init] 0-tier-dht:
Initialization of volume 'tier-dht' failed, review your volfile again
[2015-05-15 08:57:58.913959] E [graph.c:322:glusterfs_graph_init] 0-tier-dht:
initializing translator failed
[2015-05-15 08:57:58.913967] E [graph.c:661:glusterfs_graph_activate] 0-graph:
init failed
[2015-05-15 08:57:58.914373] W [glusterfsd.c:1219:cleanup_and_exit] (--> 0-:
received signum (0), shutting down
(END)

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are on the CC list for the bug.
You are the assignee for the bug.


More information about the Bugs mailing list