[Gluster-devel] Expanding Volumes

Atin Mukherjee atin.mukherjee83 at gmail.com
Tue Jun 9 13:24:37 UTC 2015 

Rebalance is broken in 3.7.1. It will be fixed in 3.7.2. Sorry for the
inconvenience.

Regards,
Atin

Sent from Samsung Galaxy S4
On 9 Jun 2015 18:34, "Jonhnny Weslley" <jw at jonhnnyweslley.net> wrote:

> Hi guys,
>
> I trying to create a pool of 4 nodes using centos7 and gluster 3.7 in a
> vagrant-based environment for test. First, I create and start a replicated
> volume using only 2 nodes (replication 2). After, I mount the volume using
> fuse and copy some files. Everything works fine.
>
> Then, I try to expand the volume previously created using the command:
>
> sudo gluster volume add-brick jged 10.10.50.73:/home/vagrant/brick
> 10.10.50.74:/home/vagrant/brick force
>
> And works again:
>
> sudo gluster volume info
>
> Volume Name: jged
> Type: Distributed-Replicate
> Volume ID: 862ab9b7-4753-4682-ba44-cbe481b1b7df
> Status: Started
> Number of Bricks: 2 x 2 = 4
> Transport-type: tcp
> Bricks:
> Brick1: 10.10.50.71:/home/vagrant/brick
> Brick2: 10.10.50.72:/home/vagrant/brick
> Brick3: 10.10.50.73:/home/vagrant/brick
> Brick4: 10.10.50.74:/home/vagrant/brick
> Options Reconfigured:
> performance.readdir-ahead: on
>
>
> But when I try to rebalance the volume (sudo gluster volume rebalance jged
> start), the gluster's  proccess in the node where the command was executed
> dies and dont start again after running 'systemctl start glusterd'. I look
> the log file (/var/log/glusterfs/etc-glusterfs-glusterd.vol.log) but I cant
> figure out what is wrong! :(
>
> Follow the tail of the log file:
>
> [2015-06-09 12:30:56.197802] I [MSGID: 100030] [glusterfsd.c:2294:main]
> 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.7.1
> (args: /usr/sbin/glusterd -p /var/run/glusterd.pid)
> [2015-06-09 12:30:56.207596] I [glusterd.c:1282:init] 0-management:
> Maximum allowed open file descriptors set to 65536
> [2015-06-09 12:30:56.207653] I [glusterd.c:1327:init] 0-management: Using
> /var/lib/glusterd as working directory
> [2015-06-09 12:30:56.211505] E [rpc-transport.c:291:rpc_transport_load]
> 0-rpc-transport: /usr/lib64/glusterfs/3.7.1/rpc-transport/rdma.so: cannot
> open shared object file: No such file or directory
> [2015-06-09 12:30:56.211521] W [rpc-transport.c:295:rpc_transport_load]
> 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not
> valid or not found on this machine
> [2015-06-09 12:30:56.211528] W [rpcsvc.c:1595:rpcsvc_transport_create]
> 0-rpc-service: cannot create listener, initing the transport failed
> [2015-06-09 12:30:56.211535] E [glusterd.c:1515:init] 0-management:
> creation of 1 listeners failed, continuing with succeeded transport
> [2015-06-09 12:30:56.213311] I
> [glusterd.c:413:glusterd_check_gsync_present] 0-glusterd: geo-replication
> module not installed in the system
> [2015-06-09 12:30:56.213454] I
> [glusterd-store.c:1986:glusterd_restore_op_version] 0-glusterd: retrieved
> op-version: 30700
> [2015-06-09 12:30:56.213523] I [glusterd.c:154:glusterd_uuid_init]
> 0-management: retrieved UUID: f264d968-5a14-459b-8f3b-569aa15c3ce2
> [2015-06-09 12:30:56.213568] I [rpc-clnt.c:972:rpc_clnt_connection_init]
> 0-glustershd: setting frame-timeout to 600
> [2015-06-09 12:30:56.213675] I [rpc-clnt.c:972:rpc_clnt_connection_init]
> 0-nfs: setting frame-timeout to 600
> [2015-06-09 12:30:56.213801] I [rpc-clnt.c:972:rpc_clnt_connection_init]
> 0-quotad: setting frame-timeout to 600
> [2015-06-09 12:30:56.213896] I [rpc-clnt.c:972:rpc_clnt_connection_init]
> 0-bitd: setting frame-timeout to 600
> [2015-06-09 12:30:56.213979] I [rpc-clnt.c:972:rpc_clnt_connection_init]
> 0-scrub: setting frame-timeout to 600
> [2015-06-09 12:30:56.214094] I [rpc-clnt.c:972:rpc_clnt_connection_init]
> 0-snapd: setting frame-timeout to 600
> [2015-06-09 12:30:56.987649] I
> [glusterd-handler.c:3387:glusterd_friend_add_from_peerinfo] 0-management:
> connect returned 0
> [2015-06-09 12:30:56.987711] I
> [glusterd-handler.c:3387:glusterd_friend_add_from_peerinfo] 0-management:
> connect returned 0
> [2015-06-09 12:30:56.987755] I
> [glusterd-handler.c:3387:glusterd_friend_add_from_peerinfo] 0-management:
> connect returned 0
> [2015-06-09 12:30:56.987801] I [rpc-clnt.c:972:rpc_clnt_connection_init]
> 0-management: setting frame-timeout to 600
> [2015-06-09 12:30:56.989874] W [socket.c:923:__socket_keepalive] 0-socket:
> failed to set TCP_USER_TIMEOUT -1000 on socket 13, Invalid argument
> [2015-06-09 12:30:56.989890] E [socket.c:3015:socket_connect]
> 0-management: Failed to set keep-alive: Invalid argument
> [2015-06-09 12:30:56.990051] I [rpc-clnt.c:972:rpc_clnt_connection_init]
> 0-management: setting frame-timeout to 600
> [2015-06-09 12:30:56.992360] W [socket.c:923:__socket_keepalive] 0-socket:
> failed to set TCP_USER_TIMEOUT -1000 on socket 14, Invalid argument
> [2015-06-09 12:30:56.992419] E [socket.c:3015:socket_connect]
> 0-management: Failed to set keep-alive: Invalid argument
> [2015-06-09 12:30:56.992629] I [rpc-clnt.c:972:rpc_clnt_connection_init]
> 0-management: setting frame-timeout to 600
> [2015-06-09 12:30:56.994163] W [socket.c:923:__socket_keepalive] 0-socket:
> failed to set TCP_USER_TIMEOUT -1000 on socket 15, Invalid argument
> [2015-06-09 12:30:56.994177] E [socket.c:3015:socket_connect]
> 0-management: Failed to set keep-alive: Invalid argument
> Final graph:
>
> +------------------------------------------------------------------------------+
>   1: volume management
>   2:     type mgmt/glusterd
>   3:     option rpc-auth.auth-glusterfs on
>   4:     option rpc-auth.auth-unix on
>   5:     option rpc-auth.auth-null on
>   6:     option transport.socket.listen-backlog 128
>   7:     option ping-timeout 30
>   8:     option transport.socket.read-fail-log off
>   9:     option transport.socket.keepalive-interval 2
>  10:     option transport.socket.keepalive-time 10
>  11:     option transport-type rdma
>  12:     option working-directory /var/lib/glusterd
>  13: end-volume
>  14:
>
> +------------------------------------------------------------------------------+
> [2015-06-09 12:30:57.007380] I
> [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread
> with index 1
> [2015-06-09 12:30:57.288783] I
> [glusterd-rpc-ops.c:464:__glusterd_friend_add_cbk] 0-glusterd: Received ACC
> from uuid: 41707504-c217-4b7a-8f3a-7150bd040d0a, host: 10.10.50.74, port: 0
> [2015-06-09 12:30:57.292480] I
> [glusterd-handler.c:2587:__glusterd_handle_friend_update] 0-glusterd:
> Received friend update from uuid: 41707504-c217-4b7a-8f3a-7150bd040d0a
> [2015-06-09 12:30:57.292542] I
> [glusterd-handler.c:2630:__glusterd_handle_friend_update] 0-management:
> Received my uuid as Friend
> [2015-06-09 12:30:57.293810] I [MSGID: 106132]
> [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already
> stopped
> [2015-06-09 12:30:57.303585] W [socket.c:3059:socket_connect] 0-nfs:
> Ignore failed connection attempt on , (No such file or directory)
> [2015-06-09 12:30:57.304172] I [MSGID: 106132]
> [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: glustershd
> already stopped
> [2015-06-09 12:30:57.324624] W [socket.c:3059:socket_connect]
> 0-glustershd: Ignore failed connection attempt on , (No such file or
> directory)
> [2015-06-09 12:30:57.324721] I [MSGID: 106132]
> [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: bitd already
> stopped
> [2015-06-09 12:30:57.324767] I [MSGID: 106132]
> [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: scrub already
> stopped
> [2015-06-09 12:30:57.343225] I [rpc-clnt.c:972:rpc_clnt_connection_init]
> 0-management: setting frame-timeout to 600
> pending frames:
> frame : type(0) op(0)
> frame : type(0) op(0)
> frame : type(0) op(0)
> patchset: git://git.gluster.com/glusterfs.git
> signal received: 6
> time of crash:
> 2015-06-09 12:30:57
> configuration details:
> argp 1
> backtrace 1
> dlfcn 1
> libpthread 1
> llistxattr 1
> setfsid 1
> spinlock 1
> epoll.h 1
> xattr.h 1
> st_atim.tv_nsec 1
> package-string: glusterfs 3.7.1
> /lib64/libglusterfs.so.0(_gf_msg_backtrace_nomem+0xb2)[0x7f3f46ca2d92]
> /lib64/libglusterfs.so.0(gf_print_trace+0x32d)[0x7f3f46cbd9ed]
> /lib64/libc.so.6(+0x35a00)[0x7f3f45696a00]
> /lib64/libc.so.6(gsignal+0x39)[0x7f3f45696989]
> /lib64/libc.so.6(abort+0x148)[0x7f3f45698098]
> /lib64/libc.so.6(+0x76197)[0x7f3f456d7197]
> /lib64/libc.so.6(__fortify_fail+0x37)[0x7f3f4576ecb7]
> /lib64/libc.so.6(+0x10be80)[0x7f3f4576ce80]
> /lib64/libc.so.6(+0x10b5bb)[0x7f3f4576c5bb]
> /lib64/libc.so.6(__snprintf_chk+0x78)[0x7f3f4576c4d8]
>
> /usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(glusterd_volume_defrag_restart+0x191)[0x7f3f3bb37931]
>
> /usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(glusterd_restart_rebalance+0x82)[0x7f3f3bb3daa2]
>
> /usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(glusterd_spawn_daemons+0x4f)[0x7f3f3bb3db1f]
> /lib64/libglusterfs.so.0(synctask_wrap+0x12)[0x7f3f46cdd482]
> /lib64/libc.so.6(+0x47570)[0x7f3f456a8570]
> ---------
>
>
> --
> Jonhnny Weslley
> http://blog.jonhnnyweslley.net
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20150609/a8bdfd05/attachment.html>

More information about the Gluster-devel mailing list