[Gluster-users] Glusterd fails to start after removing /brick1/gv2/.glusterfs

Fri Apr 24 18:34:28 UTC 2015

Note: this was fixed by removing the files in /var/lib/glusterd like 
this:

for file in /var/lib/glusterd/*; do if ! echo $file | grep 'hooks' 
 >/dev/null 2>&1;then rm -rf $file; fi; done

I could swear I'd done this once before during the reinstall, but 
whatever.

On 2015-04-24 11:29, Ernie Dunbar wrote:
> I've nuked my gluster brick, configuration, and files with the intent
> to rebuild them. This is what happens when I try to start glusterd
> with the --debug option:
> 
> root at nfs1:/etc/apt/sources.list.d# glusterd --debug
> [2015-04-24 18:26:07.978598] I [MSGID: 100030]
> [glusterfsd.c:2018:main] 0-glusterd: Started running glusterd version
> 3.6.2 (args: glusterd --debug)
> [2015-04-24 18:26:07.978685] D
> [logging.c:1763:__gf_log_inject_timer_event] 0-logging-infra: Starting
> timer now. Timeout = 120, current buf size = 5
> [2015-04-24 18:26:07.979021] D [MSGID: 0] [glusterfsd.c:613:get_volfp]
> 0-glusterfsd: loading volume file /etc/glusterfs/glusterd.vol
> [2015-04-24 18:26:07.982180] I [glusterd.c:1214:init] 0-management:
> Maximum allowed open file descriptors set to 65536
> [2015-04-24 18:26:07.982238] I [glusterd.c:1259:init] 0-management:
> Using /var/lib/glusterd as working directory
> [2015-04-24 18:26:07.982320] D
> [glusterd.c:391:glusterd_rpcsvc_options_build] 0-: listen-backlog
> value: 128
> [2015-04-24 18:26:07.982532] D [rpcsvc.c:2198:rpcsvc_init]
> 0-rpc-service: RPC service inited.
> [2015-04-24 18:26:07.982559] D [rpcsvc.c:1801:rpcsvc_program_register]
> 0-rpc-service: New program registered: GF-DUMP, Num: 123451501, Ver:
> 1, Port: 0
> [2015-04-24 18:26:07.982595] D
> [rpc-transport.c:262:rpc_transport_load] 0-rpc-transport: attempt to
> load file
> /usr/lib/x86_64-linux-gnu/glusterfs/3.6.2/rpc-transport/socket.so
> [2015-04-24 18:26:07.983291] D [socket.c:3799:socket_init]
> 0-socket.management: SSL support on the I/O path is NOT enabled
> [2015-04-24 18:26:07.983325] D [socket.c:3802:socket_init]
> 0-socket.management: SSL support for glusterd is NOT enabled
> [2015-04-24 18:26:07.983345] D [socket.c:3819:socket_init]
> 0-socket.management: using system polling thread
> [2015-04-24 18:26:07.983369] D [name.c:550:server_fill_address_family]
> 0-socket.management: option address-family not specified, defaulting
> to inet
> [2015-04-24 18:26:07.983457] D
> [rpc-transport.c:262:rpc_transport_load] 0-rpc-transport: attempt to
> load file
> /usr/lib/x86_64-linux-gnu/glusterfs/3.6.2/rpc-transport/rdma.so
> [2015-04-24 18:26:07.983895] D
> [rpc-transport.c:300:rpc_transport_load] 0-rpc-transport: dlsym
> (gf_rpc_transport_reconfigure) on
> /usr/lib/x86_64-linux-gnu/glusterfs/3.6.2/rpc-transport/rdma.so:
> undefined symbol: reconfigure
> librdmacm: couldn't read ABI version.
> librdmacm: assuming: 4
> CMA: unable to get RDMA device list
> [2015-04-24 18:26:07.984165] W [rdma.c:4221:__gf_rdma_ctx_create]
> 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such
> device)
> [2015-04-24 18:26:07.984191] E [rdma.c:4519:init] 0-rdma.management:
> Failed to initialize IB Device
> [2015-04-24 18:26:07.984215] E
> [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma'
> initialization failed
> [2015-04-24 18:26:07.984280] W [rpcsvc.c:1524:rpcsvc_transport_create]
> 0-rpc-service: cannot create listener, initing the transport failed
> [2015-04-24 18:26:07.984310] D [rpcsvc.c:1801:rpcsvc_program_register]
> 0-rpc-service: New program registered: GlusterD svc peer, Num:
> 1238437, Ver: 2, Port: 0
> [2015-04-24 18:26:07.984330] D [rpcsvc.c:1801:rpcsvc_program_register]
> 0-rpc-service: New program registered: GlusterD svc cli read-only,
> Num: 1238463, Ver: 2, Port: 0
> [2015-04-24 18:26:07.984351] D [rpcsvc.c:1801:rpcsvc_program_register]
> 0-rpc-service: New program registered: GlusterD svc mgmt, Num:
> 1238433, Ver: 2, Port: 0
> [2015-04-24 18:26:07.984378] D [rpcsvc.c:1801:rpcsvc_program_register]
> 0-rpc-service: New program registered: GlusterD svc mgmt v3, Num:
> 1238433, Ver: 3, Port: 0
> [2015-04-24 18:26:07.984398] D [rpcsvc.c:1801:rpcsvc_program_register]
> 0-rpc-service: New program registered: Gluster Portmap, Num: 34123456,
> Ver: 1, Port: 0
> [2015-04-24 18:26:07.984418] D [rpcsvc.c:1801:rpcsvc_program_register]
> 0-rpc-service: New program registered: Gluster Handshake, Num:
> 14398633, Ver: 2, Port: 0
> [2015-04-24 18:26:07.984440] D [rpcsvc.c:1801:rpcsvc_program_register]
> 0-rpc-service: New program registered: Gluster MGMT Handshake, Num:
> 1239873, Ver: 1, Port: 0
> [2015-04-24 18:26:07.984495] D [rpcsvc.c:2198:rpcsvc_init]
> 0-rpc-service: RPC service inited.
> [2015-04-24 18:26:07.984518] D [rpcsvc.c:1801:rpcsvc_program_register]
> 0-rpc-service: New program registered: GF-DUMP, Num: 123451501, Ver:
> 1, Port: 0
> [2015-04-24 18:26:07.984544] D
> [rpc-transport.c:262:rpc_transport_load] 0-rpc-transport: attempt to
> load file
> /usr/lib/x86_64-linux-gnu/glusterfs/3.6.2/rpc-transport/socket.so
> [2015-04-24 18:26:07.984629] D [socket.c:3684:socket_init]
> 0-socket.management: disabling nodelay
> [2015-04-24 18:26:07.984657] D [socket.c:3799:socket_init]
> 0-socket.management: SSL support on the I/O path is NOT enabled
> [2015-04-24 18:26:07.984676] D [socket.c:3802:socket_init]
> 0-socket.management: SSL support for glusterd is NOT enabled
> [2015-04-24 18:26:07.984696] D [socket.c:3819:socket_init]
> 0-socket.management: using system polling thread
> [2015-04-24 18:26:07.984788] D [rpcsvc.c:1801:rpcsvc_program_register]
> 0-rpc-service: New program registered: GlusterD svc cli, Num: 1238463,
> Ver: 2, Port: 0
> [2015-04-24 18:26:07.984818] D [rpcsvc.c:1801:rpcsvc_program_register]
> 0-rpc-service: New program registered: Gluster Handshake (CLI
> Getspec), Num: 14398633, Ver: 2, Port: 0
> [2015-04-24 18:26:07.984865] D
> [glusterd-utils.c:8288:glusterd_sm_tr_log_init] 0-: returning 0
> [2015-04-24 18:26:07.984893] D [glusterd.c:1484:init] 0-management:
> cannot get run-with-valgrind value
> [2015-04-24 18:26:08.063588] D
> [glusterd.c:446:glusterd_check_gsync_present] 0-glusterd: Returning 0
> [2015-04-24 18:26:08.063710] D
> [glusterd.c:572:glusterd_crt_georep_folders] 0-: Returning 0
> [2015-04-24 18:26:10.094132] E [store.c:432:gf_store_handle_retrieve]
> 0-: Path corresponding to /var/lib/glusterd/glusterd.info, returned
> error: (No such file or directory)
> [2015-04-24 18:26:10.094197] D [store.c:437:gf_store_handle_retrieve]
> 0-: Returning -1
> [2015-04-24 18:26:10.094221] D
> [glusterd-store.c:1935:glusterd_retrieve_op_version] 0-: Unable to get
> store handle!
> [2015-04-24 18:26:10.094251] E [store.c:432:gf_store_handle_retrieve]
> 0-: Path corresponding to /var/lib/glusterd/glusterd.info, returned
> error: (No such file or directory)
> [2015-04-24 18:26:10.094272] D [store.c:437:gf_store_handle_retrieve]
> 0-: Returning -1
> [2015-04-24 18:26:10.094290] D
> [glusterd-store.c:2093:glusterd_retrieve_uuid] 0-: Unable to get
> storehandle!
> [2015-04-24 18:26:10.094308] D
> [glusterd-store.c:2112:glusterd_retrieve_uuid] 0-: Returning -1
> [2015-04-24 18:26:10.094325] I
> [glusterd-store.c:2063:glusterd_restore_op_version] 0-management:
> Detected new install. Setting op-version to maximum : 30600
> [2015-04-24 18:26:10.094409] D
> [glusterd-utils.c:513:glusterd_volinfo_new] 0-management: Returning 0
> [2015-04-24 18:26:10.094448] E [store.c:432:gf_store_handle_retrieve]
> 0-: Path corresponding to /var/lib/glusterd/vols/gv2/info, returned
> error: (No such file or directory)
> [2015-04-24 18:26:10.094470] D [store.c:437:gf_store_handle_retrieve]
> 0-: Returning -1
> [2015-04-24 18:26:10.094488] E
> [glusterd-store.c:2623:glusterd_store_update_volinfo] 0-management:
> volinfo handle is NULL
> [2015-04-24 18:26:10.094506] E
> [glusterd-store.c:2868:glusterd_store_retrieve_volume] 0-management:
> Failed to update volinfo for gv2 volume
> [2015-04-24 18:26:10.094525] D
> [glusterd-utils.c:950:glusterd_volume_brickinfos_delete] 0-management:
> Returning 0
> [2015-04-24 18:26:10.094546] D [store.c:458:gf_store_handle_destroy]
> 0-: Returning 0
> [2015-04-24 18:26:10.094565] D
> [glusterd-utils.c:994:glusterd_volinfo_delete] 0-management: Returning
> 0
> [2015-04-24 18:26:10.094590] E
> [glusterd-store.c:3062:glusterd_store_retrieve_volumes] 0-: Unable to
> restore volume: gv2
> [2015-04-24 18:26:10.094612] D
> [glusterd-store.c:3094:glusterd_store_retrieve_volumes] 0-: Returning
> with -1
> [2015-04-24 18:26:10.094634] D
> [glusterd-store.c:4321:glusterd_restore] 0-: Returning -1
> [2015-04-24 18:26:10.094659] E [xlator.c:425:xlator_init]
> 0-management: Initialization of volume 'management' failed, review
> your volfile again
> [2015-04-24 18:26:10.094683] E [graph.c:322:glusterfs_graph_init]
> 0-management: initializing translator failed
> [2015-04-24 18:26:10.094702] E [graph.c:525:glusterfs_graph_activate]
> 0-graph: init failed
> [2015-04-24 18:26:10.095143] D
> [logging.c:1740:gf_log_flush_extra_msgs] 0-logging-infra: Log buffer
> size reduced. About to flush 2 extra log messages
> [2015-04-24 18:26:10.095176] D
> [logging.c:1743:gf_log_flush_extra_msgs] 0-logging-infra: Just flushed
> 2 extra log messages
> [2015-04-24 18:26:10.095395] W [glusterfsd.c:1194:cleanup_and_exit]
> (--> 0-: received signum (0), shutting down
> [2015-04-24 18:26:10.095426] D
> [glusterfsd-mgmt.c:2244:glusterfs_mgmt_pmap_signout] 0-fsd-mgmt:
> portmapper signout arguments not given
> 
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users