[Bugs] [Bug 1637743] Glusterd seems to be attempting to start the same brick process twice

Tue Oct 9 22:36:13 UTC 2018

https://bugzilla.redhat.com/show_bug.cgi?id=1637743

--- Comment #1 from Shyamsundar <srangana at redhat.com> ---
### Log analysis ###
###### Logs pointing to changelog initialization failure ######
[2018-10-06 18:20:11.032759] E [socket.c:905:__socket_server_bind]
0-socket.0627e719755f46f294fee9f266f72d60-changelog: binding to  failed:
Address already in use
[2018-10-06 18:20:11.032784] E [socket.c:907:__socket_server_bind]
0-socket.0627e719755f46f294fee9f266f72d60-changelog: Port is already in use
[2018-10-06 18:20:11.032804] W [rpcsvc.c:1795:rpcsvc_create_listener]
0-rpc-service: listening on transport failed

(other instance)
[2018-10-05 16:57:15.646645] E [socket.c:905:__socket_server_bind]
0-socket.patchy-changelog: binding to  failed: Address already in use
[2018-10-05 16:57:15.646667] E [socket.c:907:__socket_server_bind]
0-socket.patchy-changelog: Port is already in use
[2018-10-05 16:57:15.646665] I [MSGID: 121050] [changetimerecorder.c:2145:init]
0-gfdbdatastore: CTR Xlator is not enabled so skip ctr init
[2018-10-05 16:57:15.646686] W [rpcsvc.c:1795:rpcsvc_create_listener]
0-rpc-service: listening on transport failed

###### Startup sequence logs ######
[2018-10-05 16:57:12.383081] W [glusterfsd.c:1481:cleanup_and_exit]
(-->/lib64/libpthread.so.0(+0x7e25) [0x7fa7d12bbe25]
-->/build/install/sbin/glusterfsd(glusterfs_sigwaiter+0xe6) [0x40a7da]
-->/build/install/sbin/glusterfsd(cleanup_and_exit+0x88) [0x408b07] ) 0-:
received signum (15), shutting down

[2018-10-05 16:57:13.395291]:++++++++++
G_LOG:./tests/bugs/glusterd/quorum-validation.t: TEST: 95 0
cluster_brick_up_status 1 patchy 127.1.1.1 /d/backends/1/patchy0 ++++++++++

[2018-10-05 16:57:13.497347]:++++++++++
G_LOG:./tests/bugs/glusterd/quorum-validation.t: TEST: 97 glusterd
--xlator-option management.working-directory=/d/backends/2/glusterd
--xlator-option management.transport.socket.bind-address=127.1.1.2
--xlator-option management.run-directory=/d/backends/2/run/gluster
--xlator-option management.glusterd-sockfile=/d/backends/2/glusterd/gd.sock
--log-file=/var/log/glusterfs/quorum-validation.t_glusterd2.log
--pid-file=/d/backends/2/glusterd.pid ++++++++++

[2018-10-05 16:57:15.447758]:++++++++++
G_LOG:./tests/bugs/glusterd/quorum-validation.t: TEST: 98 1
cluster_brick_up_status 1 patchy 127.1.1.1 /d/backends/1/patchy0 ++++++++++

<CLI instance 1>
[2018-10-05 16:57:15.544023] I [MSGID: 100030] [glusterfsd.c:2691:main]
0-/build/install/sbin/glusterfsd: Started running
/build/install/sbin/glusterfsd version 6dev (args:
/build/install/sbin/glusterfsd -s 127.1.1.1 --volfile-id
patchy.127.1.1.1.d-backends-1-patchy0 -p
/d/backends/1/run/gluster/vols/patchy/127.1.1.1-d-backends-1-patchy0.pid -S
/var/run/gluster/f96de9a0bebaefd6.socket --brick-name /d/backends/1/patchy0 -l
/var/log/glusterfs/bricks/d-backends-1-patchy0.log --xlator-option
*-posix.glusterd-uuid=c8779063-a544-453b-925f-6eaf388bb107 --process-name brick
--brick-port 49154 --xlator-option patchy-server.listen-port=49154
--xlator-option transport.socket.bind-address=127.1.1.1)

[2018-10-05 16:57:15.552571] W [MSGID: 101002] [options.c:975:xl_opt_validate]
0-glusterfs: option 'address-family' is deprecated, preferred is
'transport.address-family', continuing with correction

[2018-10-05 16:57:15.552961] I [dict.c:1178:data_to_uint16]
(-->/build/install/lib/glusterfs/6dev/rpc-transport/socket.so(socket_client_get_remote_sockaddr+0x1dc)
[0x7ff8ab57025a]
-->/build/install/lib/glusterfs/6dev/rpc-transport/socket.so(+0x12773)
[0x7ff8ab56f773] -->/build/install/lib/libglusterfs.so.0(data_to_uint16+0x11a)
[0x7ff8b698adfb] ) 0-dict: key null, unsigned integer type asked, has integer
type [Invalid argument]

[2018-10-05 16:57:15.557053] I [MSGID: 101190]
[event-epoll.c:622:event_dispatch_epoll_worker] 0-epoll: Started thread with
index 1

<CLI instance 2>
[2018-10-05 16:57:15.599487] I [MSGID: 100030] [glusterfsd.c:2691:main]
0-/build/install/sbin/glusterfsd: Started running
/build/install/sbin/glusterfsd version 6dev (args:
/build/install/sbin/glusterfsd -s 127.1.1.1 --volfile-id
patchy.127.1.1.1.d-backends-1-patchy0 -p
/d/backends/1/run/gluster/vols/patchy/127.1.1.1-d-backends-1-patchy0.pid -S
/var/run/gluster/f96de9a0bebaefd6.socket --brick-name /d/backends/1/patchy0 -l
/var/log/glusterfs/bricks/d-backends-1-patchy0.log --xlator-option
*-posix.glusterd-uuid=c8779063-a544-453b-925f-6eaf388bb107 --process-name brick
--brick-port 49155 --xlator-option patchy-server.listen-port=49155
--xlator-option transport.socket.bind-address=127.1.1.1)

[2018-10-05 16:57:15.607454] W [MSGID: 101002] [options.c:975:xl_opt_validate]
0-glusterfs: option 'address-family' is deprecated, preferred is
'transport.address-family', continuing with correction

[2018-10-05 16:57:15.607880] I [dict.c:1178:data_to_uint16]
(-->/build/install/lib/glusterfs/6dev/rpc-transport/socket.so(socket_client_get_remote_sockaddr+0x1dc)
[0x7f26f7afc25a]
-->/build/install/lib/glusterfs/6dev/rpc-transport/socket.so(+0x12773)
[0x7f26f7afb773] -->/build/install/lib/libglusterfs.so.0(data_to_uint16+0x11a)
[0x7f2702f16dfb] ) 0-dict: key null, unsigned integer type asked, has integer
type [Invalid argument]

[2018-10-05 16:57:15.612344] I [MSGID: 101190]
[event-epoll.c:622:event_dispatch_epoll_worker] 0-epoll: Started thread with
index 1

<NOTE: Double dlsym of various xlator shared libraries>
[2018-10-05 16:57:15.639477] I [MSGID: 101097]
[xlator.c:341:xlator_dynload_newway] 0-xlator: dlsym(xlator_api) on
/build/install/lib/glusterfs/6dev/xlator/storage/posix.so: undefined symbol:
xlator_api. Fall back to old symbols
[2018-10-05 16:57:15.639665] I [MSGID: 101097]
[xlator.c:341:xlator_dynload_newway] 0-xlator: dlsym(xlator_api) on
/build/install/lib/glusterfs/6dev/xlator/features/trash.so: undefined symbol:
xlator_api. Fall back to old symbols
[2018-10-05 16:57:15.639835] I [MSGID: 101097]
[xlator.c:341:xlator_dynload_newway] 0-xlator: dlsym(xlator_api) on
/build/install/lib/glusterfs/6dev/xlator/storage/posix.so: undefined symbol:
xlator_api. Fall back to old symbols
[2018-10-05 16:57:15.640019] I [MSGID: 101097]
[xlator.c:341:xlator_dynload_newway] 0-xlator: dlsym(xlator_api) on
/build/install/lib/glusterfs/6dev/xlator/features/trash.so: undefined symbol:
xlator_api. Fall back to old symbols

-- 
You are receiving this mail because:
You are on the CC list for the bug.
You are the assignee for the bug.