<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p><tt><font size="+1">Hi,</font></tt></p>
<p><tt><font size="+1">I'm trying to host several GlusterFS versions
on the same machine (3.8.8, 4.1.6 and 5.2), not to be ran
together of course.</font></tt></p>
<p><tt><font size="+1">I built them with the following procedure
(examples with 3.8.8):</font></tt></p>
<p><tt><font size="+1">
<blockquote type="cite"><tt><font size="+1">git clone
<a class="moz-txt-link-freetext" href="https://github.com/gluster/glusterfs">https://github.com/gluster/glusterfs</a> .<br>
git checkout v3.8.8<br>
./autogen<br>
./configure --program-suffix="-3.8.8"<br>
make<br>
sudo make install<br>
sudo cp -a extras/systemd/glusterd.service
/etc/systemd/system/glusterd-3.8.8.service<br>
sudo systemctl load glusterd-3.8.8<br>
</font></tt></blockquote>
I had to edit the service for it to execute the right version
of glusterd:</font></tt></p>
<p><tt><font size="+1">
<blockquote type="cite">ExecStart=/usr/local/sbin/glusterd<b>-3.8.8</b>
-p /var/run/glusterd.pid --log-level $LOG_LEVEL
$GLUSTERD_OPTIONS</blockquote>
</font></tt></p>
<p><tt><font size="+1">And I had to create symlinks for glusterd:</font></tt></p>
<p><tt><font size="+1">
<blockquote type="cite"><tt><font size="+1">cd /usr/local/sbin<br>
ln -s glusterd-3.8.8 glusterfsd-3.8.8</font></tt></blockquote>
I also ran ldconfig for good mesure...</font></tt></p>
<p><tt><font size="+1">
<blockquote type="cite"><tt><font size="+1">sudo ldconfig</font></tt></blockquote>
</font></tt></p>
<p><tt><font size="+1">When I run glusterd in the foreground (not
even with systemd) I'm left with some errors and the process
exits (errors emphasized):</font></tt></p>
<p><tt><font size="+1">
<blockquote type="cite"><a class="moz-txt-link-abbreviated" href="mailto:user@host0:~/glusterfs-3.8.8">user@host0:~/glusterfs-3.8.8</a> on
e5f3a990c [!☡]# sudo glusterd-3.8.8 --debug<br>
[2019-01-01 16:23:37.120684] I [MSGID: 100030]
[glusterfsd.c:2454:main] 0-glusterd-3.8.8: Started running
glusterd-3.8.8 version 3.8.8 (args: glusterd-3.8.8 --debug)<br>
[2019-01-01 16:23:37.120765] D
[logging.c:1791:__gf_log_inject_timer_event]
0-logging-infra: Starting timer now. Timeout = 120, current
buf size = 5<br>
[2019-01-01 16:23:37.121187] D [MSGID: 0]
[glusterfsd.c:660:get_volfp] 0-glusterfsd: loading volume
file /usr/local/etc/glusterfs/glusterd.vol<br>
[2019-01-01 16:23:37.137003] I [MSGID: 106478]
[glusterd.c:1379:init] 0-management: Maximum allowed open
file descriptors set to 65536<br>
[2019-01-01 16:23:37.137064] I [MSGID: 106479]
[glusterd.c:1428:init] 0-management: Using /var/lib/glusterd
as working directory<br>
[2019-01-01 16:23:37.137262] D [MSGID: 0]
[glusterd.c:406:glusterd_rpcsvc_options_build] 0-glusterd:
listen-backlog value: 128<br>
[2019-01-01 16:23:37.137683] D [rpcsvc.c:2316:rpcsvc_init]
0-rpc-service: RPC service inited.<br>
[2019-01-01 16:23:37.137723] D
[rpcsvc.c:1866:rpcsvc_program_register] 0-rpc-service: New
program registered: GF-DUMP, Num: 123451501, Ver: 1, Port: 0<br>
[2019-01-01 16:23:37.137798] D
[rpc-transport.c:283:rpc_transport_load] 0-rpc-transport:
attempt to load file
/usr/local/lib/glusterfs/3.8.8/rpc-transport/socket.so<br>
[2019-01-01 16:23:37.151778] D [socket.c:3938:socket_init]
0-socket.management: Configued transport.tcp-user-timeout=0<br>
[2019-01-01 16:23:37.151823] D [socket.c:4021:socket_init]
0-socket.management: SSL support on the I/O path is NOT
enabled<br>
[2019-01-01 16:23:37.151862] D [socket.c:4024:socket_init]
0-socket.management: SSL support for glusterd is NOT enabled<br>
[2019-01-01 16:23:37.151890] D [socket.c:4041:socket_init]
0-socket.management: using system polling thread<br>
[2019-01-01 16:23:37.151927] D
[name.c:584:server_fill_address_family] 0-socket.management:
option address-family not specified, defaulting to inet<br>
[2019-01-01 16:23:37.152173] D
[rpc-transport.c:283:rpc_transport_load] 0-rpc-transport:
attempt to load file
/usr/local/lib/glusterfs/3.8.8/rpc-transport/rdma.so<br>
[2019-01-01 16:23:37.155510] D
[rpc-transport.c:321:rpc_transport_load] 0-rpc-transport:
dlsym (gf_rpc_transport_reconfigure) on
/usr/local/lib/glusterfs/3.8.8/rpc-transport/rdma.so:
undefined symbol: reconfigure<br>
[2019-01-01 16:23:37.155830] W [MSGID: 103071]
[rdma.c:4589:__gf_rdma_ctx_create] 0-rpc-transport/rdma:
rdma_cm event channel creation failed [No such device]<br>
[2019-01-01 16:23:37.155884] W [MSGID: 103055]
[rdma.c:4896:init] 0-rdma.management: Failed to initialize
IB Device<br>
[2019-01-01 16:23:37.155920] W
[rpc-transport.c:354:rpc_transport_load] 0-rpc-transport:
'rdma' initialization failed<br>
[2019-01-01 16:23:37.156224] W
[rpcsvc.c:1638:rpcsvc_create_listener] 0-rpc-service: cannot
create listener, initing the transport failed<br>
<b>[2019-01-01 16:23:37.156258] E [MSGID: 106243]
[glusterd.c:1652:init] 0-management: creation of 1
listeners failed, continuing with succeeded transport</b><br>
[2019-01-01 16:23:37.156300] D
[rpcsvc.c:1866:rpcsvc_program_register] 0-rpc-service: New
program registered: GlusterD svc peer, Num: 1238437, Ver: 2,
Port: 0<br>
[2019-01-01 16:23:37.156332] D
[rpcsvc.c:1866:rpcsvc_program_register] 0-rpc-service: New
program registered: GlusterD svc cli read-only, Num:
1238463, Ver: 2, Port: 0<br>
[2019-01-01 16:23:37.156356] D
[rpcsvc.c:1866:rpcsvc_program_register] 0-rpc-service: New
program registered: GlusterD svc mgmt, Num: 1238433, Ver: 2,
Port: 0<br>
[2019-01-01 16:23:37.156384] D
[rpcsvc.c:1866:rpcsvc_program_register] 0-rpc-service: New
program registered: GlusterD svc mgmt v3, Num: 1238433, Ver:
3, Port: 0<br>
[2019-01-01 16:23:37.156414] D
[rpcsvc.c:1866:rpcsvc_program_register] 0-rpc-service: New
program registered: Gluster Portmap, Num: 34123456, Ver: 1,
Port: 0<br>
[2019-01-01 16:23:37.156438] D
[rpcsvc.c:1866:rpcsvc_program_register] 0-rpc-service: New
program registered: Gluster Handshake, Num: 14398633, Ver:
2, Port: 0<br>
[2019-01-01 16:23:37.156468] D
[rpcsvc.c:1866:rpcsvc_program_register] 0-rpc-service: New
program registered: Gluster MGMT Handshake, Num: 1239873,
Ver: 1, Port: 0<br>
[2019-01-01 16:23:37.156591] D [rpcsvc.c:2316:rpcsvc_init]
0-rpc-service: RPC service inited.<br>
[2019-01-01 16:23:37.156619] D
[rpcsvc.c:1866:rpcsvc_program_register] 0-rpc-service: New
program registered: GF-DUMP, Num: 123451501, Ver: 1, Port: 0<br>
[2019-01-01 16:23:37.156665] D
[rpc-transport.c:283:rpc_transport_load] 0-rpc-transport:
attempt to load file
/usr/local/lib/glusterfs/3.8.8/rpc-transport/socket.so<br>
[2019-01-01 16:23:37.156854] D [socket.c:3887:socket_init]
0-socket.management: disabling nodelay<br>
[2019-01-01 16:23:37.156882] D [socket.c:3938:socket_init]
0-socket.management: Configued transport.tcp-user-timeout=0<br>
[2019-01-01 16:23:37.156912] D [socket.c:4021:socket_init]
0-socket.management: SSL support on the I/O path is NOT
enabled<br>
[2019-01-01 16:23:37.156933] D [socket.c:4024:socket_init]
0-socket.management: SSL support for glusterd is NOT enabled<br>
[2019-01-01 16:23:37.156961] D [socket.c:4041:socket_init]
0-socket.management: using system polling thread<br>
[2019-01-01 16:23:37.157095] D
[rpcsvc.c:1866:rpcsvc_program_register] 0-rpc-service: New
program registered: GlusterD svc cli, Num: 1238463, Ver: 2,
Port: 0<br>
[2019-01-01 16:23:37.157125] D
[rpcsvc.c:1866:rpcsvc_program_register] 0-rpc-service: New
program registered: Gluster Handshake (CLI Getspec), Num:
14398633, Ver: 2, Port: 0<br>
[2019-01-01 16:23:37.157282] D [MSGID: 0]
[glusterd-utils.c:6379:glusterd_sm_tr_log_init] 0-glusterd:
returning 0<br>
[2019-01-01 16:23:37.157318] D [MSGID: 0]
[glusterd.c:1720:init] 0-management: cannot get
run-with-valgrind value<br>
<b>[2019-01-01 16:23:37.170633] E [MSGID: 106229]
[glusterd.c:455:glusterd_check_gsync_present] 0-glusterd:
geo-replication module not working as desired</b><br>
[2019-01-01 16:23:37.171476] D [MSGID: 0]
[glusterd.c:465:glusterd_check_gsync_present] 0-glusterd:
Returning -1<br>
<b>[2019-01-01 16:23:37.171572] E [MSGID: 101019]
[xlator.c:433:xlator_init] 0-management: Initialization of
volume 'management' failed, review your volfile again</b><br>
<b>[2019-01-01 16:23:37.171613] E [MSGID: 101066]
[graph.c:324:glusterfs_graph_init] 0-management:
initializing translator failed</b><br>
<b>[2019-01-01 16:23:37.171649] E [MSGID: 101176]
[graph.c:673:glusterfs_graph_activate] 0-graph: init
failed</b><br>
[2019-01-01 16:23:37.173130] D
[logging.c:1765:gf_log_flush_extra_msgs] 0-logging-infra:
Log buffer size reduced. About to flush 5 extra log messages<br>
[2019-01-01 16:23:37.173187] D
[logging.c:1768:gf_log_flush_extra_msgs] 0-logging-infra:
Just flushed 5 extra log messages<br>
[2019-01-01 16:23:37.173280] W [MSGID: 100032]
[glusterfsd.c:1327:cleanup_and_exit] 0-: received signum
(1), shutting down<br>
[2019-01-01 16:23:37.173335] D
[glusterfsd-mgmt.c:2385:glusterfs_mgmt_pmap_signout]
0-fsd-mgmt: portmapper signout arguments not given</blockquote>
<br>
</font></tt></p>
<p><tt><font size="+1">I can't pin what particular error caused the
process to exit and I failed findind informations on the
'management' volume fail... any hints?</font></tt></p>
<p><tt><font size="+1">Thanks,<br>
Raphael.<br>
</font></tt></p>
</body>
</html>