[Gluster-users] Reliably mounting a gluster volume

[Kevin Lemonnier]

Hi,

As we were discussing in the "gluster volume not mounted on boot" you
should probably just go with AutoFS, not ideal but I don't see any
other reliable solutions.


On Fri, Oct 21, 2016 at 02:46:05PM +0200, Paul Boven wrote:
> Hi everyone,
> 
> For the past few days I've been experimenting with Gluster and systemd. 
> The issue I'm trying to solve is that my gluster servers always fail to 
> self-mount their gluster volume locally on boot. Apparently this is 
> because the mount happens right after glusterd has been started, but 
> before it is ready to serve the volume.
> 
> I'm doing a refresh of our internal gluster based KVM system, bringing 
> it to Ubuntu 16.04LTS. As the Ubuntu gluster package as shipped still 
> has this boot/mount issue, and to simplify things a bit, I've removed 
> all SystemV and Upstart that ships with the current Ubuntu Gluster 
> package, aiming for a systemd-only solution. Ubuntu 16.04LTS uses systemd.
> 
> The problem, in my opinion, stems from the fact that in the Unit file 
> for glusterd, it is declared as a 'forking' kind of service. This means 
> that as soon as the double fork happens, systemd has no option but to 
> consider the service as available, and continues with the rest of its 
> work. I try to delay the mounting of my /gluster by adding 
> "x-systemd.requires=glusterd.service" but for the reasons above, that 
> still causes the mount to happen immediately after glusterd has started, 
> and then the mount fails.
> 
> Is there a way for systemd to know when the gluster service is actually 
> able to service a mount request, so one can delay this step of the boot 
> process?
> 
> In the Unit file, I have:
> [Unit]
> Requires=rpcbind.service
> After=network.target rpcbind.service network-online.target
> 
> The curious thing is that, according to gluster.log, the gluster client 
> does find out on which hostnames the subvolumes are available. However, 
> it seems that talking to both the local (0-gv0-client-0) as remote 
> (0-gv0-client-1) fails. For the service on localhost, the error is 
> 'failed to get the port number for remote subvolume'. For the remote 
> volume, it is 'no route to host'. But at this stage, local networking 
> (which is fully static and on the same network) should already be up.
> 
> Some error messages during the mount:
> 
> [12:15:50.749137] E [MSGID: 114058] 
> [client-handshake.c:1524:client_query_portmap_cbk] 0-gv0-client-0: 
> failed to get the port number for remote subvolume. Please run 'gluster 
> volume status' on server to see if brick process is running.
> [12:15:50.749178] I [MSGID: 114018] [client.c:2042:client_rpc_notify] 
> 0-gv0-client-0: disconnected from gv0-client-0. Client process will keep 
> trying to connect to glusterd until brick's port is available
> [12:15:53.679570] E [socket.c:2278:socket_connect_finish] 
> 0-gv0-client-1: connection to 10.0.0.3:24007 failed (No route to host)
> [12:15:53.679611] E [MSGID: 108006] [afr-common.c:3880:afr_notify] 
> 0-gv0-replicate-0: All subvolumes are down. Going offline until atleast 
> one of them comes back up.
> 
> Once the machine has fully booted and I log in, simply typing 'mount 
> /gluster' always succeeds. I would really appreciate your help in making 
> this happening on boot without intervention.
> 
> Regards, Paul Boven.
> -- 
> Paul Boven <boven at jive.eu> +31 (0)521-596547
> Unix/Linux/Networking specialist
> Joint Institute for VLBI in Europe - www.jive.eu
> VLBI - It's a fringe science
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users

-- 
Kevin Lemonnier
PGP Fingerprint : 89A5 2283 04A0 E6E9 0111
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Digital signature
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20161021/02c3e26b/attachment.sig>