[Gluster-users] 3.1: Multiple networks and Client access

Craig Carl craig at gluster.com
Mon Oct 18 05:10:31 UTC 2010

Udo - 
With 3.1 when you mount/create/change a volume those changes are propagated via RPC to all of the other Gluster servers and clients. When you created the volume using 10.10.x.x IP addresses those IPs where what got sent to the client. In previous versions you could have just edited the client side configuration file and change or added the 192. addresses but not in this version, due to DVM. There should be a way to make multiple networks work so I will file a bug. 
In the meantime I think I have a workaround. If you use names instead of IP addresses and then make sure DNS or host files are setup properly is should work as Gluster does export via all interfaces. For example if the servers have these IPs - 

server1 -, 
server2 -, 
server3 -, 

#gluster volume create test-ext stripe 3 server1:/ext server2:/ext server3:/ext 

You would just need to make sure that hosts on the 10.10.x.x resolve the servername to its 10. IP, and clients on the 192.x resolve to the 192 IP. Should be a simple change to the /etc/host files. 

Please let me know if this works so I can include that information in my bug report. 



Craig Carl 

Senior Systems Engineer; Gluster, Inc. 
Cell - ( 408) 829-9953 (California, USA) 
Office - ( 408) 770-1884 
Gtalk - craig.carl at gmail.com 
Twitter - @gluster 
Installing Gluster Storage Platform, the movie! 

From: "Udo Waechter" <udo.waechter at uni-osnabrueck.de> 
To: gluster-users at gluster.org 
Sent: Sunday, October 17, 2010 12:57:18 AM 
Subject: Re: [Gluster-users] 3.1: Multiple networks and Client access 

although I totally forgot about the firewall, this problem is not related to it. 

The ports you mentioned are open. 

I have created another volume using the ip-adresses from the external interfaces 

$ gluster volume create test-ext stripe 3 192.168.x.x1:/ext 192.168.x.x2:/ext 192.168.x.x3:/ext 

and this can not only be mounted, it also works perfectly. 

How could this work 
$ gluster volume create test-ext stripe 3 10.10.x.x1:/ext 10.10.x.x2:/ext 10.10.x.x3:/ext 
if the ip-addresses of the 10.10.x.x network are not reachable from the 192.168.x.x network? 

I have read somewhere in the docs that by default, glusterd listens on all interfaces, but does it also by default export everything to all interfaces? 

Thanks again, 

On 16.10.2010, at 16:37, Jacob Shucart wrote: 

> Udo, 
> It sounds to me like a firewall is blocking access to the Gluster system preventing some of the traffic from happening. Please see: 
> http://www.gluster.com/community/documentation/index.php/Gluster_3.1:_Installing_GlusterFS_on_Red_Hat_Package_Manager_%28RPM%29_Distributions 
> This lists the ports that need to be open both between Gluster storage nodes and between the client. This also needs to be open on all storage nodes to all clients. 
> Please try this and let us know if this helps. 
> -Jacob 
> ----- Original Message ----- 
> From: "Udo Waechter" <udo.waechter at uni-osnabrueck.de> 
> To: gluster-users at gluster.org 
> Sent: Saturday, October 16, 2010 5:07:15 AM 
> Subject: [Gluster-users] 3.1: Multiple networks and Client access 
> Hello all, 
> I just ran over Gluster 3.1 and got it up and running in notime. Thanks for the great FS. 
> I have a question regarding multiple network. 
> We have all our servers connected via an non-routed internal switch (a private subnet through which only the servers shall communicate with each other), additionally the official subnet to enable their reachability from the outside world. 
> The internal IPnetwork is 10.10.x.x and the external shall be 192.168.x.x (for the sake of the question). 
> I have now set up 3 servers as a GlusterFS. With a Striped Volume over the 10.10.x.x network. 
> All clients that are within this network can mount and use the volume. 
> The problem arises with the clients from the other 192.168.x network. They can mount the volume via: 
> mount -t glusterfs 192.168.x.x:volname /mnt 
> but they fail to use the mountpoint (ls and so) 
> I have read another post (http://bit.ly/a3Tm6n) but the solution mentioned there does not seem to work with version 3.1 any more. 
> The server(s) complain about "transport.socket..." not being allowed in the volume file. 
> The client from the 192.168.x.x network tries to contact the bricks via the 10.10.x network and not via 192.168.x.x (which would work) 
> Is there a solution to this problem? 
> As I understand it, I would need to define the bricks with their 192.168.x.x ip-adresses but this would mean that the internal network won't be used. 
> Can I define one brick with both ip-adresses (or 
> Setting the "auth.allow" option to both subnets did not solve the problem (except for the mount being possible after setting it) 
> Any help is highly appreciated. 
> Thanks, 
> udo. 
> -- 
> ---[ Institute of Cognitive Science @ University of Osnabrueck 
> ---[ Albrechtstrasse 28, D-49076 Osnabrueck, 969-3362 
> ---[ Documentation: https://doc.ikw.uni-osnabrueck.de 
> _______________________________________________ 
> Gluster-users mailing list 
> Gluster-users at gluster.org 
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users 

:: udo waechter - root at zoide.net :: N 52º16'30.5" E 8º3'10.1" 
:: genuine input for your ears: http://auriculabovinari.de 
:: your eyes: http://ezag.zoide.net 
:: your brain: http://zoide.net 

Gluster-users mailing list 
Gluster-users at gluster.org 

More information about the Gluster-users mailing list