[Gluster-users] Stupid question re multiple networks

Thu Nov 13 21:56:09 UTC 2014

Hi all
may I just add a question to the topic?
In a scenario with (say) two or three servers, one distributed volume,
native (fuse) clients, and stable filesystems with more read than write
operations, what is the typical amount of traffic between the servers,
compared to the client-server traffic?
In case of replicated volume instead of distributed?
What is the server-server traffic made up of, in case of native clients? If
it is just hearbeat-like traffic?
In other words, could be there a performance gain in separating
client-server traffic on one network and server-server traffic on a
different network? Or not?

Thanks a lot in advance for any answer!

    Mario

On Thu, Nov 13, 2014 at 8:15 PM, Jeff Darcy <jdarcy at redhat.com> wrote:

> > AFAIK multiple network scenario only works if you are not using Gluster
> via
> > FUSE mounts or gfapi from remote hosts. It will work for NFS access or
> when
> > you set up something like Samba with CTDB. Just not with native Gluster
> as
> > the server always tells the clients which addresses to connect to: ie
> your
> > storage hosts will always supply the connection details of the hosts that
> > are configured in gluster to your storage clients.
> >
> > I wonder if this could be gaffer-taped with some bridging/vlan/arp
> spoofing
> > trickery but I'm not sure I'd trust such a hack.
> >
> > It would be *really* nice if there was a way to set up gluster so you
> could
> > specify different IPs for backend and frontend operations.
>
> As you suggest, there are various kinds of "trickery" that can be used
> to fake multi-network support even for native mounts.  I've seen it done
> via split-horizon DNS, explicit host routes, and iptables.  *Proper*
> support for multiple networks is part of the proposed 4.0 feature set.
>
> http://www.gluster.org/community/documentation/index.php/Planning40
>
> In fact, I would greatly appreciate your help defining what "proper"
> means in this context.  Clearly, we need to add the concept of a network
> to our (informal) object model, and sort out the host/address/network
> relationships.  Then we need a way to direct certain traffic flows to
> certain networks.  The question is: how do we present this to the user?
> Let's take a whack at how to define networks etc. using the CLI's
> current object-verb syntax (even though it's a bit clunky).
>
>    gluster> network add user-net 1.2.3.0/24
>    gluster> network add back-end 5.6.0.0/16
>    gluster> peer probe 1.2.3.4
>    gluster> peer probe 5.6.7.8
>
> So far, so good.  Note that on the second probe we should be able to
> recognize that this is just a new address (on another network) for the
> host we already added with the first probe.  Heartbeats, quorums, etc.
> should also be aware of multi-homed hosts.  Maybe there's a better
> syntax, but this will do for now.  Let's add a volume.
>
>    gluster> volume create silly-vol 1.2.3.4:/brick
>
> So, which network address should the daemon for 1.2.3.4:/brick expose
> for clients?  Which address should it use for internal traffic such as
> rebalance or self-heal?  This is where it gets tricky.  Let's start by
> saying that *by default* all traffic is on the interface specified on
> the "volume create" line.  If we want to do something different...
>
>    # ONLY redirect rebalance traffic.
>    gluster> volume route silly-vol rebalance back-end
>
> Now rebalance traffic goes through 5.6.7.8 instead.  Is that intuitive?
> What about these?
>
>    # Export a volume on multiple networks.
>    gluster> volume route silly-vol client user-net some-other-net
>
>    # Redirect rebalance, self-heal, anything else we think of.
>    gluster> volume route silly-vol all-mgmt back-end
>
>    # Redirect GLOBALLY instead of per volume.
>    gluster> cluster route rebalance back-end
>
> Does this seem like it's heading in the right direction?  It doesn't
> look too bad to me, but my perspective is hardly typical.  Is there
> something *users* would like to be able to do with multiple networks
> that can't be expressed this way, or is there some better way to define
> how these multiple networks should be used?  Please let us know.
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-users
>

-- 
Mario Benincasa
Via del Conservatorio 55
00186 Roma - Italy
tel.   +39 066872917
mob.   +39 3346210331
fax:   +39 0697656510
email: mbeninca at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20141113/238fd958/attachment.html>