[Gluster-devel] How do we identify peers?

Justin Clift jclift at redhat.com
Wed Jun 26 14:21:49 UTC 2013


On 26/06/2013, at 1:42 PM, Kaushal M wrote:
> Your concept seems similar to approaches 2-3 that I described, except
> that its described in much more detail. The connection-group command
> does to job associating peers with addresses.
> But we still have the question of how do we identify peers, both
> internally and externally. If we can come to an decision on this, then
> I feel we have the solution to our problem.

Cool. :)

My thinking (so far) is that we should use the UUID of a host as its
main identifier (internally).  And _not_ care (from a peer point of view)
which interface is in use.

We could instead control the interfaces used for volumes through a defined
grouping mechanism (eg like in my email, or whatever else we decide on
instead).

Example setup (based on my home dev lab here) is:

  server 1  (hostname "gluster1" atm)
  GlusterFS UUID = AAAAA-AAAAA-AAAAA-AAAAA-AAAAA
  Ethernet IP = 192.168.1.201 (DNS: gluster1)
  Infiniband port 0 = 172.16.101.1 (DNS: gluster1-1)
  Infiniband port 1 = 172.16.102.1 (DNS: gluster1-2)

  server 2  (hostname "gluster2" atm)
  GlusterFS UUID = BBBBB-BBBBB-BBBBB-BBBBB-BBBBB
  Ethernet IP = 192.168.1.202 (DNS: gluster2)
  Infiniband port 0 = 172.16.101.2 (DNS: gluster2-1)
  Infiniband port 1 = 172.16.102.2 (DNS: gluster2-2)

  server 3  (hostname "packstack" atm)
  GlusterFS UUID = CCCCC-CCCCC-CCCCC-CCCCC-CCCCC
  Ethernet IP = 192.168.1.203 (DNS: packstack)
  Infiniband port 0 = 172.16.101.3 (DNS: packstack-1)
  Infiniband port 1 = 172.16.102.3 (DNS: packstack-2)

So, lets say I've just installed Gluster (next upcoming version) on them.

I start glusterd on them all (first time), then peer probe each of them from
each other.  Shouldn't matter which address on them gets probed.  When a
server is probed they tell each other their (self-generated?) UUID's so they
can recognise each other.

(an optimisation might be for each server to also share each others list of
IP addresses on their attached network cards, but that's neither here nor
there)

  gluster1> $ sudo gluster peer probe gluster2-2
  peer probe: success
  gluster1> $ sudo gluster peer probe packstack
  peer probe: success
  $ sudo gluster peer status
  Number of Peers: 2

  Hostname: gluster2
  Port: 24007
  Uuid: BBBBB-BBBBB-BBBBB-BBBBB-BBBBB
  State: Peer in Cluster (Connected)

  Hostname: packstack
  Port: 24007
  Uuid: CCCCC-CCCCC-CCCCC-CCCCC-CCCCC
  State: Peer in Cluster (Connected)

For the "what should be displayed as hostname in user visible output"
type of question, I can see why you're suggesting aliases.  Each of
my example servers has several DNS names (based on network interface
IP) and it's own hostname.  But, I'm not 100% sure every OS we support
will reliably report back a nice "hostname" if it's asked.

So, I think supporting "aliases" as a stand in for host names makes
sense there.  Something like:

  gluster1> $ sudo gluster peer alias 172.16.102.2 gluster2
  peer alias: alias BBBBB-BBBBB-BBBBB-BBBBB-BBBBB to "gluster2" success
  gluster1> $ sudo gluster peer alias packstack1-2 packstack
  peer alias: alias CCCCC-CCCCC-CCCCC-CCCCC-CCCCC to "packstack" success

The first argument after "alias" is a contactable gluster box, which
gluster can go and fetch its UUID from.  From then on, any place which
needs a "hostname" for display can use the configured alias (2nd argument
after "alias" above) for that server.

Now, with controlling which volumes can be connected from where, I think
the "connection groups" idea in my earlier email has merit, or at least
is a step in the right direction.  (and I'm now thinking that using
"connection groups" in the peer probe command isn't the right way to go,
they should just be for volumes)

We could create "connection groups" for volumes as per that email, and
use them to control what has access to which volumes.  When something
needs to be displayed, it could use the above defined alias + IP
address.  Something like:

  $ sudo gluster connection-group create client-group gluster1:192.168.1.201 gluster2:192.168.1.202 packstack:192.168.1.203
  connection-group created: client-group
  $ sudo gluster volume groupdefault myvolume client-group
  volume myvolume default group: client-group

I think this would give us sane ability to both use consistent naming
for gluster servers, plus use multiple network interfaces intelligently.

(Btw, hopefully this email isn't too long to read.)

Thoughts? :)

Regards and best wishes,

Justin Clift

--
Open Source and Standards @ Red Hat

twitter.com/realjustinclift





More information about the Gluster-devel mailing list