[Gluster-users] infiniband replicated distributed setup.--- network setup question...

[Matthew Temple]

Just got back to this after long delay.    I have a slightly different
question now.   I have a working distributed/replicated setup with 4 bricks.
They are called:    gf2ib-1, gf2ib-2,  gf2ib-1r, gf2ib-2r.

Running the "gluster peer status"  command from gf2ib-1 I get:

Number of Peers: 3

Hostname: gf2ib-2
Uuid: ed599401-25e2-41bb-8e41-b00247a3f3a0
State: Peer in Cluster (Connected)

Hostname: gf2ib-1r
Uuid: 8ea05f85-b416-4047-9e65-9a8185534bb6
State: Peer in Cluster (Connected)

Hostname: gf2ib-2r
Uuid: 34c58361-b3ef-4201-998b-09ff5c73558e
State: Peer in Cluster (Connected)

That's what I expect.   But from the other three peers, I get a variant of
this (the IP address of the first):

[root at gf2-2 rcadmin]# gluster peer status
Number of Peers: 3

Hostname: 172.24.215.202
Uuid: f5edea20-9467-48ed-b4f1-dc566a9b6d02
State: Peer in Cluster (Connected)

Hostname: gf2ib-1r
Uuid: 8ea05f85-b416-4047-9e65-9a8185534bb6
State: Peer in Cluster (Connected)

Hostname: gf2ib-2r
Uuid: 34c58361-b3ef-4201-998b-09ff5c73558e
State: Peer in Cluster (Connected)

Sure enough, when I look at /var/lib/glusterd/peers/* I see something like
this:

uuid=f5edea20-9467-48ed-b4f1-dc566a9b6d02
state=3
hostname1=172.24.215.202

Maybe it doesn't matter, but can I stop the volume and change that hostname
line?   The machine I'm looking at
does't have trouble with it's DNS entry:

[root at gf2-2r rcadmin]# nslookup 172.24.215.202
Server: 155.52.45.100
Address: 155.52.45.100#53

Non-authoritative answer:
202.215.24.172.in-addr.arpa name = gf2ib-1.dfci.harvard.edu.

The "volume info gf2" command gives the *names* of the 4 bricks, too.
Can anyone explain?

Matt Temple

------
Matt Temple
Director, Research Computing
Dana-Farber Cancer Institute.


On Wed, Dec 19, 2012 at 1:30 AM, Shawn Heisey <gluster at elyograg.org> wrote:

> On 12/18/2012 7:33 PM, Matthew Temple wrote:
>
>> From any of those nodes, peer status is wrong:
>> For instance, if I ssh over to 2-ib,
>> and ask for a peer status, it shows the peers to to be
>>
>> 1-ib-r
>> 2-ib-r
>> and *155.52.48.1* (the last is the ethernet side of, not the IB side)
>>
>> I guess my question is this:   How do I get the first node to use the ib
>> side when
>> connecting to the other nodes. (both of these networks are public.)   Do
>> I have to
>> do something interesting with routes?
>>
>
> Just probe the first server by name from any of the other three.  That
> will fix the peers.
>
> Thanks,
> Shawn
>
> ______________________________**_________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://supercolony.gluster.**org/mailman/listinfo/gluster-**users<http://supercolony.gluster.org/mailman/listinfo/gluster-users>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20130118/afcdc43a/attachment.html>