[Gluster-users] infiniband replicated distributed setup.--- network setup question...
mht at mail.dfci.harvard.edu
Wed Dec 19 02:33:22 UTC 2012
We have two networks and I have 4 servers I'm trying to turn into a
distributed/replicated Gluster storage cluster.
Each of the servers has two addresses, one ethernet, one IB.
The externet addresses are 184.108.40.206-4 and
the Infiniband addresses are 172.24.215.1-4
I login to the first server ("1") which is 220.127.116.11.
Let's say the servers are named 1,2,1R and 2R (on the ethernet side), but
1-IB, 2-IB, 1-IB-R and 2-IB-R on he Infiniband side.
The Volume is to be mounted by other servers in the IB network by native
so, I login to "1" over the ethernet network and do a peer probe.
All looks good from "1" with peer probe and peer status.
I probe the peers like this, successfully
gluster peer probe 2-ib
gluster peer probe 1-ib-r
gluster peer prove 2-ib-r
Peer status looks right.
I issue the following command which fails, of course:
gluster volume create gf2 replica 2 transport tcp,rdma \
1-ib:/mnt/d0-0 1-ib-r:/mnt/d0-0 2-ib:/mnt/d0-0 2-ib-r:/mnt/d0-0
>From any of those nodes, peer status is wrong:
For instance, if I ssh over to 2-ib,
and ask for a peer status, it shows the peers to to be
and *18.104.22.168* (the last is the ethernet side of, not the IB side)
I guess my question is this: How do I get the first node to use the ib
connecting to the other nodes. (both of these networks are public.) Do I
do something interesting with routes?
I need to fix this. Don't most people with IB also have an internet side
How do their networks and any local routing info get set up?
Am I ,missing something really obvious?
Director, Research Computing
Dana-Farber Cancer Institute.
We have two
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Gluster-users