[Gluster-users] Peer Probe
Tony Saenz
tony at filmsolutions.com
Mon Feb 25 21:50:02 UTC 2013
Trying to first get this working with IPoIB
[root at fpsgluster ~]# ibhosts
Ca : 0x00117500007937b2 ports 1 "fpsgluster2 qib0"
Ca : 0x0011750000792af2 ports 1 "fpsgluster qib0"
I'm able to ping the other box from Infiniband to Infiniband card
Ifconfig uses the ioctl access method to get the full address information, which limits hardware addresses to 8 bytes.
Because Infiniband address has 20 bytes, only the first 8 bytes are displayed correctly.
Ifconfig is obsolete! For replacement check ip.
ib0 Link encap:InfiniBand HWaddr 80:00:00:03:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00
inet addr:10.0.4.35 Bcast:10.0.4.255 Mask:255.255.255.0
inet6 addr: fe80::211:7500:79:2af2/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:2044 Metric:1
RX packets:1567 errors:0 dropped:0 overruns:0 frame:0
TX packets:587 errors:0 dropped:24 overruns:0 carrier:0
collisions:0 txqueuelen:256
RX bytes:342622 (334.5 KiB) TX bytes:96554 (94.2 KiB)
[root at fpsgluster2 ~]# ifconfig ib0
Ifconfig uses the ioctl access method to get the full address information, which limits hardware addresses to 8 bytes.
Because Infiniband address has 20 bytes, only the first 8 bytes are displayed correctly.
Ifconfig is obsolete! For replacement check ip.
ib0 Link encap:InfiniBand HWaddr 80:00:00:03:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00
inet addr:10.0.4.34 Bcast:10.0.4.255 Mask:255.255.255.0
inet6 addr: fe80::211:7500:79:37b2/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:65520 Metric:1
RX packets:599 errors:0 dropped:0 overruns:0 frame:0
TX packets:1558 errors:0 dropped:8 overruns:0 carrier:0
collisions:0 txqueuelen:256
RX bytes:95180 (92.9 KiB) TX bytes:346728 (338.6 KiB)
[root at fpsgluster ~]# ping -I ib0 10.0.4.34
PING 10.0.4.34 (10.0.4.34) from 10.0.4.35 ib0: 56(84) bytes of data.
64 bytes from 10.0.4.34: icmp_seq=1 ttl=64 time=12.6 ms
64 bytes from 10.0.4.34: icmp_seq=2 ttl=64 time=0.184 ms
/etc/hosts looks correct
[root at fpsgluster2 ~]# cat /etc/hosts | grep ib
10.0.4.35 fpsglusterib
10.0.4.34 fpsgluster2ib
[root at fpsgluster ~]# cat /etc/hosts| grep ib
10.0.4.35 fpsglusterib
10.0.4.34 fpsgluster2ib
I haven't created the new volume yet as I can't get the peer probe to work off the Infiniband card. It's only seeing the NIC cards I currently have it hooked in to.
On Feb 25, 2013, at 11:57 AM, harry mangalam <harry.mangalam at uci.edu>
wrote:
> It /might be/probably is/ DNS-related.
>
> Are you trying to do this with RDMA or IPoIB?
>
> If IPoIB, are ALL your /etc/hosts files in sync (IB names separate and
> distinct from the ethernet interfaces) and responsive to the appropriate
> interfaces?
>
> Do the IB interfaces show up as distinct (and connected) on an 'ifconfig -a'
> and 'ibstat' dump?
>
> Do all the peers show up on an 'ibhosts' query?
>
> What is the output of:
> gluster volume status <your_volume>
> and
> gluster volume status <your_volume> detail
>
>
> hjm
>
> On Monday, February 25, 2013 07:46:00 PM Tony Saenz wrote:
>> It shows this but it's still going through my NIC cards and not the
>> Infiniband. (Checked the traffic on the cards themselves)
>>
>> [root at fpsgluster ~]# gluster peer status
>> Number of Peers: 1
>>
>> Hostname: fpsgluster2
>> Uuid: 9b7e7c2d-f05b-4cc8-b55a-571e383328d0
>> State: Peer in Cluster (Connected)
>>
>> On Feb 25, 2013, at 10:51 AM, Torbjørn Thorsen <torbjorn at trollweb.no> wrote:
>>> Your error message seems to indicate that the peer is already in the
>>> storage pool ?
>>> What is the output of "gluster peer status" ?
>>>
>>> On Mon, Feb 25, 2013 at 7:28 PM, Tony Saenz <tony at filmsolutions.com>
> wrote:
>>>> Any help please? The regular NICs are fine which is what it currently
>>>> sees but I'd like to move them over to the Infiniband cards.>>
>>>> On Feb 22, 2013, at 1:50 PM, Anthony Saenz <tony at filmsolutions.com>
> wrote:
>>>>> Hey,
>>>>>
>>>>> I was wondering if I could get a bit of help.. I installed a new
>>>>> Infiniband card into my servers but I'm unable to get it to come up as
>>>>> a peer. Is there something I'm missing?
>>>>>
>>>>> [root at fpsgluster testvault]# gluster peer probe fpsgluster2ib
>>>>> Probe on host fpsgluster2ib port 0 already in peer list
>>>>>
>>>>> [root at fpsgluster testvault]# yum list installed | grep gluster
>>>>> glusterfs.x86_64 3.3.1-1.el6
>>>>> installed
>>>>> glusterfs-devel.x86_64 3.3.1-1.el6
>>>>> installed
>>>>> glusterfs-fuse.x86_64 3.3.1-1.el6
>>>>> installed
>>>>> glusterfs-geo-replication.x86_64 3.3.1-1.el6
>>>>> installed
>>>>> glusterfs-rdma.x86_64 3.3.1-1.el6
>>>>> installed
>>>>> glusterfs-server.x86_64 3.3.1-1.el6
>>>>> installed
>>>>>
>>>>> Thanks.
>>>>
>>>> _______________________________________________
>>>> Gluster-users mailing list
>>>> Gluster-users at gluster.org
>>>> http://supercolony.gluster.org/mailman/listinfo/gluster-users
>>>
>>> --
>>> Vennlig hilsen
>>> Torbjørn Thorsen
>>> Utvikler / driftstekniker
>>>
>>> Trollweb Solutions AS
>>> - Professional Magento Partner
>>> www.trollweb.no
>>>
>>> Telefon dagtid: +47 51215300
>>> Telefon kveld/helg: For kunder med Serviceavtale
>>>
>>> Besøksadresse: Luramyrveien 40, 4313 Sandnes
>>> Postadresse: Maurholen 57, 4316 Sandnes
>>>
>>> Husk at alle våre standard-vilkår alltid er gjeldende
>>
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> http://supercolony.gluster.org/mailman/listinfo/gluster-users
>
> ---
> Harry Mangalam - Research Computing, OIT, Rm 225 MSTB, UC Irvine
> [m/c 2225] / 92697 Google Voice Multiplexer: (949) 478-4487
> 415 South Circle View Dr, Irvine, CA, 92697 [shipping]
> MSTB Lat/Long: (33.642025,-117.844414) (paste into Google Maps)
> ---
> "Something must be done. [X] is something. Therefore, we must do it."
> Bruce Schneier, on American response to just about anything.
More information about the Gluster-users
mailing list