[Gluster-users] glusterfs over rdma ... not.

[Dan Cyr]

Try upgrading the firmware. I had this exact problem last week and the firmware update solved it.

Dan

From: gluster-users-bounces at gluster.org [mailto:gluster-users-bounces at gluster.org] On Behalf Of Harry Mangalam
Sent: Saturday, November 05, 2011 8:17 AM
To: gluster-users at gluster.org; landman at scalableinformatics.com
Subject: Re: [Gluster-users] glusterfs over rdma ... not.


Thanks for the help, Joe,

This looks like the problem:

==================================

$ ibv_devinfo

Failed to get IB devices list: Function not implemented

==================================

There appears to be a problem.. Either i don';t have a valid IB driver or the device isn't recognized. lsmod says:

$ lsmod |egrep '(ib|rdma)'

rdma_cm 31030 0

ib_cm 40316 1 rdma_cm

iw_cm 9385 1 rdma_cm

ib_sa 22006 2 rdma_cm,ib_cm

ib_addr 6766 1 rdma_cm

ib_mthca 140934 0

ib_mad 41321 3 ib_cm,ib_sa,ib_mthca

ib_core 64935 6 rdma_cm,ib_cm,iw_cm,ib_sa,ib_mthca,ib_mad

so the card seems to be recognized and the driver is loaded (google says this card is suboptimal for such an infrastructure but I inherited this testbed so will have to deal with it for now).

hjm

On Friday 04 November 2011 18:32:36 Joe Landman wrote:

> On 11/04/2011 09:06 PM, Harry Mangalam wrote:

> > OK - finished some tests over tcp and ironed out a lot of

> > problems. rdma is next; should be snap now....

> >

> > [I must admit that this is my 1st foray into the land of IB, so

> > some of the following may be obvious to a non-naive admin..]

>

> > except that while I can create and start the volume with rdma as transport:

> First things first:

>

>

> ibv_devinfo

>

> What does it report?

>

> > ==================================

> >

> > root at pbs3:~

> >

> > 622 $ gluster volume info glrdma

> >

> > Volume Name: glrdma

> >

> > Type: Distribute

> >

> > Status: Started

> >

> > Number of Bricks: 4

> >

> > Transport-type: tcp,rdma

> >

> > Bricks:

> >

> > Brick1: pbs1:/data2

> >

> > Brick2: pbs2:/data2

> >

> > Brick3: pbs3:/data2

> >

> > Brick4: pbs3:/data

> >

> > ==================================

> >

> > I can't mount the damn thing. This seems to be a fairly frequent

> > problem according to google.. Again, all servers and clients are

> > ubuntu 10.04.3/64b, running self-compiled 3.3b1.

>

> try a

>

> showmount -e pbs2

>

> Also, did you enable the auth bits as last time?

--

Harry Mangalam - Research Computing, OIT, Rm 225 MSTB, UC Irvine

[ZOT 2225] / 92697 Google Voice Multiplexer: (949) 478-4487

MSTB Lat/Long: (33.642025,-117.844414) (paste into Google Maps)

--

This signature has been OCCUPIED!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20111105/439aa470/attachment.html>