[Gluster-users] glusterfs over rdma ... not.

[Harry Mangalam]

OK - will do - ugh.

hjm

On Saturday 05 November 2011 11:34:53 Dan Cyr wrote:
> Try upgrading the firmware. I had this exact problem last week and
> the firmware update solved it.
> 
> Dan
> 
> From: gluster-users-bounces at gluster.org
> [mailto:gluster-users-bounces at gluster.org] On Behalf Of Harry
> Mangalam Sent: Saturday, November 05, 2011 8:17 AM
> To: gluster-users at gluster.org; landman at scalableinformatics.com
> Subject: Re: [Gluster-users] glusterfs over rdma ... not.
> 
> 
> Thanks for the help, Joe,
> 
> This looks like the problem:
> 
> ==================================
> 
> $ ibv_devinfo
> 
> Failed to get IB devices list: Function not implemented
> 
> ==================================
> 
> There appears to be a problem.. Either i don';t have a valid IB
> driver or the device isn't recognized. lsmod says:
> 
> $ lsmod |egrep '(ib|rdma)'
> 
> rdma_cm 31030 0
> 
> ib_cm 40316 1 rdma_cm
> 
> iw_cm 9385 1 rdma_cm
> 
> ib_sa 22006 2 rdma_cm,ib_cm
> 
> ib_addr 6766 1 rdma_cm
> 
> ib_mthca 140934 0
> 
> ib_mad 41321 3 ib_cm,ib_sa,ib_mthca
> 
> ib_core 64935 6 rdma_cm,ib_cm,iw_cm,ib_sa,ib_mthca,ib_mad
> 
> so the card seems to be recognized and the driver is loaded (google
> says this card is suboptimal for such an infrastructure but I
> inherited this testbed so will have to deal with it for now).
> 
> hjm
> 
> On Friday 04 November 2011 18:32:36 Joe Landman wrote:
> > On 11/04/2011 09:06 PM, Harry Mangalam wrote:
> > > OK - finished some tests over tcp and ironed out a lot of
> > > 
> > > problems. rdma is next; should be snap now....
> > > 
> > > 
> > > 
> > > [I must admit that this is my 1st foray into the land of IB, so
> > > 
> > > some of the following may be obvious to a non-naive admin..]
> > 
> > > except that while I can create and start the volume with rdma as 
transport:
> > First things first:
> > 
> > 
> > 
> > 
> > 
> > ibv_devinfo
> > 
> > 
> > 
> > What does it report?
> > 
> > > ==================================
> > > 
> > > 
> > > 
> > > root at pbs3:~
> > > 
> > > 
> > > 
> > > 622 $ gluster volume info glrdma
> > > 
> > > 
> > > 
> > > Volume Name: glrdma
> > > 
> > > 
> > > 
> > > Type: Distribute
> > > 
> > > 
> > > 
> > > Status: Started
> > > 
> > > 
> > > 
> > > Number of Bricks: 4
> > > 
> > > 
> > > 
> > > Transport-type: tcp,rdma
> > > 
> > > 
> > > 
> > > Bricks:
> > > 
> > > 
> > > 
> > > Brick1: pbs1:/data2
> > > 
> > > 
> > > 
> > > Brick2: pbs2:/data2
> > > 
> > > 
> > > 
> > > Brick3: pbs3:/data2
> > > 
> > > 
> > > 
> > > Brick4: pbs3:/data
> > > 
> > > 
> > > 
> > > ==================================
> > > 
> > > 
> > > 
> > > I can't mount the damn thing. This seems to be a fairly
> > > frequent
> > > 
> > > problem according to google.. Again, all servers and clients
> > > are
> > > 
> > > ubuntu 10.04.3/64b, running self-compiled 3.3b1.
> > 
> > try a
> > 
> > 
> > 
> > showmount -e pbs2
> > 
> > 
> > 
> > Also, did you enable the auth bits as last time?
> 
> --
> 
> Harry Mangalam - Research Computing, OIT, Rm 225 MSTB, UC Irvine
> 
> [ZOT 2225] / 92697 Google Voice Multiplexer: (949) 478-4487
> 
> MSTB Lat/Long: (33.642025,-117.844414) (paste into Google Maps)
> 
> --
> 
> This signature has been OCCUPIED!

-- 
Harry Mangalam - Research Computing, OIT, Rm 225 MSTB, UC Irvine
[ZOT 2225] / 92697  Google Voice Multiplexer: (949) 478-4487 
MSTB Lat/Long: (33.642025,-117.844414) (paste into Google Maps)
--
This signature has been OCCUPIED!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20111105/8a51d0dc/attachment.html>