[Gluster-users] glusterfs over rdma ... not.

[Harry Mangalam]

Thanks for the help, Joe,

This looks like the problem:

==================================
$ ibv_devinfo
Failed to get IB devices list: Function not implemented
==================================

There appears to be a problem..  Either i don';t have a valid IB 
driver or the device isn't recognized.  lsmod says:

$ lsmod |egrep '(ib|rdma)'
rdma_cm                31030  0 
ib_cm                  40316  1 rdma_cm
iw_cm                   9385  1 rdma_cm
ib_sa                  22006  2 rdma_cm,ib_cm
ib_addr                 6766  1 rdma_cm
ib_mthca              140934  0 
ib_mad                 41321  3 ib_cm,ib_sa,ib_mthca
ib_core                64935  6 
rdma_cm,ib_cm,iw_cm,ib_sa,ib_mthca,ib_mad

so the card seems to be recognized and the driver is loaded (google 
says this card is suboptimal for such an infrastructure but I 
inherited this testbed so will have to deal with it for now).

hjm


On Friday 04 November 2011 18:32:36 Joe Landman wrote:
> On 11/04/2011 09:06 PM, Harry Mangalam wrote:
> > OK - finished some tests over tcp and ironed out a lot of
> > problems. rdma is next; should be snap now....
> > 
> > [I must admit that this is my 1st foray into the land of IB, so
> > some of the following may be obvious to a non-naive admin..]
> 
> > except that while I can create and start the volume with rdma as 
transport:
> First things first:
> 
> 
> 	ibv_devinfo
> 
> What does it report?
> 
> > ==================================
> > 
> > root at pbs3:~
> > 
> > 622 $ gluster volume info glrdma
> > 
> > Volume Name: glrdma
> > 
> > Type: Distribute
> > 
> > Status: Started
> > 
> > Number of Bricks: 4
> > 
> > Transport-type: tcp,rdma
> > 
> > Bricks:
> > 
> > Brick1: pbs1:/data2
> > 
> > Brick2: pbs2:/data2
> > 
> > Brick3: pbs3:/data2
> > 
> > Brick4: pbs3:/data
> > 
> > ==================================
> > 
> > I can't mount the damn thing. This seems to be a fairly frequent
> > problem according to google.. Again, all servers and clients are
> > ubuntu 10.04.3/64b, running self-compiled 3.3b1.
> 
> try a
> 
> 	showmount -e pbs2
> 
> Also, did you enable the auth bits as last time?

-- 
Harry Mangalam - Research Computing, OIT, Rm 225 MSTB, UC Irvine
[ZOT 2225] / 92697  Google Voice Multiplexer: (949) 478-4487 
MSTB Lat/Long: (33.642025,-117.844414) (paste into Google Maps)
--
This signature has been OCCUPIED!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20111105/78972f36/attachment.html>