[Gluster-users] Fedora 11 - 2.6.31 Kernel - Fuse 2.8.0 - Infiniband

[Nathan Stratton]

On Tue, 22 Sep 2009, Anand Avati wrote:

>>  Hate to post again, but anyone have any ideas on this?
>
> What does the server log have to say? Can you also check if port-1 is
> the active port in ibv_devinfo? Looks like ib-verbs messaging is not
> happening. Does ibv_srq_pingpong give sane results?

[    3.890311] ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April 4, 2008)
[    3.890315] ib_mthca: Initializing 0000:08:00.0
[    3.890354] ib_mthca 0000:08:00.0: PCI INT A -> GSI 28 (level, low) -> IRQ 28
[    7.899804] ADDRCONF(NETDEV_UP): ib0: link is not ready
[    7.902722] ADDRCONF(NETDEV_CHANGE): ib0: link becomes ready
[   17.935013] ib0: no IPv6 routers present


[root at xen1 ~]# ibv_devinfo
hca_id:	mthca0
 	fw_ver:				3.5.0
 	node_guid:			0005:ad00:0003:27e8
 	sys_image_guid:			0005:ad00:0100:d050
 	vendor_id:			0x02c9
 	vendor_part_id:			23108
 	hw_ver:				0xA1
 	board_id:			MT_0270110001
 	phys_port_cnt:			2
 		port:	1
 			state:			active (4)
 			max_mtu:		2048 (4)
 			active_mtu:		2048 (4)
 			sm_lid:			2
 			port_lid:		11
 			port_lmc:		0x00

 		port:	2
 			state:			down (1)
 			max_mtu:		2048 (4)
 			active_mtu:		512 (2)
 			sm_lid:			0
 			port_lid:		0
 			port_lmc:		0x00



[root at xen1 ~]# ibv_srq_pingpong 10.13.0.220
   local address:  LID 0x000b, QPN 0x300406, PSN 0x1ace41
   local address:  LID 0x000b, QPN 0x300407, PSN 0x6ba197
   local address:  LID 0x000b, QPN 0x300408, PSN 0xa6f895
   local address:  LID 0x000b, QPN 0x300409, PSN 0xf054c0
   local address:  LID 0x000b, QPN 0x30040a, PSN 0xea4bd3
   local address:  LID 0x000b, QPN 0x30040b, PSN 0xfe3039
   local address:  LID 0x000b, QPN 0x30040c, PSN 0x037fa4
   local address:  LID 0x000b, QPN 0x30040d, PSN 0x1feccd
   local address:  LID 0x000b, QPN 0x30040e, PSN 0x22daed
   local address:  LID 0x000b, QPN 0x30040f, PSN 0xcaa26b
   local address:  LID 0x000b, QPN 0x300410, PSN 0xe87f33
   local address:  LID 0x000b, QPN 0x300411, PSN 0x84bb4a
   local address:  LID 0x000b, QPN 0x300412, PSN 0x09286e
   local address:  LID 0x000b, QPN 0x300413, PSN 0xecf483
   local address:  LID 0x000b, QPN 0x300414, PSN 0xd55285
   local address:  LID 0x000b, QPN 0x300415, PSN 0xdd7065
   remote address: LID 0x0004, QPN 0x460406, PSN 0x198bcb
   remote address: LID 0x0004, QPN 0x460407, PSN 0x645159
   remote address: LID 0x0004, QPN 0x460408, PSN 0x4a1a2f
   remote address: LID 0x0004, QPN 0x460409, PSN 0x8dff52
   remote address: LID 0x0004, QPN 0x46040a, PSN 0xe317fd
   remote address: LID 0x0004, QPN 0x46040b, PSN 0x12da1b
   remote address: LID 0x0004, QPN 0x460418, PSN 0xc8e0de
   remote address: LID 0x0004, QPN 0x460419, PSN 0xfc6e7f
   remote address: LID 0x0004, QPN 0x46041a, PSN 0xa3ffb7
   remote address: LID 0x0004, QPN 0x46041b, PSN 0x0cc86d
   remote address: LID 0x0004, QPN 0x46041c, PSN 0x107a0d
   remote address: LID 0x0004, QPN 0x46041d, PSN 0xe2661c
   remote address: LID 0x0004, QPN 0x46041e, PSN 0xfb8fd8
   remote address: LID 0x0004, QPN 0x46041f, PSN 0xc438a5
   remote address: LID 0x0004, QPN 0x460420, PSN 0x0be0ff
   remote address: LID 0x0004, QPN 0x460421, PSN 0x91b657

This one ends odd, I think it should tell me more info, but it just sits 
there.

-Nathan