[Gluster-users] RDMA/Ethernet wi ROCEE - failed to modify QP to RTR

[Amar Tumballi]

Hi Ben,

Did any RDMA/Ethernet users see this Gluster error?  If so do you know what
> caused it and how to fix?  If you haven't seen it, what RPMs and
> configuration do you use specific to RDMA/Ethernet?
>
>
We haven't tested Gluster with RDMA/Ethernet till date. If you have the
setup, can you try testing with master branch code with below patches
applied. (apply the patches in order).

http://review.gluster.com/148
http://review.gluster.com/149
http://review.gluster.com/201

Let us know if it works for you with these patches.

Regards,
Amar



> [2011-11-10 10:30:20.595801] C
> [rdma.c:2417:rdma_connect_qp]0-rpc-transport/rdma: Failed to modify QP to
> RTR
> [2011-11-10 10:30:20.595930] E [rdma.c:4159:rdma_handshake_pollin]
> 0-rpc-transport/rdma: rdma.management: failed to connect with remote QP
>
> I see this when I run RDMA over Ethernet using ROCEE RPMs, but when I run
> over Infiniband using RHEL 6.2-, it runs fine.  On the same Ethernet
> configuration, Gluster/TCP runs fine, NFS/RDMA runs fine as does AMQP app.
>  But qperf and rping utilities fail in the same way.  Firmware on the HCAs
> is not the latest, is it worth risk to upgrade?
>
> I went into debugger and found line where qperf fails, it's near line 2056
> in rdma.c in qperf sources (qperf-debuginfo,  I did Makefile)
>
> (gdb)
> 2088        } else if (dev->trans == IBV_QPT_RC) {
> (gdb)
> 2090            flags = IBV_QP_STATE              |
> (gdb)
> 2097            if (ibv_modify_qp(dev->qp, &rtr_attr, flags) != 0)
> (gdb)
> 2098                error(SYS, "failed to modify QP to RTR");
> (gdb)
>
> Gluster fails in rdma_connect_qp() calling the same routine, but perhaps
> with different parameters.
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20111114/845ac44d/attachment.html>