[Gluster-devel] rdma: misuse of ibv_ack_cq_events api

[Lorne Li]

Hello folks,

I think ibv_ack_cq_events may be used incorrectly in 
gf_rdma_recv_completion_proc
and gf_rdma_send_completion_proc functions in 
rpc/rpc-transport/rdma/src/rdma.c.

Function gf_rdma_recv_completion_proc calls ibv_ack_cq_events(event_cq, 
num_wr)
before it returns, where the num_wr is the number of work completions 
returned
by ibv_poll_cq function. Therefore, this function regards the number of work
completions(ibv_wc) as the number of completion queue events.

But if I have understood correctly, completion queue events are produced by
ibv_get_cq_event instead of ibv_poll_cq. So after ibv_get_cq_event producing
an event, users should call `ibv_ack_cq_events(event_cq, 1)` to ack this 
event.

Looking forward to your reply. Thanks.

Lorne

------------------------------------------------------------------------------
Here is example code of `ibv_get_cq_event` man page, which supports my 
opinion.

     /* https://linux.die.net/man/3/ibv_get_cq_event */

     /* Wait for the completion event */
     if (ibv_get_cq_event(channel, &ev_cq, &ev_ctx)) {
         fprintf(stderr, "Failed to get cq_event\n");
         return 1;
     }

     /* Ack the event */
     ibv_ack_cq_events(ev_cq, 1);

     /* Request notification upon the next completion event */
     if (ibv_req_notify_cq(ev_cq, 0)) {
         fprintf(stderr, "Couldn't request CQ notification\n");
         return 1;
     }