[Bugs] [Bug 1341184] New: Call_bail of a frame due to not able to find a saved frame in reply

[bugzilla at redhat.com]

https://bugzilla.redhat.com/show_bug.cgi?id=1341184

            Bug ID: 1341184
           Summary: Call_bail of a frame due to not able to find a saved
                    frame in reply
           Product: GlusterFS
           Version: 3.6.10
         Component: rpc
          Keywords: Triaged
          Assignee: bugs at gluster.org
          Reporter: ndevos at redhat.com
                CC: bugs at gluster.org, rgowdapp at redhat.com
        Depends On: 1340361



+++ This bug was initially created as a clone of Bug #1340361 +++

Description of problem:

This was observed on one of user's production setup.

[raghu at unused 01610290]$ grep "cannot lookup the saved" <client-log>
[2016-02-15 22:40:02.575925] C [rpc-clnt.c:452:rpc_clnt_fill_request_info]
<client-log>: cannot lookup the saved frame corresponding to xid (14161323)

Above is the log-message indicating that a reply from server couldn't lookup a
saved-frame to unwind the response. The xid (0xd815ab) matches to one of the
unaccounted call-bails. Similar msg was seen from other log.

As for the RCA for not able to lookup a saved frame for response, I saw in
rpc-clnt:

1. Submit the request to transport for transmission to brick.
2. Save the frame for future reference while processing reply.

Now, if we get a response between 1 and 2 (before we were able to save the
frame), we would be saving the frame of a request whose reply is already
received. This can result in call-bail.


Version-Release number of selected component (if applicable):
Zero day bug, present in all releases

How reproducible:
Racy. Not consistent

Steps to Reproduce:
1.
2.
3.

Actual results:
A reply didn't have an associated saved-frame in rpc-clnt layer.

Expected results:
A reply should always have a saved-frame in rpc-clnt layer.


Additional info:

--- Additional comment from Vijay Bellur on 2016-05-27 09:36:29 CEST ---

REVIEW: http://review.gluster.org/14547 (rpc-clnt: save the frame before
submitting request to transport) posted (#1) for review on master by
Raghavendra G (rgowdapp at redhat.com)


Referenced Bugs:

https://bugzilla.redhat.com/show_bug.cgi?id=1340361
[Bug 1340361] Call_bail of a frame due to not able to find a saved frame in
reply
-- 
You are receiving this mail because:
You are on the CC list for the bug.
You are the assignee for the bug.