[Bugs] [Bug 1425556] New: glusterd log is flooded with stale disconnect rpc messages

[bugzilla at redhat.com]

https://bugzilla.redhat.com/show_bug.cgi?id=1425556

            Bug ID: 1425556
           Summary: glusterd log is flooded with stale disconnect rpc
                    messages
           Product: GlusterFS
           Version: 3.10
         Component: glusterd
          Keywords: Triaged
          Severity: low
          Priority: low
          Assignee: bugs at gluster.org
          Reporter: amukherj at redhat.com
                CC: bugs at gluster.org, gyadav at redhat.com
        Depends On: 1421724



+++ This bug was initially created as a clone of Bug #1421724 +++

Description of problem:

glusterd log is flooded with following error message:

<snip>

[2017-02-13 14:27:59.706023] W
[glusterd-handler.c:5684:__glusterd_brick_rpc_notify] 0-management: got
disconnect from stale rpc on /tmp/b6
[2017-02-13 14:27:59.706442] W
[glusterd-handler.c:5684:__glusterd_brick_rpc_notify] 0-management: got
disconnect from stale rpc on /tmp/b7
[2017-02-13 14:27:59.706920] W
[glusterd-handler.c:5684:__glusterd_brick_rpc_notify] 0-management: got
disconnect from stale rpc on /tmp/b8
[2017-02-13 14:28:02.707970] W
[glusterd-handler.c:5684:__glusterd_brick_rpc_notify] 0-management: got
disconnect from stale rpc on /tmp/b6
[2017-02-13 14:28:02.708449] W
[glusterd-handler.c:5684:__glusterd_brick_rpc_notify] 0-management: got
disconnect from stale rpc on /tmp/b7
[2017-02-13 14:28:02.708937] W
[glusterd-handler.c:5684:__glusterd_brick_rpc_notify] 0-management: got
disconnect from stale rpc on /tmp/b8
[2017-02-13 14:28:05.709992] W
[glusterd-handler.c:5684:__glusterd_brick_rpc_notify] 0-management: got
disconnect from stale rpc on /tmp/b6
[2017-02-13 14:28:05.710565] W
[glusterd-handler.c:5684:__glusterd_brick_rpc_notify] 0-management: got
disconnect from stale rpc on /tmp/b7
[2017-02-13 14:28:05.711114] W
[glusterd-handler.c:5684:__glusterd_brick_rpc_notify] 0-management: got
disconnect from stale rpc on /tmp/b8
[2017-02-13 14:28:08.710356] W
[glusterd-handler.c:5684:__glusterd_brick_rpc_notify] 0-management: got
disconnect from stale rpc on /tmp/b6
[2017-02-13 14:28:08.710733] W
[glusterd-handler.c:5684:__glusterd_brick_rpc_notify] 0-management: got
disconnect from stale rpc on /tmp/b7
[2017-02-13 14:28:08.711139] W
[glusterd-handler.c:5684:__glusterd_brick_rpc_notify] 0-management: got
disconnect from stale rpc on /tmp/b8
[2017-02-13 14:28:11.711076] W
[glusterd-handler.c:5684:__glusterd_brick_rpc_notify] 0-management: got
disconnect from stale rpc on /tmp/b6
[2017-02-13 14:28:11.711179] W
[glusterd-handler.c:5684:__glusterd_brick_rpc_notify] 0-management: got
disconnect from stale rpc on /tmp/b7
[2017-02-13 14:28:11.711269] W
[glusterd-handler.c:5684:__glusterd_brick_rpc_notify] 0-management: got
disconnect from stale rpc on /tmp/b8

</snip>

refer "steps reproducible" section for the reproducer

Version-Release number of selected component (if applicable):
mainline

How reproducible:
Always

Steps to Reproduce:
1. create & start a volume (brick mux is off)
2. turn on brick mux
3. stop and start the volume

Actual results:


Expected results:


Additional info:

--- Additional comment from Worker Ant on 2017-02-21 08:04:32 EST ---

REVIEW: https://review.gluster.org/16699 (glusterd: unref brickinfo object on
volume stop) posted (#1) for review on master by Atin Mukherjee
(amukherj at redhat.com)

--- Additional comment from Worker Ant on 2017-02-21 11:56:12 EST ---

REVIEW: https://review.gluster.org/16702 (glusterd : Fix for flooding of logs
when multiplex is enable) posted (#1) for review on master by Gaurav Yadav
(gyadav at redhat.com)

--- Additional comment from Worker Ant on 2017-02-21 12:25:13 EST ---

COMMIT: https://review.gluster.org/16699 committed in master by Atin Mukherjee
(amukherj at redhat.com) 
------
commit 9cdfbdced23cd43b8738636a3ed906c8d4267d67
Author: Atin Mukherjee <amukherj at redhat.com>
Date:   Tue Feb 21 18:33:14 2017 +0530

    glusterd: unref brickinfo object on volume stop

    If brick multiplexing is enabled, on a volume stop glusterd was not
    unrefing the brickinfo rpc object which lead to a flood of stale rpc
    logs.

    Change-Id: I18fedcd6921042ef2e945605466194b7b53fe2f7
    BUG: 1421724
    Signed-off-by: Atin Mukherjee <amukherj at redhat.com>
    Reviewed-on: https://review.gluster.org/16699
    Smoke: Gluster Build System <jenkins at build.gluster.org>
    NetBSD-regression: NetBSD Build System <jenkins at build.gluster.org>
    Reviewed-by: Jeff Darcy <jdarcy at redhat.com>
    CentOS-regression: Gluster Build System <jenkins at build.gluster.org>
    Reviewed-by: Samikshan Bairagya <samikshan at gmail.com>


Referenced Bugs:

https://bugzilla.redhat.com/show_bug.cgi?id=1421724
[Bug 1421724] glusterd log is flooded with stale disconnect rpc messages
-- 
You are receiving this mail because:
You are on the CC list for the bug.
You are the assignee for the bug.