[Bugs] [Bug 1264245] New: glusterd lost track of brick port numbers after brick daemon dies

[bugzilla at redhat.com]

https://bugzilla.redhat.com/show_bug.cgi?id=1264245

            Bug ID: 1264245
           Summary: glusterd lost track of brick port numbers after brick
                    daemon dies
           Product: GlusterFS
           Version: 3.7.4
         Component: glusterd
          Severity: high
          Assignee: bugs at gluster.org
          Reporter: eivind at pacbell.net
                CC: bugs at gluster.org, gluster-bugs at redhat.com



Description of problem:
After a brick daemon dies, glusterd lost track of new/future brick listen
ports.
Two different error scenarios can happen:
a) A replacement brick from the same node where a brick daemon previously died
will not be healed.
b) A new volume created using a brick from same server where a brick daemon
previously died will not be replicated (by the client)


Version-Release number of selected component (if applicable):
3.7.4


How reproducible:
Every time.


Steps to Reproduce (both scenario a+b):
1a. Create a distributed-replicated 1x2 volume
2a. kill -9 <brick-pid>
3a. stop + delete volume
4a. replace-brick with another brick on same node where <brick-pid> died
(healing works)
5a. kill -9 <replacement-brick-pid>
6a. replace-brick with yet another brick (healing fails because wrong pid is
used to connect to new brick)
7a. grep "Connection refused" /var/log/glusterfs/glustershd.log

1b. Create a distributed-replicated 1x2 volume
2b. kill -9 <brick-pid>
3b. stop + delete volume
4b. Create new 1x2 volume using same (cleaned) bricks as in 1b
5b. mount it.
6b. On client, grep "Connection refused" /var/log/glusterfs/<volname>.log

Actual results:

a. # grep "Connection refused" /var/log/glusterfs/glustershd.log
[2015-09-18 00:55:24.717023] E [socket.c:2278:socket_connect_finish]
0-voltest-client-0: connection to 192.168.1.3:49152 failed (Connection refused)

b. # grep "Connection refused" /var/log/glusterfs/voltest.log
[2015-09-18 00:44:59.117344] E [socket.c:2278:socket_connect_finish]
4-voltest-client-0: connection to 192.168.1.3:49152 failed (Connection refused)



Expected results:


Additional info:

Restarting glusterd after the brick daemon is killed will prevent the
"Connection refused" in both a) and b)

-- 
You are receiving this mail because:
You are on the CC list for the bug.
You are the assignee for the bug.