[Bugs] [Bug 1215114] New: gluster peer probe hangs

bugzilla at redhat.com bugzilla at redhat.com
Fri Apr 24 10:49:37 UTC 2015


https://bugzilla.redhat.com/show_bug.cgi?id=1215114

            Bug ID: 1215114
           Summary: gluster peer probe hangs
           Product: GlusterFS
           Version: 3.5.2
         Component: glusterd
          Severity: medium
          Assignee: bugs at gluster.org
          Reporter: free.aaa at gmail.com
                CC: bugs at gluster.org, gluster-bugs at redhat.com



Created attachment 1018395
  --> https://bugzilla.redhat.com/attachment.cgi?id=1018395&action=edit
gfs1 and gfs3 debug logging

Initial data:
All nodes have the same:
1) os distribution (proxmox v3.3, kernel 2.6.32-32-pve)
2) disabled selinux
3) default iptables with ALLOW action
4) version of glusterfs
      - ii  glusterfs-client                 3.5.2-1
      - ii  glusterfs-common                 3.5.2-1
      - ii  glusterfs-server                 3.5.2-1
5) correct and working DNS forward and reverse resolution

6) 
192.168.9.53/gfs3 \ 
192.168.9.54/gfs4  - already in cluster
192.168.9.56/gfs6 /

192.168.9.51/gfs1 - want to add to the cluster
192.168.9.52/gfs2 - want to add to the cluster

gfs1 and gfs2 previously were part of another glusterfs cluster, but I stopped
all services and removed /var/lib/glusterd directory.

7) cluster contains several production distributed-replicated volumes

Description of problem: 
When I do gluster peer probe gfs1 or gfs2 from any node in the cluster the
command hangs and after timeout new peer has status "Probe Sent to Peer":
gfs3#gluster peer status
Number of Peers: 3

Hostname: gfs6
Uuid: 6bd6ee25-e257-4703-b500-330741b90471
State: Peer in Cluster (Connected)

Hostname: gfs4
Uuid: bb1bed20-25bf-43b0-8faa-49f1b5b9ae59
State: Peer in Cluster (Connected)

Hostname: gfs1
Uuid: c5cd8152-c239-474a-977b-9c6b35edd857
State: Probe Sent to Peer (Connected) 

-- gfs3 added peer gfs1 ------------
gfs3#cat /var/lib/glusterd/peers/c5cd8152-c239-474a-977b-9c6b35edd857
uuid=c5cd8152-c239-474a-977b-9c6b35edd857
state=1
hostname1=gfs1

-- but gfs1 did not add gfs3
gfs1#cat /var/lib/glusterd/peers/192.168.9.53
uuid=00000000-0000-0000-0000-000000000000
state=8
hostname1=192.168.9.53


Debug logging showed that gfs3 sent req, gfs1 sent resp, gfs3 received response
and after that gfs3 does not do anything until command timeouts (see attach).
So gfs3 does not complete peer handshake with gfs1.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
You are the assignee for the bug.


More information about the Bugs mailing list