[Gluster-users] Fixing a rejected peer

[Atin Mukherjee]

On Tue, Mar 6, 2018 at 6:00 AM, Jamie Lawrence <jlawrence at squaretrade.com>
wrote:

> Hello,
>
> So I'm seeing a rejected peer with 3.12.6. This is with a replica 3 volume.
>
> It actually began as the same problem with a different peer. I noticed
> with (call it) gluster-2, when I couldn't make a new volume.  I compared
> /var/lib/glusterd between them, and found that somehow the options in one
> of the vols differed. (I suspect this was due to attempting to create the
> volume via the Ovirt GUI; suffice to say I'm not using it for things like
> this in the future.) So I stopped the daemons and corrected that (gluster-2
> had a tiering entry the others didn't).
>

When you say the others didn't how many peers are you talking about? Are
they all running 3.12.6? We had a bug
https://bugzilla.redhat.com/show_bug.cgi?id=1544637 which could lead you to
such situations but that has been fixed in 3.12.6. So if all of the nodes
are running with the same version i.e. 3.12.6 and the cluster.op-version is
set to latest, then ideally you shouldn't see this problem. Could you
clarify?


>
> Started things back up and now gluster-3 is being rejected by the other
> two. The error is below.
>
> I'm tempted to repeat - down things, copy the checksum the "good" ones
> agree on, start things; but given that this has turned into a
> balloon-squeezing exercise, I want to make sure I'm not doing this the
> wrong way.
>

Yes, that's the way. Copy /var/lib/glusterd/vols/<volname>/ from the good
node to the rejected one and restart glusterd service on the rejected peer.


>
> What is the currently accepted best method for fixing this?
>
> And given that this happened on a nearly brand-new deployment, it worries
> me a bit that this happened while nothing hinky was going on - I installed
> Gluster manually, but the rest of the systems management has been via
> Ovirt. Has anyone else seen issues with this?
>
> Thanks,
>
> -j
>
>
>
> [2018-03-06 00:14:06.141281] I [MSGID: 106490] [glusterd-handler.c:2891:__glusterd_handle_probe_query]
> 0-glusterd: Received probe from uuid: 77cdfbba-348c-43fe-ab3d-00621904ea9c
> [2018-03-06 00:14:06.145540] I [MSGID: 106493] [glusterd-handler.c:2954:__glusterd_handle_probe_query]
> 0-glusterd: Responded to sc5-gluster-1, op_ret: 0, op_errno: 0, ret: 0
> [2018-03-06 00:14:06.145697] I [MSGID: 106490] [glusterd-handler.c:2891:__glusterd_handle_probe_query]
> 0-glusterd: Received probe from uuid: c1877e0d-ccb2-401e-83a6-e4a680af683a
> [2018-03-06 00:14:06.145831] I [MSGID: 106493] [glusterd-handler.c:2954:__glusterd_handle_probe_query]
> 0-glusterd: Responded to sc5-gluster-10g-2, op_ret: 0, op_errno: 0, ret: 0
> [2018-03-06 00:14:06.149357] I [MSGID: 106490] [glusterd-handler.c:2540:__
> glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from
> uuid: c1877e0d-ccb2-401e-83a6-e4a680af683a
> [2018-03-06 00:14:06.149631] E [MSGID: 106010] [glusterd-utils.c:3374:
> glusterd_compare_friend_volume] 0-management: Version of Cksums
> sc5-ovirt_engine differ. local cksum = 53769889, remote cksum = 2068896937
> on peer sc5-gluster-2
> [2018-03-06 00:14:06.149774] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp]
> 0-glusterd: Responded to sc5-gluster-2 (0), ret: 0, op_ret: -1
> [2018-03-06 00:14:06.151393] I [MSGID: 106490] [glusterd-handler.c:2540:__
> glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from
> uuid: 77cdfbba-348c-43fe-ab3d-00621904ea9c
> [2018-03-06 00:14:06.152127] E [MSGID: 106010] [glusterd-utils.c:3374:
> glusterd_compare_friend_volume] 0-management: Version of Cksums
> sc5-ovirt_engine differ. local cksum = 53769889, remote cksum = 2068896937
> on peer sc5-gluster-10g-1
> [2018-03-06 00:14:06.152314] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp]
> 0-glusterd: Responded to sc5-gluster-10g-1 (0), ret: 0, op_ret: -1
> [2018-03-06 00:14:06.164819] I [MSGID: 106143] [glusterd-pmap.c:295:pmap_registry_bind]
> 0-pmap: adding brick /gluster-bricks/sc5_ovirt_engine/sc5_ovirt_engine on
> port 49152
> [2018-03-06 00:14:06.443882] I [MSGID: 106487] [glusterd-handler.c:1485:__
> glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180306/98d311db/attachment.html>