[Gluster-users] Upgrade from 6.9 to 7.7 stuck (peer is rejected)

[mabi]

Dear all,

Thanks to this fix I could successfully upgrade from GlusterFS 6.9 to 7.8 but now, 1 week later after the upgrade, I have rebooted my third node (arbiter node) and unfortunately the bricks do not want to come up on that node. I get the same following error message:

[2020-10-26 06:21:59.726705] E [MSGID: 106012] [glusterd-utils.c:3682:glusterd_compare_friend_volume] 0-management: Cksums of quota configuration of volume myvol-private differ. local cksum = 0, remote cksum = 66908910 on peer node2.domain
[2020-10-26 06:21:59.726871] I [MSGID: 106493] [glusterd-handler.c:3715:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to node2.domain (0), ret: 0, op_ret: -1
[2020-10-26 06:21:59.728164] I [MSGID: 106490] [glusterd-handler.c:2434:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 5f4ccbf4-33f6-4298-8b31-213553223349
[2020-10-26 06:21:59.728969] E [MSGID: 106012] [glusterd-utils.c:3682:glusterd_compare_friend_volume] 0-management: Cksums of quota configuration of volume myvol-private differ. local cksum = 0, remote cksum = 66908910 on peer node1.domain
[2020-10-26 06:21:59.729099] I [MSGID: 106493] [glusterd-handler.c:3715:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to node1.domain (0), ret: 0, op_ret: -1

Can someone please advise what I need to do in order to have my arbiter node up and running again as soon as possible?

Thank you very much in advance for your help.

Best regards,
Mabi

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Monday, September 7, 2020 5:49 AM, Sanju Rakonde <srakonde at redhat.com> wrote:

> Hi,
>
> issue https://github.com/gluster/glusterfs/issues/1332 is fixed now with https://github.com/gluster/glusterfs/commit/865cca1190e233381f975ff36118f46e29477dcf.
>
> It will be backported to release-7 and release-8 branches soon.
>
> On Mon, Sep 7, 2020 at 1:14 AM Strahil Nikolov <hunter86_bg at yahoo.com> wrote:
>
>> Your e-mail got in the spam...
>>
>> If you haven't fixed the issue, check Hari's topic about quota issues (based on the error message you provided) : https://medium.com/@harigowtham/glusterfs-quota-fix-accounting-840df33fcd3a
>>
>> Most probably there is a quota issue and you need to fix it.
>>
>> Best Regards,
>> Strahil Nikolov
>>
>> В неделя, 23 август 2020 г., 11:05:27 Гринуич+3, mabi <mabi at protonmail.ch> написа:
>>
>> Hello,
>>
>> So to be precise I am exactly having the following issue:
>>
>> https://github.com/gluster/glusterfs/issues/1332
>>
>> I could not wait any longer to find some workarounds or quick fixes so I decided to downgrade my rejected from 7.7 back to 6.9 which worked.
>>
>> I would be really glad if someone could fix this issue or provide me a workaround which works because version 6 of GlusterFS is not supported anymore so I would really like to move on to the stable version 7.
>>
>> Thank you very much in advance.
>>
>> Best regards,
>> Mabi
>>
>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>>
>> On Saturday, August 22, 2020 7:53 PM, mabi <mabi at protonmail.ch> wrote:
>>
>>> Hello,
>>>
>>> I just started an upgrade of my 3 nodes replica (incl arbiter) of GlusterFS from 6.9 to 7.7 but unfortunately after upgrading the first node, that node gets rejected due to the following error:
>>>
>>> [2020-08-22 17:43:00.240990] E [MSGID: 106012] [glusterd-utils.c:3537:glusterd_compare_friend_volume] 0-management: Cksums of quota configuration of volume myvolume differ. local cksum = 3013120651, remote cksum = 0 on peer myfirstnode.domain.tld
>>>
>>> So glusterd process is running but not glusterfsd.
>>>
>>> I am exactly in the same issue as described here:
>>>
>>> https://www.gitmemory.com/Adam2Marsh
>>>
>>> But I do not see any solutions or workaround. So now I am stuck with a degraded GlusterFS cluster.
>>>
>>> Could someone please advise me as soon as possible on what I should do? Is there maybe any workarounds?
>>>
>>> Thank you very much in advance for your response.
>>>
>>> Best regards,
>>> Mabi
>>
>> ________
>>
>> Community Meeting Calendar:
>>
>> Schedule -
>> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
>> Bridge: https://bluejeans.com/441850968
>>
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> https://lists.gluster.org/mailman/listinfo/gluster-users
>> ________
>>
>> Community Meeting Calendar:
>>
>> Schedule -
>> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
>> Bridge: https://bluejeans.com/441850968
>>
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> https://lists.gluster.org/mailman/listinfo/gluster-users
>
> --
> Thanks,
> Sanju
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20201026/6a27ebd2/attachment.html>