[Gluster-users] glusterd crashes on Assertion failed: rsp.op == txn_op_info.op

Sanju Rakonde srakonde at redhat.com
Fri Jun 21 05:54:39 UTC 2019


Thanks for the update Olaf.

You are hitting the bug, which is mentioned by Atin in this mail thread.
I'm not sure whether we can backport the fix to release-5 branch. I will
update you regarding this in early next week.

On Thu, Jun 20, 2019 at 7:30 PM Olaf Buitelaar <olaf.buitelaar at gmail.com>
wrote:

> Hi Sanju,
>
> going through the stacks i noticed that this function was in
> between; glusterd_volume_rebalance_use_rsp_dict
> So it might after all have todo something with the rebalancing logic.
> I've checked the cmd_history.log and exactly on the time of crash time
> command was executed;
> [2019-06-19 07:25:03.108360]  : volume rebalance ovirt-data status :
> SUCCESS preceding a couple of other status checks of rebalancing. The
> complete batch of 2 mins before, all reported success.
> These commands are executed by ovirt about every 2 minutes, to pull for
> the status of gluster.
> I'm sure no actual rebalancing tasks were running, also checked the last
> time that was @2019-06-08 21:13:02 and was completed successfully
> Hopefully this is additional useful info.
>
> Thanks Olaf
>
> Op do 20 jun. 2019 om 14:52 schreef Olaf Buitelaar <
> olaf.buitelaar at gmail.com>:
>
>> Hi Sanju,
>>
>> you can download the coredump here;
>> http://edgecastcdn.net/0004FA/files/core_dump.zip (around 20MB)
>>
>> Thanks Olaf
>>
>> Op do 20 jun. 2019 om 08:35 schreef Sanju Rakonde <srakonde at redhat.com>:
>>
>>> Olaf,
>>>
>>> Can you please paste complete backtrace from the core file, so that we
>>> can analyse what is wrong here.
>>>
>>> On Wed, Jun 19, 2019 at 10:31 PM Olaf Buitelaar <
>>> olaf.buitelaar at gmail.com> wrote:
>>>
>>>> Hi Atin,
>>>>
>>>> Thank you for pointing out this bug report, however no rebalancing task
>>>> was running during this event. So maybe something else is causing this?
>>>> According the report this should be fixed in gluster 6, unfortunate
>>>> ovirt doesn't seem to officially support that version, so i'm stuck on the
>>>> 5 branch for now.
>>>> Any chance this will be back ported?
>>>>
>>>> Thanks Olaf
>>>>
>>>>
>>>> Op wo 19 jun. 2019 om 17:57 schreef Atin Mukherjee <amukherj at redhat.com
>>>> >:
>>>>
>>>>> Please see - https://bugzilla.redhat.com/show_bug.cgi?id=1655827
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jun 19, 2019 at 5:52 PM Olaf Buitelaar <
>>>>> olaf.buitelaar at gmail.com> wrote:
>>>>>
>>>>>> Dear All,
>>>>>>
>>>>>> Has anybody seen this error on gluster 5.6;
>>>>>> [glusterd-rpc-ops.c:1388:__glusterd_commit_op_cbk]
>>>>>> (-->/lib64/libgfrpc.so.0(+0xec60) [0x7fbfb7801c60]
>>>>>> -->/usr/lib64/glusterfs/5.6/xlator/mgmt/glusterd.so(+0x79b7a)
>>>>>> [0x7fbfac50db7a]
>>>>>> -->/usr/lib64/glusterfs/5.6/xlator/mgmt/glusterd.so(+0x77393)
>>>>>> [0x7fbfac50b393] ) 0-: Assertion failed: rsp.op == txn_op_info.op
>>>>>>
>>>>>> checking the code;
>>>>>> https://github.com/gluster/glusterfs/blob/6fd8281ac9af58609979f660ece58c2ed1100e72/xlators/mgmt/glusterd/src/glusterd-rpc-ops.c#L1388
>>>>>>
>>>>>> doesn't seem to reveal much on what could causing this.
>>>>>>
>>>>>> It's the second time this occurs.
>>>>>>
>>>>>> Attached the full stack.
>>>>>>
>>>>>> Thanks Olaf
>>>>>> _______________________________________________
>>>>>> Gluster-users mailing list
>>>>>> Gluster-users at gluster.org
>>>>>> https://lists.gluster.org/mailman/listinfo/gluster-users
>>>>>
>>>>> _______________________________________________
>>>> Gluster-users mailing list
>>>> Gluster-users at gluster.org
>>>> https://lists.gluster.org/mailman/listinfo/gluster-users
>>>
>>>
>>>
>>> --
>>> Thanks,
>>> Sanju
>>>
>>

-- 
Thanks,
Sanju
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20190621/47c3d3f1/attachment.html>


More information about the Gluster-users mailing list