[Gluster-users] Elasticsearch facing CorruptIndexException exception with GlusterFs 3.10.1

Mon May 8 09:19:12 UTC 2017

Wait, wait we are discussing this issue only. Expect a reply in some time
:-)

On Mon, May 8, 2017 at 2:19 PM, Abhijit Paul <er.abhijitpaul at gmail.com>
wrote:

> poking for previous mail reply
>
> On Sun, May 7, 2017 at 1:06 AM, Abhijit Paul <er.abhijitpaul at gmail.com>
> wrote:
>
>>  https://pkalever.wordpress.com/2017/03/14/elasticsearch-wit
>> h-gluster-block/
>> here used tested environment is Fedora ,
>> but i am using RHEL based Oracle linux so does gluster-block compatible
>> with RHEL as well? What i needs to change & make it work?
>>
>> On Fri, May 5, 2017 at 5:42 PM, Pranith Kumar Karampuri <
>> pkarampu at redhat.com> wrote:
>>
>>>
>>>
>>> On Fri, May 5, 2017 at 5:40 PM, Pranith Kumar Karampuri <
>>> pkarampu at redhat.com> wrote:
>>>
>>>>
>>>>
>>>> On Fri, May 5, 2017 at 5:36 PM, Abhijit Paul <er.abhijitpaul at gmail.com>
>>>> wrote:
>>>>
>>>>> So should i start using gluster-block with elasticsearch in kubernetes
>>>>> environment?
>>>>>
>>>>> My expectation from gluster-block is, it should not CorruptIndex
>>>>> of elasticsearch...and issue facing in previous mails.
>>>>>
>>>>> Please let me know whether should i processed with above mentioned
>>>>> combination.
>>>>>
>>>>
>>>> We are still in the process of fixing the failure scenarios of
>>>> tcmu-runner dying and failingover in the multipath scenarios.
>>>>
>>>
>>> Prasanna did test that elasticsearch itself worked fine in gluster-block
>>> environment when all the machines are up etc i.e. success path. We are
>>> doing failure path testing and fixing things at the moment.
>>>
>>>
>>>>
>>>>
>>>>>
>>>>> On Fri, May 5, 2017 at 5:06 PM, Pranith Kumar Karampuri <
>>>>> pkarampu at redhat.com> wrote:
>>>>>
>>>>>> Abhijit we just started making the efforts to get all of this stable.
>>>>>>
>>>>>> On Fri, May 5, 2017 at 4:45 PM, Abhijit Paul <
>>>>>> er.abhijitpaul at gmail.com> wrote:
>>>>>>
>>>>>>> I yet to try gluster-block with elasticsearch...but carious to know
>>>>>>> does this combination plays well in kubernetes environment?
>>>>>>>
>>>>>>> On Fri, May 5, 2017 at 12:14 PM, Abhijit Paul <
>>>>>>> er.abhijitpaul at gmail.com> wrote:
>>>>>>>
>>>>>>>> thanks Krutika for the alternative.
>>>>>>>>
>>>>>>>> @*Prasanna @**Pranith*
>>>>>>>> I was going thorough the mentioned blog post and saw that used
>>>>>>>> tested environment was Fedora ,
>>>>>>>> but i am using RHEL based Oracle linux so does gluster-block
>>>>>>>> compatible with RHEL as well?
>>>>>>>>
>>>>>>>> On Fri, May 5, 2017 at 12:03 PM, Krutika Dhananjay <
>>>>>>>> kdhananj at redhat.com> wrote:
>>>>>>>>
>>>>>>>>> Yeah, there are a couple of cache consistency issues with
>>>>>>>>> performance translators that are causing these exceptions.
>>>>>>>>> Some of them were fixed by 3.10.1. Some still remain.
>>>>>>>>>
>>>>>>>>> Alternatively you can give gluster-block + elasticsearch a try,
>>>>>>>>> which doesn't require solving all these caching issues.
>>>>>>>>> Here's a blog post on the same - https://pkalever.wordpress.com
>>>>>>>>> /2017/03/14/elasticsearch-with-gluster-block/
>>>>>>>>>
>>>>>>>>> Adding Prasanna and Pranith who worked on this, in case you need
>>>>>>>>> more info on this.
>>>>>>>>>
>>>>>>>>> -Krutika
>>>>>>>>>
>>>>>>>>> On Fri, May 5, 2017 at 12:15 AM, Abhijit Paul <
>>>>>>>>> er.abhijitpaul at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Thanks for the reply, i will try it out but i am also facing one
>>>>>>>>>> more issue "i.e. replicated volumes returning different
>>>>>>>>>> timestamps"
>>>>>>>>>> so is this because of Bug 1426548 - Openshift Logging
>>>>>>>>>> ElasticSearch FSLocks when using GlusterFS storage backend
>>>>>>>>>> <https://bugzilla.redhat.com/show_bug.cgi?id=1426548> ?
>>>>>>>>>>
>>>>>>>>>> *FYI i am using glusterfs 3.10.1 tar.gz*
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Abhijit
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, May 4, 2017 at 10:58 PM, Amar Tumballi <
>>>>>>>>>> atumball at redhat.com> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, May 4, 2017 at 10:41 PM, Abhijit Paul <
>>>>>>>>>>> er.abhijitpaul at gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Since i am new to gluster, can please provide how to turn off/disable
>>>>>>>>>>>> "perf xlator options"?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> $ gluster volume set <volname> performance.stat-prefetch off
>>>>>>>>>>> $ gluster volume set <volname> performance.read-ahead off
>>>>>>>>>>> $ gluster volume set <volname> performance.write-behind off
>>>>>>>>>>> $ gluster volume set <volname> performance.io-cache off
>>>>>>>>>>> $ gluster volume set <volname> performance.quick-read off
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Amar
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, May 3, 2017 at 8:51 PM, Atin Mukherjee <
>>>>>>>>>>>>> amukherj at redhat.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I think there is still some pending stuffs in some of the
>>>>>>>>>>>>>> gluster perf xlators to make that work complete. Cced the relevant folks
>>>>>>>>>>>>>> for more information. Can you please turn off all the perf xlator options
>>>>>>>>>>>>>> as a work around to move forward?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, May 3, 2017 at 8:04 PM, Abhijit Paul <
>>>>>>>>>>>>>> er.abhijitpaul at gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Dear folks,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I setup Glusterfs(3.10.1) NFS type as persistence volume for
>>>>>>>>>>>>>>> Elasticsearch(5.1.2) but currently facing issue with *"CorruptIndexException"
>>>>>>>>>>>>>>> *with Elasticseach logs and due to that index health trued
>>>>>>>>>>>>>>> RED in Elasticsearch.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Later found that there was an issue with gluster < 3.10 (
>>>>>>>>>>>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=1390050) but
>>>>>>>>>>>>>>> even after *upgrading to 3.10.1 issue is still there.*
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> *So curios to know what would be the root cause to fix this
>>>>>>>>>>>>>>> issue.*
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> Abhijit
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>> Gluster-users mailing list
>>>>>>>>>>>>>>> Gluster-users at gluster.org
>>>>>>>>>>>>>>> http://lists.gluster.org/mailman/listinfo/gluster-users
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Gluster-users mailing list
>>>>>>>>>>>> Gluster-users at gluster.org
>>>>>>>>>>>> http://lists.gluster.org/mailman/listinfo/gluster-users
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Amar Tumballi (amarts)
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Gluster-users mailing list
>>>>>>>>>> Gluster-users at gluster.org
>>>>>>>>>> http://lists.gluster.org/mailman/listinfo/gluster-users
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Pranith
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Pranith
>>>>
>>>
>>>
>>>
>>> --
>>> Pranith
>>>
>>
>>
>

-- 
Pranith
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170508/8c6a130c/attachment.html>