[Gluster-users] Elasticsearch facing CorruptIndexException exception with GlusterFs 3.10.1

Mon May 8 09:32:43 UTC 2017

Abhijit,
         We released gluster-block v0.2 just this Friday for which RHEL
packages are yet to be built.

+Kaleb,
     Could you help with this please?

Prasanna,
     Could you let Abhijit know the rpm versions for tcmu-runner and other
packages so that this feature can be used?


On Mon, May 8, 2017 at 2:49 PM, Pranith Kumar Karampuri <pkarampu at redhat.com
> wrote:

> Wait, wait we are discussing this issue only. Expect a reply in some time
> :-)
>
> On Mon, May 8, 2017 at 2:19 PM, Abhijit Paul <er.abhijitpaul at gmail.com>
> wrote:
>
>> poking for previous mail reply
>>
>> On Sun, May 7, 2017 at 1:06 AM, Abhijit Paul <er.abhijitpaul at gmail.com>
>> wrote:
>>
>>>  https://pkalever.wordpress.com/2017/03/14/elasticsearch-wit
>>> h-gluster-block/
>>> here used tested environment is Fedora ,
>>> but i am using RHEL based Oracle linux so does gluster-block compatible
>>> with RHEL as well? What i needs to change & make it work?
>>>
>>> On Fri, May 5, 2017 at 5:42 PM, Pranith Kumar Karampuri <
>>> pkarampu at redhat.com> wrote:
>>>
>>>>
>>>>
>>>> On Fri, May 5, 2017 at 5:40 PM, Pranith Kumar Karampuri <
>>>> pkarampu at redhat.com> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Fri, May 5, 2017 at 5:36 PM, Abhijit Paul <er.abhijitpaul at gmail.com
>>>>> > wrote:
>>>>>
>>>>>> So should i start using gluster-block with elasticsearch in kubernetes
>>>>>> environment?
>>>>>>
>>>>>> My expectation from gluster-block is, it should not CorruptIndex
>>>>>> of elasticsearch...and issue facing in previous mails.
>>>>>>
>>>>>> Please let me know whether should i processed with above mentioned
>>>>>> combination.
>>>>>>
>>>>>
>>>>> We are still in the process of fixing the failure scenarios of
>>>>> tcmu-runner dying and failingover in the multipath scenarios.
>>>>>
>>>>
>>>> Prasanna did test that elasticsearch itself worked fine in
>>>> gluster-block environment when all the machines are up etc i.e. success
>>>> path. We are doing failure path testing and fixing things at the moment.
>>>>
>>>>
>>>>>
>>>>>
>>>>>>
>>>>>> On Fri, May 5, 2017 at 5:06 PM, Pranith Kumar Karampuri <
>>>>>> pkarampu at redhat.com> wrote:
>>>>>>
>>>>>>> Abhijit we just started making the efforts to get all of this stable.
>>>>>>>
>>>>>>> On Fri, May 5, 2017 at 4:45 PM, Abhijit Paul <
>>>>>>> er.abhijitpaul at gmail.com> wrote:
>>>>>>>
>>>>>>>> I yet to try gluster-block with elasticsearch...but carious to know
>>>>>>>> does this combination plays well in kubernetes environment?
>>>>>>>>
>>>>>>>> On Fri, May 5, 2017 at 12:14 PM, Abhijit Paul <
>>>>>>>> er.abhijitpaul at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> thanks Krutika for the alternative.
>>>>>>>>>
>>>>>>>>> @*Prasanna @**Pranith*
>>>>>>>>> I was going thorough the mentioned blog post and saw that used
>>>>>>>>> tested environment was Fedora ,
>>>>>>>>> but i am using RHEL based Oracle linux so does gluster-block
>>>>>>>>> compatible with RHEL as well?
>>>>>>>>>
>>>>>>>>> On Fri, May 5, 2017 at 12:03 PM, Krutika Dhananjay <
>>>>>>>>> kdhananj at redhat.com> wrote:
>>>>>>>>>
>>>>>>>>>> Yeah, there are a couple of cache consistency issues with
>>>>>>>>>> performance translators that are causing these exceptions.
>>>>>>>>>> Some of them were fixed by 3.10.1. Some still remain.
>>>>>>>>>>
>>>>>>>>>> Alternatively you can give gluster-block + elasticsearch a try,
>>>>>>>>>> which doesn't require solving all these caching issues.
>>>>>>>>>> Here's a blog post on the same - https://pkalever.wordpress.com
>>>>>>>>>> /2017/03/14/elasticsearch-with-gluster-block/
>>>>>>>>>>
>>>>>>>>>> Adding Prasanna and Pranith who worked on this, in case you need
>>>>>>>>>> more info on this.
>>>>>>>>>>
>>>>>>>>>> -Krutika
>>>>>>>>>>
>>>>>>>>>> On Fri, May 5, 2017 at 12:15 AM, Abhijit Paul <
>>>>>>>>>> er.abhijitpaul at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Thanks for the reply, i will try it out but i am also facing one
>>>>>>>>>>> more issue "i.e. replicated volumes returning different
>>>>>>>>>>> timestamps"
>>>>>>>>>>> so is this because of Bug 1426548 - Openshift Logging
>>>>>>>>>>> ElasticSearch FSLocks when using GlusterFS storage backend
>>>>>>>>>>> <https://bugzilla.redhat.com/show_bug.cgi?id=1426548> ?
>>>>>>>>>>>
>>>>>>>>>>> *FYI i am using glusterfs 3.10.1 tar.gz*
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Abhijit
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, May 4, 2017 at 10:58 PM, Amar Tumballi <
>>>>>>>>>>> atumball at redhat.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, May 4, 2017 at 10:41 PM, Abhijit Paul <
>>>>>>>>>>>> er.abhijitpaul at gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Since i am new to gluster, can please provide how to turn off/disable
>>>>>>>>>>>>> "perf xlator options"?
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>> $ gluster volume set <volname> performance.stat-prefetch off
>>>>>>>>>>>> $ gluster volume set <volname> performance.read-ahead off
>>>>>>>>>>>> $ gluster volume set <volname> performance.write-behind off
>>>>>>>>>>>> $ gluster volume set <volname> performance.io-cache off
>>>>>>>>>>>> $ gluster volume set <volname> performance.quick-read off
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Amar
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, May 3, 2017 at 8:51 PM, Atin Mukherjee <
>>>>>>>>>>>>>> amukherj at redhat.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I think there is still some pending stuffs in some of the
>>>>>>>>>>>>>>> gluster perf xlators to make that work complete. Cced the relevant folks
>>>>>>>>>>>>>>> for more information. Can you please turn off all the perf xlator options
>>>>>>>>>>>>>>> as a work around to move forward?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, May 3, 2017 at 8:04 PM, Abhijit Paul <
>>>>>>>>>>>>>>> er.abhijitpaul at gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Dear folks,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I setup Glusterfs(3.10.1) NFS type as persistence volume
>>>>>>>>>>>>>>>> for Elasticsearch(5.1.2) but currently facing issue with *"CorruptIndexException"
>>>>>>>>>>>>>>>> *with Elasticseach logs and due to that index health trued
>>>>>>>>>>>>>>>> RED in Elasticsearch.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Later found that there was an issue with gluster < 3.10 (
>>>>>>>>>>>>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=1390050) but
>>>>>>>>>>>>>>>> even after *upgrading to 3.10.1 issue is still there.*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> *So curios to know what would be the root cause to fix this
>>>>>>>>>>>>>>>> issue.*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>> Abhijit
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>> Gluster-users mailing list
>>>>>>>>>>>>>>>> Gluster-users at gluster.org
>>>>>>>>>>>>>>>> http://lists.gluster.org/mailman/listinfo/gluster-users
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> Gluster-users mailing list
>>>>>>>>>>>>> Gluster-users at gluster.org
>>>>>>>>>>>>> http://lists.gluster.org/mailman/listinfo/gluster-users
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Amar Tumballi (amarts)
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Gluster-users mailing list
>>>>>>>>>>> Gluster-users at gluster.org
>>>>>>>>>>> http://lists.gluster.org/mailman/listinfo/gluster-users
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Pranith
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Pranith
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Pranith
>>>>
>>>
>>>
>>
>
>
> --
> Pranith
>


-- 
Pranith
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170508/3efedac1/attachment.html>