[heketi-devel] Working Heketi

Tue Aug 29 07:53:19 UTC 2017

Just for the sake of completeness. This is one of the exceptions after a
couple of minutes pushing logs into a 3 node ES cluster..

org.apache.lucene.index.CorruptIndexException: compound sub-files must have
a valid codec header and footer: file is too small (0 bytes)
(resource=BufferedChecksumIndexInput(MMapIndexInput(path="/data/data/nodes/0/indices/PVlM9sYYT6akalR-ZN-BpQ/2/index/_6b.fdx")))

Christopher Schmidt <fakod666 at gmail.com> schrieb am Di., 29. Aug. 2017 um
08:22 Uhr:

> OK, tried gluster/gluster-centos:latest with turned off all performance
> translators:
> ----------------------------
> I cannot see that much lock issues in the glusterd logs as before (so this
> image is somehow different and uses 3.10.2)
> The ES operator successfully creates a stateful set with 3 distributed
> volumes, but "gluster volume list" shows 10 volumes.
> 3 started, 1 stopped, 6 created. So I assume something sill goes wrong,
> right?
>
>
> gluster/gluster-centos:latest with turned on all performance translators:
> -----------------------------
> much faster volume creation. Created 3 distributed volumes and "gluster
> volume list" reports 3 ones started.
> fine.
>
>
>
> I have two questions now:
> the scripts in gluster-containers seem to create different images than in
> gluster/gluster-centos:latest (or is this just a version issue?)
> I did not want to refer to a "latest" tagged image, thats why i've created
> my own one (without changing any Dockerfile or script)
> So what is the recommendation here?
>
> The performance translators seem to be a bit tricky.
> Disabling all results in non working dynamic provisioned K8s volumes (as
> long as orphaned created gluster volumes are an issue).
> Enabling all has issues f.e. with Kafka or ElasticSearch.
> So, how can I find a working set of translators? Testing?
>
>
> thanks for helping anyway, Christopher
>
>
> Jose A. Rivera <jarrpa at redhat.com> schrieb am Mo., 28. Aug. 2017 um
> 18:58 Uhr:
>
>> Yes, please try that. :)
>>
>> On Mon, Aug 28, 2017 at 8:33 AM, Christopher Schmidt <fakod666 at gmail.com>
>> wrote:
>> >
>> >
>> > Jose A. Rivera <jarrpa at redhat.com> schrieb am Mo., 28. Aug. 2017 um
>> 14:01
>> > Uhr:
>> >>
>> >> Hey Christopher,
>> >>
>> >> On Mon, Aug 28, 2017 at 12:37 AM, Christopher Schmidt
>> >> <fakod666 at gmail.com> wrote:
>> >> > Hi,
>> >> >
>> >> > sorry for the confusion.
>> >> > I opened this thread, to find an overall working configuration.
>> >> >
>> >> > My setup is as follows:
>> >> > - Container Linux (CoreOS)
>> >> > - No local installation (except Kubelet, everything (!) else runs in
>> >> > Containers)
>> >> > - I am using this deploy scripts for GlusterFS
>> >> >
>> >> >
>> https://github.com/gluster/gluster-kubernetes/tree/master/deploy/kube-templates
>> >> > - I've created my own containers with
>> >> > https://github.com/gluster/gluster-containers/tree/master/CentOS
>> (so its
>> >> > Gluster 1.10.5)
>> >> > - I am using the original container from docker hub heketi/heketi:5
>> >> > (with a
>> >> > secret as DB backup, not the gk-deploy script)
>> >> > - Kubernetes v1.6.8_coreos.0 with 3 masters and 6 minions, gluster
>> pods
>> >> > on
>> >> > all minions (3 x 50GB partitions for Gluster on each node, rbac
>> enabled)
>> >> > - for network policies I use Canal (deployed as pods as well)
>> >> >
>> >> > Issues:
>> >> > - I've learned that Kubernetes 1.7.x and GlusterFS is broken, since
>> 1.7
>> >> > requires GlusterFS > 1.10, right?
>> >>
>> >> I imagine you mean GlusterFS 3.10. :) And no, the issue is because the
>> >> current GlusterFS container uses systemd, which conflicts with the new
>> >> shared PIC namespace feature in recent versions of Kube and Docker:
>> >>
>> >>
>> https://github.com/kubernetes/kubernetes/issues/48937#issuecomment-319957314
>> >
>> >
>> > oh sure 3.10.. So this fixes also the unsupported mount option I've had?
>> >
>> >>
>> >>
>> >>
>> >> > - I was testing K8s 1.5.x with Heketi/GlusterFS and it was fine for
>> me
>> >> > (but
>> >> > really only small manual tests a couple of months ago)
>> >> > - With my configuration (see above) no version of Gluster or Heketi
>> >> > works
>> >> > with K8s 1.6.x with dynamic provisioning.
>> >> >
>> >> > However, this is not the usual GlusterFS setup you are testing. So I
>> am
>> >> > sure
>> >> > I do something wrong. What is noticeable is that creating a volume
>> takes
>> >> > a
>> >> > couple of minutes (depending on the volume type) and Heketi has some
>> >> > issues
>> >> > with that.
>> >> > So what can I do? Where to start testing/changing my configuration?
>> >>
>> >> Nothing looks immediately wrong to me. Have you tried the same
>> >> container images that we use in gluster-kubernetes? The only other
>> >> difference I note is that you're using Canal whereas out test setups
>> >> use weave-net.
>> >
>> >
>> > "No", I've build my own image. But all I did is to "docker build -t
>> > innoq/gluster-centos:gluster3u10_centos7-3.10.5 ."
>> > from here:
>> https://github.com/gluster/gluster-containers/tree/master/CentOS
>> >
>> > anyway, you mean I should try image: gluster/gluster-centos:latest?
>> >
>> >>
>> >>
>> >> --Jose
>> >>
>> >> > regards Christopher
>> >> >
>> >> >
>> >> > Humble Chirammal <hchiramm at redhat.com> schrieb am Di., 22. Aug.
>> 2017 um
>> >> > 09:51 Uhr:
>> >> >>
>> >> >> On Tue, Aug 22, 2017 at 1:14 PM, Michael Adam <obnox at samba.org>
>> wrote:
>> >> >>>
>> >> >>> On 2017-08-22 at 07:00 +0000, Christopher Schmidt wrote:
>> >> >>> > Hi guys,
>> >> >>> >
>> >> >>> > I know it's difficult to support various installations and
>> >> >>> > combinations
>> >> >>> > of
>> >> >>> > tools.
>> >> >>> > But I'll give some Kubernetes training and talks in
>> >> >>> > September/October.
>> >> >>> > So what should I recommend? So far I didn't find a working
>> GlusterFS
>> >> >>> > Heketi, neither with Kubernetes 1.6.x nor 1.7.x
>> >> >>>
>> >> >>> Hi Christopher,
>> >> >>>
>> >> >>> This statement seems surprising to me.
>> >> >>> The vagrant environment and the complex tests
>> >> >>> in the gluster-kubernetes repo
>> >> >>> (github.com/gluster/gluster-kubernetes)
>> >> >>> prove that heketi and gluster work in kube 1.6.
>> >> >>>
>> >> >>> Can you give some more details on what's going wrong?
>> >> >>
>> >> >>
>> >> >> There is another thread opened by Christopher
>> >> >>
>> http://lists.gluster.org/pipermail/heketi-devel/2017-August/000166.html
>> >> >>
>> >> >>
>> >> >>>
>> >> >>>
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Cheers,
>> >> >> Humble
>> >> >>
>> >> >>
>> >> >
>> >> > _______________________________________________
>> >> > heketi-devel mailing list
>> >> > heketi-devel at gluster.org
>> >> > http://lists.gluster.org/mailman/listinfo/heketi-devel
>> >> >
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/heketi-devel/attachments/20170829/84fa0f33/attachment.html>