[heketi-devel] Working Heketi

Christopher Schmidt fakod666 at gmail.com
Tue Aug 29 06:22:34 UTC 2017


OK, tried gluster/gluster-centos:latest with turned off all performance
translators:
----------------------------
I cannot see that much lock issues in the glusterd logs as before (so this
image is somehow different and uses 3.10.2)
The ES operator successfully creates a stateful set with 3 distributed
volumes, but "gluster volume list" shows 10 volumes.
3 started, 1 stopped, 6 created. So I assume something sill goes wrong,
right?


gluster/gluster-centos:latest with turned on all performance translators:
-----------------------------
much faster volume creation. Created 3 distributed volumes and "gluster
volume list" reports 3 ones started.
fine.



I have two questions now:
the scripts in gluster-containers seem to create different images than in
gluster/gluster-centos:latest (or is this just a version issue?)
I did not want to refer to a "latest" tagged image, thats why i've created
my own one (without changing any Dockerfile or script)
So what is the recommendation here?

The performance translators seem to be a bit tricky.
Disabling all results in non working dynamic provisioned K8s volumes (as
long as orphaned created gluster volumes are an issue).
Enabling all has issues f.e. with Kafka or ElasticSearch.
So, how can I find a working set of translators? Testing?


thanks for helping anyway, Christopher


Jose A. Rivera <jarrpa at redhat.com> schrieb am Mo., 28. Aug. 2017 um
18:58 Uhr:

> Yes, please try that. :)
>
> On Mon, Aug 28, 2017 at 8:33 AM, Christopher Schmidt <fakod666 at gmail.com>
> wrote:
> >
> >
> > Jose A. Rivera <jarrpa at redhat.com> schrieb am Mo., 28. Aug. 2017 um
> 14:01
> > Uhr:
> >>
> >> Hey Christopher,
> >>
> >> On Mon, Aug 28, 2017 at 12:37 AM, Christopher Schmidt
> >> <fakod666 at gmail.com> wrote:
> >> > Hi,
> >> >
> >> > sorry for the confusion.
> >> > I opened this thread, to find an overall working configuration.
> >> >
> >> > My setup is as follows:
> >> > - Container Linux (CoreOS)
> >> > - No local installation (except Kubelet, everything (!) else runs in
> >> > Containers)
> >> > - I am using this deploy scripts for GlusterFS
> >> >
> >> >
> https://github.com/gluster/gluster-kubernetes/tree/master/deploy/kube-templates
> >> > - I've created my own containers with
> >> > https://github.com/gluster/gluster-containers/tree/master/CentOS (so
> its
> >> > Gluster 1.10.5)
> >> > - I am using the original container from docker hub heketi/heketi:5
> >> > (with a
> >> > secret as DB backup, not the gk-deploy script)
> >> > - Kubernetes v1.6.8_coreos.0 with 3 masters and 6 minions, gluster
> pods
> >> > on
> >> > all minions (3 x 50GB partitions for Gluster on each node, rbac
> enabled)
> >> > - for network policies I use Canal (deployed as pods as well)
> >> >
> >> > Issues:
> >> > - I've learned that Kubernetes 1.7.x and GlusterFS is broken, since
> 1.7
> >> > requires GlusterFS > 1.10, right?
> >>
> >> I imagine you mean GlusterFS 3.10. :) And no, the issue is because the
> >> current GlusterFS container uses systemd, which conflicts with the new
> >> shared PIC namespace feature in recent versions of Kube and Docker:
> >>
> >>
> https://github.com/kubernetes/kubernetes/issues/48937#issuecomment-319957314
> >
> >
> > oh sure 3.10.. So this fixes also the unsupported mount option I've had?
> >
> >>
> >>
> >>
> >> > - I was testing K8s 1.5.x with Heketi/GlusterFS and it was fine for me
> >> > (but
> >> > really only small manual tests a couple of months ago)
> >> > - With my configuration (see above) no version of Gluster or Heketi
> >> > works
> >> > with K8s 1.6.x with dynamic provisioning.
> >> >
> >> > However, this is not the usual GlusterFS setup you are testing. So I
> am
> >> > sure
> >> > I do something wrong. What is noticeable is that creating a volume
> takes
> >> > a
> >> > couple of minutes (depending on the volume type) and Heketi has some
> >> > issues
> >> > with that.
> >> > So what can I do? Where to start testing/changing my configuration?
> >>
> >> Nothing looks immediately wrong to me. Have you tried the same
> >> container images that we use in gluster-kubernetes? The only other
> >> difference I note is that you're using Canal whereas out test setups
> >> use weave-net.
> >
> >
> > "No", I've build my own image. But all I did is to "docker build -t
> > innoq/gluster-centos:gluster3u10_centos7-3.10.5 ."
> > from here:
> https://github.com/gluster/gluster-containers/tree/master/CentOS
> >
> > anyway, you mean I should try image: gluster/gluster-centos:latest?
> >
> >>
> >>
> >> --Jose
> >>
> >> > regards Christopher
> >> >
> >> >
> >> > Humble Chirammal <hchiramm at redhat.com> schrieb am Di., 22. Aug. 2017
> um
> >> > 09:51 Uhr:
> >> >>
> >> >> On Tue, Aug 22, 2017 at 1:14 PM, Michael Adam <obnox at samba.org>
> wrote:
> >> >>>
> >> >>> On 2017-08-22 at 07:00 +0000, Christopher Schmidt wrote:
> >> >>> > Hi guys,
> >> >>> >
> >> >>> > I know it's difficult to support various installations and
> >> >>> > combinations
> >> >>> > of
> >> >>> > tools.
> >> >>> > But I'll give some Kubernetes training and talks in
> >> >>> > September/October.
> >> >>> > So what should I recommend? So far I didn't find a working
> GlusterFS
> >> >>> > Heketi, neither with Kubernetes 1.6.x nor 1.7.x
> >> >>>
> >> >>> Hi Christopher,
> >> >>>
> >> >>> This statement seems surprising to me.
> >> >>> The vagrant environment and the complex tests
> >> >>> in the gluster-kubernetes repo
> >> >>> (github.com/gluster/gluster-kubernetes)
> >> >>> prove that heketi and gluster work in kube 1.6.
> >> >>>
> >> >>> Can you give some more details on what's going wrong?
> >> >>
> >> >>
> >> >> There is another thread opened by Christopher
> >> >>
> http://lists.gluster.org/pipermail/heketi-devel/2017-August/000166.html
> >> >>
> >> >>
> >> >>>
> >> >>>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Cheers,
> >> >> Humble
> >> >>
> >> >>
> >> >
> >> > _______________________________________________
> >> > heketi-devel mailing list
> >> > heketi-devel at gluster.org
> >> > http://lists.gluster.org/mailman/listinfo/heketi-devel
> >> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/heketi-devel/attachments/20170829/b6b97b31/attachment-0001.html>


More information about the heketi-devel mailing list