[heketi-devel] Working Heketi

[Christopher Schmidt]

Jose A. Rivera <jarrpa at redhat.com> schrieb am Mo., 28. Aug. 2017 um
14:01 Uhr:

> Hey Christopher,
>
> On Mon, Aug 28, 2017 at 12:37 AM, Christopher Schmidt
> <fakod666 at gmail.com> wrote:
> > Hi,
> >
> > sorry for the confusion.
> > I opened this thread, to find an overall working configuration.
> >
> > My setup is as follows:
> > - Container Linux (CoreOS)
> > - No local installation (except Kubelet, everything (!) else runs in
> > Containers)
> > - I am using this deploy scripts for GlusterFS
> >
> https://github.com/gluster/gluster-kubernetes/tree/master/deploy/kube-templates
> > - I've created my own containers with
> > https://github.com/gluster/gluster-containers/tree/master/CentOS (so its
> > Gluster 1.10.5)
> > - I am using the original container from docker hub heketi/heketi:5
> (with a
> > secret as DB backup, not the gk-deploy script)
> > - Kubernetes v1.6.8_coreos.0 with 3 masters and 6 minions, gluster pods
> on
> > all minions (3 x 50GB partitions for Gluster on each node, rbac enabled)
> > - for network policies I use Canal (deployed as pods as well)
> >
> > Issues:
> > - I've learned that Kubernetes 1.7.x and GlusterFS is broken, since 1.7
> > requires GlusterFS > 1.10, right?
>
> I imagine you mean GlusterFS 3.10. :) And no, the issue is because the
> current GlusterFS container uses systemd, which conflicts with the new
> shared PIC namespace feature in recent versions of Kube and Docker:
>
> https://github.com/kubernetes/kubernetes/issues/48937#issuecomment-319957314


oh sure 3.10.. So this fixes also the unsupported mount option I've had?


>
>
> > - I was testing K8s 1.5.x with Heketi/GlusterFS and it was fine for me
> (but
> > really only small manual tests a couple of months ago)
> > - With my configuration (see above) no version of Gluster or Heketi works
> > with K8s 1.6.x with dynamic provisioning.
> >
> > However, this is not the usual GlusterFS setup you are testing. So I am
> sure
> > I do something wrong. What is noticeable is that creating a volume takes
> a
> > couple of minutes (depending on the volume type) and Heketi has some
> issues
> > with that.
> > So what can I do? Where to start testing/changing my configuration?
>
> Nothing looks immediately wrong to me. Have you tried the same
> container images that we use in gluster-kubernetes? The only other
> difference I note is that you're using Canal whereas out test setups
> use weave-net.
>

"No", I've build my own image. But all I did is to "docker build -t
innoq/gluster-centos:gluster3u10_centos7-3.10.5 ."
from here: https://github.com/gluster/gluster-containers/tree/master/CentOS

anyway, you mean I should try image: gluster/gluster-centos:latest?


>
> --Jose
>
> > regards Christopher
> >
> >
> > Humble Chirammal <hchiramm at redhat.com> schrieb am Di., 22. Aug. 2017 um
> > 09:51 Uhr:
> >>
> >> On Tue, Aug 22, 2017 at 1:14 PM, Michael Adam <obnox at samba.org> wrote:
> >>>
> >>> On 2017-08-22 at 07:00 +0000, Christopher Schmidt wrote:
> >>> > Hi guys,
> >>> >
> >>> > I know it's difficult to support various installations and
> combinations
> >>> > of
> >>> > tools.
> >>> > But I'll give some Kubernetes training and talks in
> September/October.
> >>> > So what should I recommend? So far I didn't find a working GlusterFS
> >>> > Heketi, neither with Kubernetes 1.6.x nor 1.7.x
> >>>
> >>> Hi Christopher,
> >>>
> >>> This statement seems surprising to me.
> >>> The vagrant environment and the complex tests
> >>> in the gluster-kubernetes repo
> >>> (github.com/gluster/gluster-kubernetes)
> >>> prove that heketi and gluster work in kube 1.6.
> >>>
> >>> Can you give some more details on what's going wrong?
> >>
> >>
> >> There is another thread opened by Christopher
> >> http://lists.gluster.org/pipermail/heketi-devel/2017-August/000166.html
> >>
> >>
> >>>
> >>>
> >>
> >>
> >>
> >>
> >> --
> >> Cheers,
> >> Humble
> >>
> >>
> >
> > _______________________________________________
> > heketi-devel mailing list
> > heketi-devel at gluster.org
> > http://lists.gluster.org/mailman/listinfo/heketi-devel
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/heketi-devel/attachments/20170828/5a4edb31/attachment.html>