[heketi-devel] Working Heketi

[Jose A. Rivera]

Yes, please try that. :)

On Mon, Aug 28, 2017 at 8:33 AM, Christopher Schmidt <fakod666 at gmail.com> wrote:
>
>
> Jose A. Rivera <jarrpa at redhat.com> schrieb am Mo., 28. Aug. 2017 um 14:01
> Uhr:
>>
>> Hey Christopher,
>>
>> On Mon, Aug 28, 2017 at 12:37 AM, Christopher Schmidt
>> <fakod666 at gmail.com> wrote:
>> > Hi,
>> >
>> > sorry for the confusion.
>> > I opened this thread, to find an overall working configuration.
>> >
>> > My setup is as follows:
>> > - Container Linux (CoreOS)
>> > - No local installation (except Kubelet, everything (!) else runs in
>> > Containers)
>> > - I am using this deploy scripts for GlusterFS
>> >
>> > https://github.com/gluster/gluster-kubernetes/tree/master/deploy/kube-templates
>> > - I've created my own containers with
>> > https://github.com/gluster/gluster-containers/tree/master/CentOS (so its
>> > Gluster 1.10.5)
>> > - I am using the original container from docker hub heketi/heketi:5
>> > (with a
>> > secret as DB backup, not the gk-deploy script)
>> > - Kubernetes v1.6.8_coreos.0 with 3 masters and 6 minions, gluster pods
>> > on
>> > all minions (3 x 50GB partitions for Gluster on each node, rbac enabled)
>> > - for network policies I use Canal (deployed as pods as well)
>> >
>> > Issues:
>> > - I've learned that Kubernetes 1.7.x and GlusterFS is broken, since 1.7
>> > requires GlusterFS > 1.10, right?
>>
>> I imagine you mean GlusterFS 3.10. :) And no, the issue is because the
>> current GlusterFS container uses systemd, which conflicts with the new
>> shared PIC namespace feature in recent versions of Kube and Docker:
>>
>> https://github.com/kubernetes/kubernetes/issues/48937#issuecomment-319957314
>
>
> oh sure 3.10.. So this fixes also the unsupported mount option I've had?
>
>>
>>
>>
>> > - I was testing K8s 1.5.x with Heketi/GlusterFS and it was fine for me
>> > (but
>> > really only small manual tests a couple of months ago)
>> > - With my configuration (see above) no version of Gluster or Heketi
>> > works
>> > with K8s 1.6.x with dynamic provisioning.
>> >
>> > However, this is not the usual GlusterFS setup you are testing. So I am
>> > sure
>> > I do something wrong. What is noticeable is that creating a volume takes
>> > a
>> > couple of minutes (depending on the volume type) and Heketi has some
>> > issues
>> > with that.
>> > So what can I do? Where to start testing/changing my configuration?
>>
>> Nothing looks immediately wrong to me. Have you tried the same
>> container images that we use in gluster-kubernetes? The only other
>> difference I note is that you're using Canal whereas out test setups
>> use weave-net.
>
>
> "No", I've build my own image. But all I did is to "docker build -t
> innoq/gluster-centos:gluster3u10_centos7-3.10.5 ."
> from here: https://github.com/gluster/gluster-containers/tree/master/CentOS
>
> anyway, you mean I should try image: gluster/gluster-centos:latest?
>
>>
>>
>> --Jose
>>
>> > regards Christopher
>> >
>> >
>> > Humble Chirammal <hchiramm at redhat.com> schrieb am Di., 22. Aug. 2017 um
>> > 09:51 Uhr:
>> >>
>> >> On Tue, Aug 22, 2017 at 1:14 PM, Michael Adam <obnox at samba.org> wrote:
>> >>>
>> >>> On 2017-08-22 at 07:00 +0000, Christopher Schmidt wrote:
>> >>> > Hi guys,
>> >>> >
>> >>> > I know it's difficult to support various installations and
>> >>> > combinations
>> >>> > of
>> >>> > tools.
>> >>> > But I'll give some Kubernetes training and talks in
>> >>> > September/October.
>> >>> > So what should I recommend? So far I didn't find a working GlusterFS
>> >>> > Heketi, neither with Kubernetes 1.6.x nor 1.7.x
>> >>>
>> >>> Hi Christopher,
>> >>>
>> >>> This statement seems surprising to me.
>> >>> The vagrant environment and the complex tests
>> >>> in the gluster-kubernetes repo
>> >>> (github.com/gluster/gluster-kubernetes)
>> >>> prove that heketi and gluster work in kube 1.6.
>> >>>
>> >>> Can you give some more details on what's going wrong?
>> >>
>> >>
>> >> There is another thread opened by Christopher
>> >> http://lists.gluster.org/pipermail/heketi-devel/2017-August/000166.html
>> >>
>> >>
>> >>>
>> >>>
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Cheers,
>> >> Humble
>> >>
>> >>
>> >
>> > _______________________________________________
>> > heketi-devel mailing list
>> > heketi-devel at gluster.org
>> > http://lists.gluster.org/mailman/listinfo/heketi-devel
>> >