<div dir="ltr"><br><br><div class="gmail_quote"><div dir="ltr">Jose A. Rivera <<a href="mailto:jarrpa@redhat.com">jarrpa@redhat.com</a>> schrieb am Mo., 28. Aug. 2017 um 14:01 Uhr:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hey Christopher,<br>
<br>
On Mon, Aug 28, 2017 at 12:37 AM, Christopher Schmidt<br>
<<a href="mailto:fakod666@gmail.com" target="_blank">fakod666@gmail.com</a>> wrote:<br>
> Hi,<br>
><br>
> sorry for the confusion.<br>
> I opened this thread, to find an overall working configuration.<br>
><br>
> My setup is as follows:<br>
> - Container Linux (CoreOS)<br>
> - No local installation (except Kubelet, everything (!) else runs in<br>
> Containers)<br>
> - I am using this deploy scripts for GlusterFS<br>
> <a href="https://github.com/gluster/gluster-kubernetes/tree/master/deploy/kube-templates" rel="noreferrer" target="_blank">https://github.com/gluster/gluster-kubernetes/tree/master/deploy/kube-templates</a><br>
> - I've created my own containers with<br>
> <a href="https://github.com/gluster/gluster-containers/tree/master/CentOS" rel="noreferrer" target="_blank">https://github.com/gluster/gluster-containers/tree/master/CentOS</a> (so its<br>
> Gluster 1.10.5)<br>
> - I am using the original container from docker hub heketi/heketi:5 (with a<br>
> secret as DB backup, not the gk-deploy script)<br>
> - Kubernetes v1.6.8_coreos.0 with 3 masters and 6 minions, gluster pods on<br>
> all minions (3 x 50GB partitions for Gluster on each node, rbac enabled)<br>
> - for network policies I use Canal (deployed as pods as well)<br>
><br>
> Issues:<br>
> - I've learned that Kubernetes 1.7.x and GlusterFS is broken, since 1.7<br>
> requires GlusterFS > 1.10, right?<br>
<br>
I imagine you mean GlusterFS 3.10. :) And no, the issue is because the<br>
current GlusterFS container uses systemd, which conflicts with the new<br>
shared PIC namespace feature in recent versions of Kube and Docker:<br>
<a href="https://github.com/kubernetes/kubernetes/issues/48937#issuecomment-319957314" rel="noreferrer" target="_blank">https://github.com/kubernetes/kubernetes/issues/48937#issuecomment-319957314</a></blockquote><div><br></div><div>oh sure 3.10.. So this fixes also the unsupported mount option I've had?</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
<br>
> - I was testing K8s 1.5.x with Heketi/GlusterFS and it was fine for me (but<br>
> really only small manual tests a couple of months ago)<br>
> - With my configuration (see above) no version of Gluster or Heketi works<br>
> with K8s 1.6.x with dynamic provisioning.<br>
><br>
> However, this is not the usual GlusterFS setup you are testing. So I am sure<br>
> I do something wrong. What is noticeable is that creating a volume takes a<br>
> couple of minutes (depending on the volume type) and Heketi has some issues<br>
> with that.<br>
> So what can I do? Where to start testing/changing my configuration?<br>
<br>
Nothing looks immediately wrong to me. Have you tried the same<br>
container images that we use in gluster-kubernetes? The only other<br>
difference I note is that you're using Canal whereas out test setups<br>
use weave-net.<br></blockquote><div><br></div><div>"No", I've build my own image. But all I did is to "docker build -t innoq/gluster-centos:gluster3u10_centos7-3.10.5 ."</div><div>from here: <a href="https://github.com/gluster/gluster-containers/tree/master/CentOS">https://github.com/gluster/gluster-containers/tree/master/CentOS</a></div><div><br></div><div>anyway, you mean I should try image: gluster/gluster-centos:latest?</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
--Jose<br>
<br>
> regards Christopher<br>
><br>
><br>
> Humble Chirammal <<a href="mailto:hchiramm@redhat.com" target="_blank">hchiramm@redhat.com</a>> schrieb am Di., 22. Aug. 2017 um<br>
> 09:51 Uhr:<br>
>><br>
>> On Tue, Aug 22, 2017 at 1:14 PM, Michael Adam <<a href="mailto:obnox@samba.org" target="_blank">obnox@samba.org</a>> wrote:<br>
>>><br>
>>> On 2017-08-22 at 07:00 +0000, Christopher Schmidt wrote:<br>
>>> > Hi guys,<br>
>>> ><br>
>>> > I know it's difficult to support various installations and combinations<br>
>>> > of<br>
>>> > tools.<br>
>>> > But I'll give some Kubernetes training and talks in September/October.<br>
>>> > So what should I recommend? So far I didn't find a working GlusterFS<br>
>>> > Heketi, neither with Kubernetes 1.6.x nor 1.7.x<br>
>>><br>
>>> Hi Christopher,<br>
>>><br>
>>> This statement seems surprising to me.<br>
>>> The vagrant environment and the complex tests<br>
>>> in the gluster-kubernetes repo<br>
>>> (<a href="http://github.com/gluster/gluster-kubernetes" rel="noreferrer" target="_blank">github.com/gluster/gluster-kubernetes</a>)<br>
>>> prove that heketi and gluster work in kube 1.6.<br>
>>><br>
>>> Can you give some more details on what's going wrong?<br>
>><br>
>><br>
>> There is another thread opened by Christopher<br>
>> <a href="http://lists.gluster.org/pipermail/heketi-devel/2017-August/000166.html" rel="noreferrer" target="_blank">http://lists.gluster.org/pipermail/heketi-devel/2017-August/000166.html</a><br>
>><br>
>><br>
>>><br>
>>><br>
>><br>
>><br>
>><br>
>><br>
>> --<br>
>> Cheers,<br>
>> Humble<br>
>><br>
>><br>
><br>
> _______________________________________________<br>
> heketi-devel mailing list<br>
> <a href="mailto:heketi-devel@gluster.org" target="_blank">heketi-devel@gluster.org</a><br>
> <a href="http://lists.gluster.org/mailman/listinfo/heketi-devel" rel="noreferrer" target="_blank">http://lists.gluster.org/mailman/listinfo/heketi-devel</a><br>
><br>
</blockquote></div></div>