<div dir="ltr"><div class="gmail_default" style="font-family:verdana,sans-serif">Hi,<br></div><div class="gmail_quote"><div dir="ltr"><div class="gmail_quote"><div dir="ltr"><div style="font-family:verdana,sans-serif"><br></div><div style="font-family:verdana,sans-serif"><br></div><div style="font-family:verdana,sans-serif">I have the following setup in place:</div><div style="font-family:verdana,sans-serif"><br></div><div style="font-family:verdana,sans-serif">1 node : RancherOS having Rancher application for Kubernetes setup </div><div style="font-family:verdana,sans-serif">2 nodes : RancherOS having Rancher agent<br></div><div style="font-family:verdana,sans-serif">1 node : CentOS 7 workstation having kubectl installed and folder cloned/downloaded from <a href="https://github.com/gluster/gluster-kubernetes" target="_blank">https://github.com/gluste<wbr>r/gluster-kubernetes</a> using which i run Heketi setup (gk-deploy -g)</div><div style="font-family:verdana,sans-serif"><br></div><div style="font-family:verdana,sans-serif">I also have rancher-glusterfs-server container running with the following configuration:</div><div><div><span style="font-family:"trebuchet ms",sans-serif">------------------------------<wbr>------------------------------<wbr>------</span><font face="trebuchet ms, sans-serif"><br></font></div><div><font face="trebuchet ms, sans-serif">[root@node-1 rancher]# cat gluster-server.sh</font></div><div><font face="trebuchet ms, sans-serif">#!/bin/bash</font></div><div><font face="trebuchet ms, sans-serif"><br></font></div><div><font face="trebuchet ms, sans-serif">sudo docker run --name=gluster-server -d \</font></div><div><font face="trebuchet ms, sans-serif"> --env 'SERVICE_NAME=gluster' \</font></div><div><font face="trebuchet ms, sans-serif"> --restart always \</font></div><div><font face="trebuchet ms, sans-serif"> --env 'GLUSTER_DATA=/srv/docker/gitl<wbr>ab' \</font></div><div><font face="trebuchet ms, sans-serif"> --publish 2222:22 \</font></div><div><font face="trebuchet ms, sans-serif"> webcenter/rancher-glusterfs-se<wbr>rver</font></div></div><div style="font-family:verdana,sans-serif"><span style="font-family:"trebuchet ms",sans-serif">------------------------------<wbr>------------------------------<wbr>------</span><br></div><div style="font-family:verdana,sans-serif"><span style="font-family:"trebuchet ms",sans-serif"><br></span></div><div style="font-family:verdana,sans-serif">In /etc/heketi/heketi.json, following is the only modified portion:</div><div><div><font face="trebuchet ms, sans-serif">------------------------------<wbr>------------------------------<wbr>------</font></div><div><font face="trebuchet ms, sans-serif"> "executor": "ssh",</font></div><div><font face="trebuchet ms, sans-serif"><br></font></div><div><font face="trebuchet ms, sans-serif"> "_sshexec_comment": "SSH username and private key file information",</font></div><div><font face="trebuchet ms, sans-serif"> "sshexec": {</font></div><div><font face="trebuchet ms, sans-serif"> "keyfile": "/var/lib/heketi/.ssh/id_rsa",</font></div><div><font face="trebuchet ms, sans-serif"> "user": "root",</font></div><div><font face="trebuchet ms, sans-serif"> "port": "22",</font></div><div><font face="trebuchet ms, sans-serif"> "fstab": "/etc/fstab"</font></div><div><font face="trebuchet ms, sans-serif"> },</font></div><div style="font-family:verdana,sans-serif"><span style="font-family:"trebuchet ms",sans-serif">------------------------------<wbr>------------------------------<wbr>------</span><br></div></div><div style="font-family:verdana,sans-serif"><br></div><div style="font-family:verdana,sans-serif">Status before running gk-deploy:</div><div style="font-family:verdana,sans-serif"><br></div><div><div><font face="trebuchet ms, sans-serif">[root@workstation deploy]# kubectl get nodes,pods,services,deployment<wbr>s</font></div><div><font face="trebuchet ms, sans-serif">NAME STATUS AGE VERSION</font></div><div><font face="trebuchet ms, sans-serif">no/node-1.c.kubernetes-174104.<wbr>internal Ready 2d v1.7.2-rancher1</font></div><div><font face="trebuchet ms, sans-serif">no/node-2.c.kubernetes-174104.<wbr>internal Ready 2d v1.7.2-rancher1</font></div><div><font face="trebuchet ms, sans-serif">no/node-3.c.kubernetes-174104.<wbr>internal Ready 2d v1.7.2-rancher1</font></div><div><font face="trebuchet ms, sans-serif"><br></font></div><div><font face="trebuchet ms, sans-serif">NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE</font></div><div><font face="trebuchet ms, sans-serif">svc/kubernetes 10.43.0.1 <none> 443/TCP 2d</font></div><div><font face="verdana, sans-serif"><br></font></div></div><div style="font-family:verdana,sans-serif"><br></div><div style="font-family:verdana,sans-serif">Now when i run 'gk-deploy -g', in the Rancher console, i see the following error:<br></div><div><font face="trebuchet ms, sans-serif" color="#ff0000">Readiness probe failed: Failed to get D-Bus connection: Operation not permitted</font><br></div><div style="font-family:verdana,sans-serif"><br></div><div style="font-family:verdana,sans-serif">From the attached gk-deploy_log i see that it failed at: </div><div><font face="trebuchet ms, sans-serif" color="#ff0000">Waiting for GlusterFS pods to start ... pods not found.</font><br></div><div><font face="trebuchet ms, sans-serif" color="#ff0000"><br></font></div><div><font color="#000000" face="trebuchet ms, sans-serif">In the kube-templates/glusterfs-d<wbr>aemonset.yaml file, i see this for Readiness probe section: </font></div><div><span style="font-family:"trebuchet ms",sans-serif">------------------------------<wbr>------------------------------<wbr>------</span><font face="trebuchet ms, sans-serif" color="#000000"><br></font></div><div><font face="trebuchet ms, sans-serif" color="#000000"><div><div> readinessProbe:</div><div> timeoutSeconds: 3</div><div> initialDelaySeconds: 40</div><div> exec:</div><div> command:</div><div> - "/bin/bash"</div><div> - "-c"</div><div> - systemctl status glusterd.service</div><div> periodSeconds: 25</div><div> successThreshold: 1</div><div> failureThreshold: 15</div></div></font></div><div><span style="font-family:"trebuchet ms",sans-serif">------------------------------<wbr>------------------------------<wbr>------</span><font face="trebuchet ms, sans-serif" color="#ff0000"><br></font></div><div><span style="font-family:"trebuchet ms",sans-serif"><br></span></div><div style="font-family:verdana,sans-serif"><br></div><div style="font-family:verdana,sans-serif">Status after running gk-deploy:<br></div><div style="font-family:verdana,sans-serif"><br></div><div><div><font face="trebuchet ms, sans-serif">[root@workstation deploy]# kubectl get nodes,pods,deployments,service<wbr>s</font></div><div><font face="trebuchet ms, sans-serif">NAME STATUS AGE VERSION</font></div><div><font face="trebuchet ms, sans-serif">no/node-1.c.kubernetes-174104.<wbr>internal Ready 2d v1.7.2-rancher1</font></div><div><font face="trebuchet ms, sans-serif">no/node-2.c.kubernetes-174104.<wbr>internal Ready 2d v1.7.2-rancher1</font></div><div><font face="trebuchet ms, sans-serif">no/node-3.c.kubernetes-174104.<wbr>internal Ready 2d v1.7.2-rancher1</font></div><div><font face="trebuchet ms, sans-serif"><br></font></div><div><font face="trebuchet ms, sans-serif">NAME READY STATUS RESTARTS AGE</font></div><div><font face="trebuchet ms, sans-serif">po/glusterfs-0s440 0/1 Running 0 1m</font></div><div><font face="trebuchet ms, sans-serif">po/glusterfs-j7dgr 0/1 Running 0 1m</font></div><div><font face="trebuchet ms, sans-serif">po/glusterfs-p6jl3 0/1 Running 0 1m</font></div><div><font face="trebuchet ms, sans-serif"><br></font></div><div><font face="trebuchet ms, sans-serif">NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE</font></div><div><font face="trebuchet ms, sans-serif">svc/kubernetes 10.43.0.1 <none> 443/TCP 2d</font></div></div><div style="font-family:verdana,sans-serif"><br></div><div style="font-family:verdana,sans-serif"><br></div><div style="font-family:verdana,sans-serif">Also, from prerequisite perspective, i was also seeing this mentioned:</div><div><div style="font-family:verdana,sans-serif"><br></div><div><font face="trebuchet ms, sans-serif">The following kernel modules must be loaded:</font></div><div><font face="trebuchet ms, sans-serif"> * dm_snapshot</font></div><div><font face="trebuchet ms, sans-serif"> * dm_mirror</font></div><div><font face="trebuchet ms, sans-serif"> * dm_thin_pool</font></div><div><font face="trebuchet ms, sans-serif"><br></font></div><div><span style="font-family:verdana,sans-serif">Where exactly is this to be checked? On all Gluster server nodes? How can i check whether it's there?</span><br></div><div><font face="trebuchet ms, sans-serif"><br></font></div><div><span style="font-family:verdana,sans-serif">I have attached topology.json and gk-deploy log for reference.</span><font face="trebuchet ms, sans-serif"><br></font></div></div><div style="font-family:verdana,sans-serif"><br></div><div style="font-family:verdana,sans-serif">Does this issue has anything to do with the host OS (RancherOS) that i am using for Gluster nodes? Any idea how i can fix this? Any help will really be appreciated.</div><div style="font-family:verdana,sans-serif"><br></div><div style="font-family:verdana,sans-serif"><br></div><div style="font-family:verdana,sans-serif"><div style="font-family:verdana,sans-serif">Thanks.</div></div></div></div></div>
</div><br></div>