<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=iso-8859-1"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Britannic Bold";
panose-1:2 11 9 3 6 7 3 2 2 4;}
@font-face
{font-family:Carlito;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link="#0563C1" vlink="#954F72"><div class=WordSection1><p class=MsoNormal>Heketi v9<o:p></o:p></p><p class=MsoNormal>K3s v1.17.4+k3s1<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><span style='font-size:12.0pt'>./gk-deploy --admin-key <key> --user-key <key> --ssh-keyfile /etc/heketi/heketi_key -n heketi -v</span><o:p></o:p></p><p class=MsoNormal>…<o:p></o:p></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Checking status of namespace matching 'heketi':<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>heketi Active 45m<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Using namespace "heketi".<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Checking glusterd status on '192.168.121.11'.<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Checking for pre-existing resources...<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'> GlusterFS pods ...<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Checking status of pods matching '--selector=glusterfs=pod':<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'><o:p> </o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Timed out waiting for pods matching '--selector=glusterfs=pod'.<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>not found.<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'> deploy-heketi pod ...<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Checking status of pods matching '--selector=deploy-heketi=pod':<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>deploy-heketi-65cf97688d-ffmq5 1/1 Running 0 44m<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>found.<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'> heketi pod ...<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Checking status of pods matching '--selector=heketi=pod':<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'><o:p> </o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Timed out waiting for pods matching '--selector=heketi=pod'.<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>not found.<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'> gluster-s3 pod ...<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Checking status of pods matching '--selector=glusterfs=s3-pod':<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'><o:p> </o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Timed out waiting for pods matching '--selector=glusterfs=s3-pod'.<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>not found.<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Creating initial resources ... /usr/local/bin/kubectl -n heketi create -f /usr/local/src/gluster-kubernetes/deploy/kube-templates/heketi-service-account.yaml 2>&1<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Error from server (AlreadyExists): error when creating "/usr/local/src/gluster-kubernetes/deploy/kube-templates/heketi-service-account.yaml": serviceaccounts "heketi-service-account" already exists<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>/usr/local/bin/kubectl -n heketi create clusterrolebinding heketi-sa-view --clusterrole=edit --serviceaccount=heketi:heketi-service-account 2>&1<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Error from server (AlreadyExists): clusterrolebindings.rbac.authorization.k8s.io "heketi-sa-view" already exists<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>/usr/local/bin/kubectl -n heketi label --overwrite clusterrolebinding heketi-sa-view glusterfs=heketi-sa-view heketi=sa-view<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>clusterrolebinding.rbac.authorization.k8s.io/heketi-sa-view not labeled<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>OK<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Determining heketi service URL ... OK<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>/usr/local/bin/kubectl -n heketi exec -i deploy-heketi-65cf97688d-ffmq5 -- heketi-cli -s http://localhost:8080 --user admin --secret '4AF996D78B7BCA8BA338A2AC54917' topology load --json=/etc/heketi/topology.json 2>&1<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Found node 192.168.121.11 on cluster c00d18275186558c0c3704395b5062e9<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Found device /dev/sda2<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Found node 192.168.121.12 on cluster c00d18275186558c0c3704395b5062e9<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Found device /dev/sda2<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Found device /dev/sda3<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Found device /dev/sda4<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Found node 192.168.121.13 on cluster c00d18275186558c0c3704395b5062e9<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Found device /dev/sda2<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Found device /dev/sda3<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Found node 192.168.121.14 on cluster c00d18275186558c0c3704395b5062e9<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Found device /dev/sda2<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Found device /dev/sda3<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Found device /dev/sda4<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>heketi topology loaded.<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>/usr/local/bin/kubectl -n heketi exec -i deploy-heketi-65cf97688d-ffmq5 -- heketi-cli -s http://localhost:8080 --user admin --secret <key> setup-openshift-heketi-storage --listfile=/tmp/heketi-storage.json 2>&1<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt'>Error: volume start: heketidbstorage: failed: Commit failed on localhost. Please check log file for details.<o:p></o:p></span></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>#kubectl -n heketi logs deploy-heketi-65cf97688d-ffmq5 reveals:<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>…<o:p></o:p></p><p class=MsoNormal>[sshexec] DEBUG 2020/04/27 23:38:32 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:176: Host: 192.168.121.12:22 Command: /bin/bash -c 'gluster --mode=script volume create heketidbstorage replica 3 192.168.121.12:/var/lib/heketi/mounts/vg_838b3e08c899600c6d98acae37592420/brick_6bb3e8abd1e0adb2fd1429d82f1db361/brick 192.168.121.13:/var/lib/heketi/mounts/vg_eca262c0ec6591e86db0da54988239c4/brick_33e0fcf98497ec673cd036d3caabff28/brick 192.168.121.14:/var/lib/heketi/mounts/vg_ff8df14e58db9e8e809883a2ae0c2f33/brick_75f49fd0f0c852a49cdbacfc27f4b578/brick '<o:p></o:p></p><p class=MsoNormal>Result: volume create: heketidbstorage: success: please start the volume to access data<o:p></o:p></p><p class=MsoNormal>[sshexec] DEBUG 2020/04/27 23:38:32 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:176: Host: 192.168.121.12:22 Command: /bin/bash -c '' <span style='font-family:Wingdings'>ç</span><o:p></o:p></p><p class=MsoNormal>Result:<o:p></o:p></p><p class=MsoNormal>[negroni] Started GET /queue/9decf1712550f3ca01a7c2005b2294af<o:p></o:p></p><p class=MsoNormal>[negroni] Completed 200 OK in 626.79µs<o:p></o:p></p><p class=MsoNormal>[sshexec] ERROR 2020/04/27 23:38:33 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:173: Failed to run command [/bin/bash -c 'gluster --mode=script volume start heketidbstorage'] on 192.168.121.12:22: Err[Process exited with status 1]: Stdout []: Stderr [volume start: heketidbstorage: failed: Commit failed on localhost. Please check log file for details.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Looks like there should have been a command issued at the <span style='font-family:Wingdings'>ç</span> <o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>--<o:p></o:p></p><p class=MsoNormal><span style='font-family:"Britannic Bold",sans-serif'>Paul Simons<o:p></o:p></span></p><p class=MsoNormal><a href="mailto:paul@thesimonet.org"><span style='color:#0563C1'>paul@thesimonet.org</span></a><o:p></o:p></p><p class=MsoNormal>Covington, WA, USA<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p></div></body></html>