[heketi-devel] Problem creating heketidbstorage while running gk-deploy

[paul at thesimonet.org]

Heketi v9

K3s v1.17.4+k3s1

 

./gk-deploy --admin-key <key> --user-key <key> --ssh-keyfile
/etc/heketi/heketi_key -n heketi -v




Checking status of namespace matching 'heketi':

heketi   Active   45m

Using namespace "heketi".

Checking glusterd status on '192.168.121.11'.

Checking for pre-existing resources...

  GlusterFS pods ...

Checking status of pods matching '--selector=glusterfs=pod':

 

Timed out waiting for pods matching '--selector=glusterfs=pod'.

not found.

  deploy-heketi pod ...

Checking status of pods matching '--selector=deploy-heketi=pod':

deploy-heketi-65cf97688d-ffmq5   1/1   Running   0     44m

found.

  heketi pod ...

Checking status of pods matching '--selector=heketi=pod':

 

Timed out waiting for pods matching '--selector=heketi=pod'.

not found.

  gluster-s3 pod ...

Checking status of pods matching '--selector=glusterfs=s3-pod':

 

Timed out waiting for pods matching '--selector=glusterfs=s3-pod'.

not found.

Creating initial resources ... /usr/local/bin/kubectl -n heketi create -f
/usr/local/src/gluster-kubernetes/deploy/kube-templates/heketi-service-accou
nt.yaml 2>&1

Error from server (AlreadyExists): error when creating
"/usr/local/src/gluster-kubernetes/deploy/kube-templates/heketi-service-acco
unt.yaml": serviceaccounts "heketi-service-account" already exists

/usr/local/bin/kubectl -n heketi create clusterrolebinding heketi-sa-view
--clusterrole=edit --serviceaccount=heketi:heketi-service-account 2>&1

Error from server (AlreadyExists):
clusterrolebindings.rbac.authorization.k8s.io "heketi-sa-view" already
exists

/usr/local/bin/kubectl -n heketi label --overwrite clusterrolebinding
heketi-sa-view glusterfs=heketi-sa-view heketi=sa-view

clusterrolebinding.rbac.authorization.k8s.io/heketi-sa-view not labeled

OK

Determining heketi service URL ... OK

/usr/local/bin/kubectl -n heketi exec -i deploy-heketi-65cf97688d-ffmq5 --
heketi-cli -s http://localhost:8080 --user admin --secret
'4AF996D78B7BCA8BA338A2AC54917' topology load
--json=/etc/heketi/topology.json 2>&1

Found node 192.168.121.11 on cluster c00d18275186558c0c3704395b5062e9

Found device /dev/sda2

Found node 192.168.121.12 on cluster c00d18275186558c0c3704395b5062e9

Found device /dev/sda2

Found device /dev/sda3

Found device /dev/sda4

Found node 192.168.121.13 on cluster c00d18275186558c0c3704395b5062e9

Found device /dev/sda2

Found device /dev/sda3

Found node 192.168.121.14 on cluster c00d18275186558c0c3704395b5062e9

Found device /dev/sda2

Found device /dev/sda3

Found device /dev/sda4

heketi topology loaded.

/usr/local/bin/kubectl -n heketi exec -i deploy-heketi-65cf97688d-ffmq5 --
heketi-cli -s http://localhost:8080 --user admin --secret <key>
setup-openshift-heketi-storage --listfile=/tmp/heketi-storage.json  2>&1

Error: volume start: heketidbstorage: failed: Commit failed on localhost.
Please check log file for details.

 

#kubectl -n heketi logs deploy-heketi-65cf97688d-ffmq5 reveals:

 




[sshexec] DEBUG 2020/04/27 23:38:32
/src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:176: Host:
192.168.121.12:22 Command: /bin/bash -c 'gluster --mode=script volume create
heketidbstorage replica 3
192.168.121.12:/var/lib/heketi/mounts/vg_838b3e08c899600c6d98acae37592420/br
ick_6bb3e8abd1e0adb2fd1429d82f1db361/brick
192.168.121.13:/var/lib/heketi/mounts/vg_eca262c0ec6591e86db0da54988239c4/br
ick_33e0fcf98497ec673cd036d3caabff28/brick
192.168.121.14:/var/lib/heketi/mounts/vg_ff8df14e58db9e8e809883a2ae0c2f33/br
ick_75f49fd0f0c852a49cdbacfc27f4b578/brick '

Result: volume create: heketidbstorage: success: please start the volume to
access data

[sshexec] DEBUG 2020/04/27 23:38:32
/src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:176: Host:
192.168.121.12:22 Command: /bin/bash -c ''  <==

Result:

[negroni] Started GET /queue/9decf1712550f3ca01a7c2005b2294af

[negroni] Completed 200 OK in 626.79µs

[sshexec] ERROR 2020/04/27 23:38:33
/src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:173: Failed to run
command [/bin/bash -c 'gluster --mode=script volume start heketidbstorage']
on 192.168.121.12:22: Err[Process exited with status 1]: Stdout []: Stderr
[volume start: heketidbstorage: failed: Commit failed on localhost. Please
check log file for details.

 

Looks like there should have been a command issued at the <== 

 

--

Paul Simons

 <mailto:paul at thesimonet.org> paul at thesimonet.org

Covington, WA, USA

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/heketi-devel/attachments/20200427/850c34a8/attachment.html>