[heketi-devel] heketi-cli kubernetes gluster pvc endpoint issue
Ercan Aydoğan
ercan.aydogan at gmail.com
Wed Feb 7 09:30:46 UTC 2018
Gluster cluster on ubuntu 16.04 and i remove with this commands
apt-get purge glusterfs-server -y --allow-change-held-packages
rm -rf /var/lib/glusterd
rm -rf /var/log/glusterfs/
wipefs -a --force /dev/nbd1
wipefs -a --force /dev/nbd2
wipefs -a --force /dev/nbd3
after reboot install with
apt-get install -y software-properties-common
add-apt-repository ppa:gluster/glusterfs-3.11
apt-get update
apt-get install -y glusterfs-server
after this i’m using
> /heketi-cli topology load --json=topology.json
but
i can’t create any volume with gluster cmd or heketi-cli maybe this is hostname or /etc/hostname issue.
my current /etc/hosts is
node 1
root at pri:/var/log/glusterfs# cat /etc/hosts
#127.0.0.1 localhost
127.0.0.1 pri.ostechnix.lan pri
::1 localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
51.15.90.60 sec.ostechnix.lan sec
163.172.151.120 third.ostechnix.lan third
root at pri:/var/log/glusterfs#
on every node i set 127.0.0.1 matching hostname.
> On 7 Feb 2018, at 11:51, Humble Chirammal <hchiramm at redhat.com> wrote:
>
> true, storage should be IP address. However afaict, it failed in "Peer in cluster" , because gluster cluster is formed with different IP/hostname and its stored in metadata. If you can delete the cluster and recreate it with "storage" in IP, it should work I believe.
>
> On Wed, Feb 7, 2018 at 2:16 PM, Ercan Aydoğan <ercan.aydogan at gmail.com <mailto:ercan.aydogan at gmail.com>> wrote:
> Hello,
>
> i have dedicated 3.11.3 version glusterfs 3 nodes. i can create volumes with both gluster own command line utility and heketi-cli it’s ok.
>
> If i use storage hostname FQDN i can create cluster with
>
> /heketi-cli topology load --json=topology.json
>
> after storageclass , secret and pvc creation i got this error.
>
> kubectl get pvc claim1 returns
>
> root at kubemaster ~ # kubectl describe pvc claim1
> Name: claim1
> Namespace: default
> StorageClass: fast
> Status: Pending
> Volume:
> Labels: <none>
> Annotations: volume.beta.kubernetes.io/storage-provisioner=kubernetes.io/glusterfs <http://volume.beta.kubernetes.io/storage-provisioner=kubernetes.io/glusterfs>
> Finalizers: []
> Capacity:
> Access Modes:
> Events:
> Type Reason Age From Message
> ---- ------ ---- ---- -------
> Warning ProvisioningFailed 21s persistentvolume-controller Failed to provision volume with StorageClass "fast": create volume error: failed to create endpoint/service error creating endpoint: Endpoints "glusterfs-dynamic-claim1" is invalid: [subsets[0].addresses[0].ip: Invalid value: "pri.ostechnix.lan": must be a valid IP address, (e.g. 10.9.8.7), subsets[0].addresses[1].ip: Invalid value: "third.ostechnix.lan": must be a valid IP address, (e.g. 10.9.8.7), subsets[0].addresses[2].ip: Invalid value: "sec.ostechnix.lan": must be a valid IP address, (e.g. 10.9.8.7)]
>
>
> my topology.json content is
>
> {
> "clusters": [
> {
> "nodes": [
> {
> "node": {
> "hostnames": {
> "manage": [
> "51.15.77.14"
> ],
> "storage": [
> "pri.ostechnix.lan"
> ]
> },
> "zone": 1
> },
> "devices": [
> "/dev/nbd1",
> "/dev/nbd2",
> "/dev/nbd3"
> ]
> },
> {
> "node": {
> "hostnames": {
> "manage": [
> "51.15.90.60"
> ],
> "storage": [
> "sec.ostechnix.lan"
> ]
> },
> "zone": 1
> },
> "devices": [
> "/dev/nbd1",
> "/dev/nbd2",
> "/dev/nbd3"
> ]
> },
> {
> "node": {
> "hostnames": {
> "manage": [
> "163.172.151.120"
> ],
> "storage": [
> "third.ostechnix.lan"
> ]
> },
> "zone": 1
> },
> "devices": [
> "/dev/nbd1",
> "/dev/nbd2",
> "/dev/nbd3"
> ]
> }
>
>
> ]
> }
> ]
> }
>
>
> Yes, it says storage must be ip for endpoint creation. But if i change
>
> manage : hostname
> storage: ip address
>
> {
> "clusters": [
> {
> "nodes": [
> {
> "node": {
> "hostnames": {
> "manage": [
> "pri.ostechnix.lan"
> ],
> "storage": [
> "51.15.77.14"
> ]
> },
> "zone": 1
> },
> "devices": [
> "/dev/nbd1",
> "/dev/nbd2",
> "/dev/nbd3"
> ]
> },
> {
> "node": {
> "hostnames": {
> "manage": [
> "sec.ostechnix.lan"
> ],
> "storage": [
> "51.15.90.60"
> ]
> },
> "zone": 1
> },
> "devices": [
> "/dev/nbd1",
> "/dev/nbd2",
> "/dev/nbd3"
> ]
> },
> {
> "node": {
> "hostnames": {
> "manage": [
> "third.ostechnix.lan"
> ],
> "storage": [
> "163.172.151.120"
> ]
> },
> "zone": 1
> },
> "devices": [
> "/dev/nbd1",
> "/dev/nbd2",
> "/dev/nbd3"
> ]
> }
>
>
> ]
> }
> ]
> }
>
> i can not create volume with heketi-cli.
>
> it says
>
> root at kubemaster ~/heketi # ./heketi-cli volume create --size=3 --replica=3
> Error: volume create: vol_207bbf81f28b959c51448b919be3bb59: failed: Host 51.15.90.60 is not in 'Peer in Cluster’ state
>
> i need advice how can fix this issue.
>
>
> _______________________________________________
> heketi-devel mailing list
> heketi-devel at gluster.org <mailto:heketi-devel at gluster.org>
> http://lists.gluster.org/mailman/listinfo/heketi-devel <http://lists.gluster.org/mailman/listinfo/heketi-devel>
>
>
>
>
> --
> Cheers,
> Humble
>
> Red Hat Storage Engineering
> Mastering KVM Virtualization: http://amzn.to/2vFTXaW <http://amzn.to/2vFTXaW>Website: http://humblec.com <http://humblec.com/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/heketi-devel/attachments/20180207/c490a90e/attachment-0001.html>
More information about the heketi-devel
mailing list