[heketi-devel] heketi-cli kubernetes gluster pvc endpoint issue

[Ercan Aydoğan]

Hello,

i have dedicated 3.11.3 version glusterfs 3 nodes. i can create volumes with both gluster own command line utility and heketi-cli it’s ok. 

If i use  storage hostname FQDN i can create cluster with 

/heketi-cli   topology  load --json=topology.json

after storageclass , secret and pvc creation i got this error.

kubectl get pvc claim1 returns 

root at kubemaster ~ # kubectl describe pvc claim1
Name:          claim1
Namespace:     default
StorageClass:  fast
Status:        Pending
Volume:        
Labels:        <none>
Annotations:   volume.beta.kubernetes.io/storage-provisioner=kubernetes.io/glusterfs
Finalizers:    []
Capacity:      
Access Modes:  
Events:
  Type     Reason              Age   From                         Message
  ----     ------              ----  ----                         -------
  Warning  ProvisioningFailed  21s   persistentvolume-controller  Failed to provision volume with StorageClass "fast": create volume error: failed to create endpoint/service error creating endpoint: Endpoints "glusterfs-dynamic-claim1" is invalid: [subsets[0].addresses[0].ip: Invalid value: "pri.ostechnix.lan": must be a valid IP address, (e.g. 10.9.8.7), subsets[0].addresses[1].ip: Invalid value: "third.ostechnix.lan": must be a valid IP address, (e.g. 10.9.8.7), subsets[0].addresses[2].ip: Invalid value: "sec.ostechnix.lan": must be a valid IP address, (e.g. 10.9.8.7)]


my topology.json content is

{
  "clusters": [
    {
      "nodes": [
        {
          "node": {
            "hostnames": {
              "manage": [
                "51.15.77.14"
              ],
              "storage": [
                "pri.ostechnix.lan"
              ]
            },
            "zone": 1
          },
          "devices": [
            "/dev/nbd1",
            "/dev/nbd2",
	    "/dev/nbd3"
          ]
        },
        {
          "node": {
            "hostnames": {
              "manage": [
                "51.15.90.60"
              ],
              "storage": [
                "sec.ostechnix.lan"
              ]
            },
            "zone": 1
          },
          "devices": [
            "/dev/nbd1",
            "/dev/nbd2",
            "/dev/nbd3"
          ]
        },
        {
          "node": {
            "hostnames": {
              "manage": [
                "163.172.151.120"
              ],
              "storage": [
                "third.ostechnix.lan"
              ]
            },
            "zone": 1
          },
          "devices": [
            "/dev/nbd1",
            "/dev/nbd2",
             "/dev/nbd3"
          ]
        }
        
       
      ]
    }
  ]
}


Yes, it says storage must be ip for endpoint creation. But if i change 

manage : hostname 
storage: ip address

{
  "clusters": [
    {
      "nodes": [
        {
          "node": {
            "hostnames": {
              "manage": [
                "pri.ostechnix.lan"
              ],
              "storage": [
                "51.15.77.14"
              ]
            },
            "zone": 1
          },
          "devices": [
            "/dev/nbd1",
            "/dev/nbd2",
	    "/dev/nbd3"
          ]
        },
        {
          "node": {
            "hostnames": {
              "manage": [
                "sec.ostechnix.lan"
              ],
              "storage": [
                "51.15.90.60"
              ]
            },
            "zone": 1
          },
          "devices": [
            "/dev/nbd1",
            "/dev/nbd2",
            "/dev/nbd3"
          ]
        },
        {
          "node": {
            "hostnames": {
              "manage": [
                "third.ostechnix.lan"
              ],
              "storage": [
                "163.172.151.120"
              ]
            },
            "zone": 1
          },
          "devices": [
            "/dev/nbd1",
            "/dev/nbd2",
             "/dev/nbd3"
          ]
        }
        
       
      ]
    }
  ]
}

i can not create volume with heketi-cli.

it says 

root at kubemaster ~/heketi # ./heketi-cli volume create --size=3 --replica=3
Error: volume create: vol_207bbf81f28b959c51448b919be3bb59: failed: Host 51.15.90.60 is not in 'Peer in Cluster’ state

i need advice how can fix this issue.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/heketi-devel/attachments/20180207/6508b2d5/attachment-0001.html>