[heketi-devel] heketi-cli kubernetes gluster pvc endpoint issue
Ercan Aydoğan
ercan.aydogan at gmail.com
Wed Feb 7 15:20:53 UTC 2018
Yes, i delete heketi.db before every try
> On 7 Feb 2018, at 18:18, Jose A. Rivera <jarrpa at redhat.com> wrote:
>
> When you deleted the cluster, did you also delete the heketi database?
>
> --Jose
>
> On Wed, Feb 7, 2018 at 3:30 AM, Ercan Aydoğan <ercan.aydogan at gmail.com> wrote:
>> Gluster cluster on ubuntu 16.04 and i remove with this commands
>>
>> apt-get purge glusterfs-server -y --allow-change-held-packages
>> rm -rf /var/lib/glusterd
>> rm -rf /var/log/glusterfs/
>> wipefs -a --force /dev/nbd1
>> wipefs -a --force /dev/nbd2
>> wipefs -a --force /dev/nbd3
>>
>> after reboot install with
>>
>> apt-get install -y software-properties-common
>> add-apt-repository ppa:gluster/glusterfs-3.11
>> apt-get update
>> apt-get install -y glusterfs-server
>>
>> after this i’m using
>>
>>> /heketi-cli topology load --json=topology.json
>>
>>
>> but
>>
>> i can’t create any volume with gluster cmd or heketi-cli maybe this is
>> hostname or /etc/hostname issue.
>>
>> my current /etc/hosts is
>>
>> node 1
>>
>>
>> root at pri:/var/log/glusterfs# cat /etc/hosts
>> #127.0.0.1 localhost
>> 127.0.0.1 pri.ostechnix.lan pri
>> ::1 localhost ip6-localhost ip6-loopback
>> ff02::1 ip6-allnodes
>> ff02::2 ip6-allrouters
>>
>> 51.15.90.60 sec.ostechnix.lan sec
>> 163.172.151.120 third.ostechnix.lan third
>> root at pri:/var/log/glusterfs#
>>
>> on every node i set 127.0.0.1 matching hostname.
>>
>>
>>
>>
>>
>>
>> On 7 Feb 2018, at 11:51, Humble Chirammal <hchiramm at redhat.com> wrote:
>>
>> true, storage should be IP address. However afaict, it failed in "Peer in
>> cluster" , because gluster cluster is formed with different IP/hostname and
>> its stored in metadata. If you can delete the cluster and recreate it with
>> "storage" in IP, it should work I believe.
>>
>> On Wed, Feb 7, 2018 at 2:16 PM, Ercan Aydoğan <ercan.aydogan at gmail.com>
>> wrote:
>>>
>>> Hello,
>>>
>>> i have dedicated 3.11.3 version glusterfs 3 nodes. i can create volumes
>>> with both gluster own command line utility and heketi-cli it’s ok.
>>>
>>> If i use storage hostname FQDN i can create cluster with
>>>
>>> /heketi-cli topology load --json=topology.json
>>>
>>> after storageclass , secret and pvc creation i got this error.
>>>
>>> kubectl get pvc claim1 returns
>>>
>>> root at kubemaster ~ # kubectl describe pvc claim1
>>> Name: claim1
>>> Namespace: default
>>> StorageClass: fast
>>> Status: Pending
>>> Volume:
>>> Labels: <none>
>>> Annotations:
>>> volume.beta.kubernetes.io/storage-provisioner=kubernetes.io/glusterfs
>>> Finalizers: []
>>> Capacity:
>>> Access Modes:
>>> Events:
>>> Type Reason Age From Message
>>> ---- ------ ---- ---- -------
>>> Warning ProvisioningFailed 21s persistentvolume-controller Failed
>>> to provision volume with StorageClass "fast": create volume error: failed to
>>> create endpoint/service error creating endpoint: Endpoints
>>> "glusterfs-dynamic-claim1" is invalid: [subsets[0].addresses[0].ip: Invalid
>>> value: "pri.ostechnix.lan": must be a valid IP address, (e.g. 10.9.8.7),
>>> subsets[0].addresses[1].ip: Invalid value: "third.ostechnix.lan": must be a
>>> valid IP address, (e.g. 10.9.8.7), subsets[0].addresses[2].ip: Invalid
>>> value: "sec.ostechnix.lan": must be a valid IP address, (e.g. 10.9.8.7)]
>>>
>>>
>>> my topology.json content is
>>>
>>> {
>>> "clusters": [
>>> {
>>> "nodes": [
>>> {
>>> "node": {
>>> "hostnames": {
>>> "manage": [
>>> "51.15.77.14"
>>> ],
>>> "storage": [
>>> "pri.ostechnix.lan"
>>> ]
>>> },
>>> "zone": 1
>>> },
>>> "devices": [
>>> "/dev/nbd1",
>>> "/dev/nbd2",
>>> "/dev/nbd3"
>>> ]
>>> },
>>> {
>>> "node": {
>>> "hostnames": {
>>> "manage": [
>>> "51.15.90.60"
>>> ],
>>> "storage": [
>>> "sec.ostechnix.lan"
>>> ]
>>> },
>>> "zone": 1
>>> },
>>> "devices": [
>>> "/dev/nbd1",
>>> "/dev/nbd2",
>>> "/dev/nbd3"
>>> ]
>>> },
>>> {
>>> "node": {
>>> "hostnames": {
>>> "manage": [
>>> "163.172.151.120"
>>> ],
>>> "storage": [
>>> "third.ostechnix.lan"
>>> ]
>>> },
>>> "zone": 1
>>> },
>>> "devices": [
>>> "/dev/nbd1",
>>> "/dev/nbd2",
>>> "/dev/nbd3"
>>> ]
>>> }
>>>
>>>
>>> ]
>>> }
>>> ]
>>> }
>>>
>>>
>>> Yes, it says storage must be ip for endpoint creation. But if i change
>>>
>>> manage : hostname
>>> storage: ip address
>>>
>>> {
>>> "clusters": [
>>> {
>>> "nodes": [
>>> {
>>> "node": {
>>> "hostnames": {
>>> "manage": [
>>> "pri.ostechnix.lan"
>>> ],
>>> "storage": [
>>> "51.15.77.14"
>>> ]
>>> },
>>> "zone": 1
>>> },
>>> "devices": [
>>> "/dev/nbd1",
>>> "/dev/nbd2",
>>> "/dev/nbd3"
>>> ]
>>> },
>>> {
>>> "node": {
>>> "hostnames": {
>>> "manage": [
>>> "sec.ostechnix.lan"
>>> ],
>>> "storage": [
>>> "51.15.90.60"
>>> ]
>>> },
>>> "zone": 1
>>> },
>>> "devices": [
>>> "/dev/nbd1",
>>> "/dev/nbd2",
>>> "/dev/nbd3"
>>> ]
>>> },
>>> {
>>> "node": {
>>> "hostnames": {
>>> "manage": [
>>> "third.ostechnix.lan"
>>> ],
>>> "storage": [
>>> "163.172.151.120"
>>> ]
>>> },
>>> "zone": 1
>>> },
>>> "devices": [
>>> "/dev/nbd1",
>>> "/dev/nbd2",
>>> "/dev/nbd3"
>>> ]
>>> }
>>>
>>>
>>> ]
>>> }
>>> ]
>>> }
>>>
>>> i can not create volume with heketi-cli.
>>>
>>> it says
>>>
>>> root at kubemaster ~/heketi # ./heketi-cli volume create --size=3 --replica=3
>>> Error: volume create: vol_207bbf81f28b959c51448b919be3bb59: failed: Host
>>> 51.15.90.60 is not in 'Peer in Cluster’ state
>>>
>>> i need advice how can fix this issue.
>>>
>>>
>>> _______________________________________________
>>> heketi-devel mailing list
>>> heketi-devel at gluster.org
>>> http://lists.gluster.org/mailman/listinfo/heketi-devel
>>>
>>
>>
>>
>> --
>> Cheers,
>> Humble
>>
>> Red Hat Storage Engineering
>> Mastering KVM Virtualization: http://amzn.to/2vFTXaW
>> Website: http://humblec.com
>>
>>
>>
>> _______________________________________________
>> heketi-devel mailing list
>> heketi-devel at gluster.org
>> http://lists.gluster.org/mailman/listinfo/heketi-devel
>>
More information about the heketi-devel
mailing list