[heketi-devel] heketi-cli kubernetes gluster pvc endpoint issue
Jose A. Rivera
jarrpa at redhat.com
Wed Feb 7 15:18:22 UTC 2018
When you deleted the cluster, did you also delete the heketi database?
--Jose
On Wed, Feb 7, 2018 at 3:30 AM, Ercan Aydoğan <ercan.aydogan at gmail.com> wrote:
> Gluster cluster on ubuntu 16.04 and i remove with this commands
>
> apt-get purge glusterfs-server -y --allow-change-held-packages
> rm -rf /var/lib/glusterd
> rm -rf /var/log/glusterfs/
> wipefs -a --force /dev/nbd1
> wipefs -a --force /dev/nbd2
> wipefs -a --force /dev/nbd3
>
> after reboot install with
>
> apt-get install -y software-properties-common
> add-apt-repository ppa:gluster/glusterfs-3.11
> apt-get update
> apt-get install -y glusterfs-server
>
> after this i’m using
>
>> /heketi-cli topology load --json=topology.json
>
>
> but
>
> i can’t create any volume with gluster cmd or heketi-cli maybe this is
> hostname or /etc/hostname issue.
>
> my current /etc/hosts is
>
> node 1
>
>
> root at pri:/var/log/glusterfs# cat /etc/hosts
> #127.0.0.1 localhost
> 127.0.0.1 pri.ostechnix.lan pri
> ::1 localhost ip6-localhost ip6-loopback
> ff02::1 ip6-allnodes
> ff02::2 ip6-allrouters
>
> 51.15.90.60 sec.ostechnix.lan sec
> 163.172.151.120 third.ostechnix.lan third
> root at pri:/var/log/glusterfs#
>
> on every node i set 127.0.0.1 matching hostname.
>
>
>
>
>
>
> On 7 Feb 2018, at 11:51, Humble Chirammal <hchiramm at redhat.com> wrote:
>
> true, storage should be IP address. However afaict, it failed in "Peer in
> cluster" , because gluster cluster is formed with different IP/hostname and
> its stored in metadata. If you can delete the cluster and recreate it with
> "storage" in IP, it should work I believe.
>
> On Wed, Feb 7, 2018 at 2:16 PM, Ercan Aydoğan <ercan.aydogan at gmail.com>
> wrote:
>>
>> Hello,
>>
>> i have dedicated 3.11.3 version glusterfs 3 nodes. i can create volumes
>> with both gluster own command line utility and heketi-cli it’s ok.
>>
>> If i use storage hostname FQDN i can create cluster with
>>
>> /heketi-cli topology load --json=topology.json
>>
>> after storageclass , secret and pvc creation i got this error.
>>
>> kubectl get pvc claim1 returns
>>
>> root at kubemaster ~ # kubectl describe pvc claim1
>> Name: claim1
>> Namespace: default
>> StorageClass: fast
>> Status: Pending
>> Volume:
>> Labels: <none>
>> Annotations:
>> volume.beta.kubernetes.io/storage-provisioner=kubernetes.io/glusterfs
>> Finalizers: []
>> Capacity:
>> Access Modes:
>> Events:
>> Type Reason Age From Message
>> ---- ------ ---- ---- -------
>> Warning ProvisioningFailed 21s persistentvolume-controller Failed
>> to provision volume with StorageClass "fast": create volume error: failed to
>> create endpoint/service error creating endpoint: Endpoints
>> "glusterfs-dynamic-claim1" is invalid: [subsets[0].addresses[0].ip: Invalid
>> value: "pri.ostechnix.lan": must be a valid IP address, (e.g. 10.9.8.7),
>> subsets[0].addresses[1].ip: Invalid value: "third.ostechnix.lan": must be a
>> valid IP address, (e.g. 10.9.8.7), subsets[0].addresses[2].ip: Invalid
>> value: "sec.ostechnix.lan": must be a valid IP address, (e.g. 10.9.8.7)]
>>
>>
>> my topology.json content is
>>
>> {
>> "clusters": [
>> {
>> "nodes": [
>> {
>> "node": {
>> "hostnames": {
>> "manage": [
>> "51.15.77.14"
>> ],
>> "storage": [
>> "pri.ostechnix.lan"
>> ]
>> },
>> "zone": 1
>> },
>> "devices": [
>> "/dev/nbd1",
>> "/dev/nbd2",
>> "/dev/nbd3"
>> ]
>> },
>> {
>> "node": {
>> "hostnames": {
>> "manage": [
>> "51.15.90.60"
>> ],
>> "storage": [
>> "sec.ostechnix.lan"
>> ]
>> },
>> "zone": 1
>> },
>> "devices": [
>> "/dev/nbd1",
>> "/dev/nbd2",
>> "/dev/nbd3"
>> ]
>> },
>> {
>> "node": {
>> "hostnames": {
>> "manage": [
>> "163.172.151.120"
>> ],
>> "storage": [
>> "third.ostechnix.lan"
>> ]
>> },
>> "zone": 1
>> },
>> "devices": [
>> "/dev/nbd1",
>> "/dev/nbd2",
>> "/dev/nbd3"
>> ]
>> }
>>
>>
>> ]
>> }
>> ]
>> }
>>
>>
>> Yes, it says storage must be ip for endpoint creation. But if i change
>>
>> manage : hostname
>> storage: ip address
>>
>> {
>> "clusters": [
>> {
>> "nodes": [
>> {
>> "node": {
>> "hostnames": {
>> "manage": [
>> "pri.ostechnix.lan"
>> ],
>> "storage": [
>> "51.15.77.14"
>> ]
>> },
>> "zone": 1
>> },
>> "devices": [
>> "/dev/nbd1",
>> "/dev/nbd2",
>> "/dev/nbd3"
>> ]
>> },
>> {
>> "node": {
>> "hostnames": {
>> "manage": [
>> "sec.ostechnix.lan"
>> ],
>> "storage": [
>> "51.15.90.60"
>> ]
>> },
>> "zone": 1
>> },
>> "devices": [
>> "/dev/nbd1",
>> "/dev/nbd2",
>> "/dev/nbd3"
>> ]
>> },
>> {
>> "node": {
>> "hostnames": {
>> "manage": [
>> "third.ostechnix.lan"
>> ],
>> "storage": [
>> "163.172.151.120"
>> ]
>> },
>> "zone": 1
>> },
>> "devices": [
>> "/dev/nbd1",
>> "/dev/nbd2",
>> "/dev/nbd3"
>> ]
>> }
>>
>>
>> ]
>> }
>> ]
>> }
>>
>> i can not create volume with heketi-cli.
>>
>> it says
>>
>> root at kubemaster ~/heketi # ./heketi-cli volume create --size=3 --replica=3
>> Error: volume create: vol_207bbf81f28b959c51448b919be3bb59: failed: Host
>> 51.15.90.60 is not in 'Peer in Cluster’ state
>>
>> i need advice how can fix this issue.
>>
>>
>> _______________________________________________
>> heketi-devel mailing list
>> heketi-devel at gluster.org
>> http://lists.gluster.org/mailman/listinfo/heketi-devel
>>
>
>
>
> --
> Cheers,
> Humble
>
> Red Hat Storage Engineering
> Mastering KVM Virtualization: http://amzn.to/2vFTXaW
> Website: http://humblec.com
>
>
>
> _______________________________________________
> heketi-devel mailing list
> heketi-devel at gluster.org
> http://lists.gluster.org/mailman/listinfo/heketi-devel
>
More information about the heketi-devel
mailing list