[Gluster-devel] [Heketi] Block store related API design discussion

[Luis Pabón]

Very good points.  Thanks Prasanna for putting this together.  I agree with
your comments in that Heketi is the high level abstraction API and it should have
an API similar of what is described by Prasanna.

I definitely do not think any File Api should be available in Heketi,
because that is an implementation of the Block API.  The Heketi API should
be similar to something like OpenStack Cinder.

I think that the actual management of the Volumes used for Block storage
and the files in them should be all managed by Heketi.  How they are
actually created is still to be determined, but we could have Heketi
create them, or have helper programs do that.

We also need to document the exact workflow to enable a file in
a Gluster volume to be exposed as a block device.  This will help
determine where the creation of the file could take place.

We can capture our decisions from these discussions in the
following page:

https://github.com/heketi/heketi/wiki/Proposed-Changes

- Luis


----- Original Message -----
From: "Humble Chirammal" <hchiramm at redhat.com>
To: "Raghavendra Talur" <rtalur at redhat.com>
Cc: "Prasanna Kalever" <pkalever at redhat.com>, "gluster-devel" <gluster-devel at gluster.org>, "Stephen Watt" <swatt at redhat.com>, "Luis Pabon" <lpabon at redhat.com>, "Michael Adam" <madam at redhat.com>, "Ramakrishna Yekulla" <rreddy at redhat.com>, "Mohamed Ashiq Liyazudeen" <mliyazud at redhat.com>
Sent: Tuesday, September 13, 2016 2:23:39 AM
Subject: Re: [Gluster-devel] [Heketi] Block store related API design discussion





----- Original Message -----
| From: "Raghavendra Talur" <rtalur at redhat.com>
| To: "Prasanna Kalever" <pkalever at redhat.com>
| Cc: "gluster-devel" <gluster-devel at gluster.org>, "Stephen Watt" <swatt at redhat.com>, "Luis Pabon" <lpabon at redhat.com>,
| "Michael Adam" <madam at redhat.com>, "Humble Chirammal" <hchiramm at redhat.com>, "Ramakrishna Yekulla"
| <rreddy at redhat.com>, "Mohamed Ashiq Liyazudeen" <mliyazud at redhat.com>
| Sent: Tuesday, September 13, 2016 11:08:44 AM
| Subject: Re: [Gluster-devel] [Heketi] Block store related API design discussion
| 
| On Mon, Sep 12, 2016 at 11:30 PM, Prasanna Kalever <pkalever at redhat.com>
| wrote:
| 
| > Hi all,
| >
| > This mail is open for discussion on gluster block store integration with
| > heketi and its REST API interface design constraints.
| >
| >
| >                          ___ Volume Request ...
| >                         |
| >                         |
| > PVC claim -> Heketi --->|
| >                         |
| >                         |
| >                         |
| >                         |
| >                         |                            __ BlockCreate
| >                         |                           |
| >                         |                           |__ BlockInfo
| >                         |                           |
| >                         |___ Block Request (APIS)-> |__ BlockResize
| >                                                     |
| >                                                     |__ BlockList
| >                                                     |
| >                                                     |__ BlockDelete
| >
| > Heketi will have block API and volume API, when user submit a Persistent
| > volume claim, Kubernetes provisioner based on the storage class(from PVC)
| > talks to heketi for storage, heketi intern calls block or volume API's
| > based on request.
| >
| 
| This is probably wrong. It won't be Heketi calling block or volume APIs. It
| would be Kubernetes calling block or volume API *of* Heketi.
| 
| 
| > With my limited understanding, heketi currently creates clusters from
| > provided nodes, creates volumes and handover them to the user.
| > For block related API's, it has to deal with files right ?
| >
| > Here is how block API's look like in short-
| > Create: heketi has to create file in the volume and export it as a iscsi
| > target device and hand it over to user.
| > Info: show block stores information across all the clusters, connection
| > info, size etc.
| > resize: resize the file in the volume, refresh connections from initiator
| > side
| > List: List the connections
| > Delete: logout the connections and delete the file in the gluster volume
| >
| > Couple of questions:
| > 1. Should Block API have sub API's such as FileCreate, FileList,
| > FileResize, File delete and etc then get it used in Block API as they
| > mostly deal with files.
| >
| 
| IMO, Heketi should not expose any File related API. It should only have
| APIs to service request for block devices; how the block devices are
| created and modified is an implementation detail.
| 
| 
| > 2. How do we create the actual file in the volume, meaning using FUSE
| > mount (which may involve an extra process running) or gfapi, again if gfapi
| > should we go with c API's, python bindings or go bindings ?
| >
| 3. Should we get targetcli related (LUN exporting) setup done from heketi
| > or do we seek help from gdeploy for this ?
| >
| 
| I would prefer to either have it in Heketi or in Kubernetes. If the API in
| Heketi promises just the creation of block device, then the rest of the
| implementation should be in Kubernetes(the export part). If the API in
| Heketi promises create and export both, I would say Heketi should have the
| implementation within itself.
| 
| 

IMO, we should not think about how the clients ( ex: k8s) use it, because there may be different clients.
We should concentrate mainly on 'in/out' of block API in Heketi. Regardless of which client, the API should act the same way.

--Humble