[Gluster-devel] [Heketi] Block store related API design discussion
Niels de Vos
ndevos at redhat.com
Mon Sep 19 10:39:26 UTC 2016
On Mon, Sep 19, 2016 at 03:34:29PM +0530, Prasanna Kalever wrote:
> On Mon, Sep 19, 2016 at 10:13 AM, Niels de Vos <ndevos at redhat.com> wrote:
> > On Tue, Sep 13, 2016 at 12:06:00PM -0400, Luis Pabón wrote:
> >> Very good points. Thanks Prasanna for putting this together. I agree with
> >> your comments in that Heketi is the high level abstraction API and it should have
> >> an API similar of what is described by Prasanna.
> >>
> >> I definitely do not think any File Api should be available in Heketi,
> >> because that is an implementation of the Block API. The Heketi API should
> >> be similar to something like OpenStack Cinder.
> >>
> >> I think that the actual management of the Volumes used for Block storage
> >> and the files in them should be all managed by Heketi. How they are
> >> actually created is still to be determined, but we could have Heketi
> >> create them, or have helper programs do that.
> >
> > Maybe a tool like qemu-img? If whatever iscsi service understand the
> > format (at the very least 'raw'), you could get functionality like
> > snapshots pretty simple.
>
> Niels,
>
> This is brilliant and subset of the Idea falls in one among my
> thoughts, only concern is about building dependencies of qemu with
> Heketi.
> But at an advantage of easy and cool snapshots solution.
And well tested as I understand that oVirt is moving to use qemu-img as
well. Other tools are able to use the qcow2 format, maybe the iscsi
servce that gets used does so too.
Has there already been a decision on what Heketi will configure as iscsi
service? I am aware of the tgt [1] and LIO/TCMU [2] projects.
Niels
1. http://stgt.sourceforge.net/
2. https://github.com/open-iscsi/tcmu-runner
http://blog.gluster.org/2016/04/using-lio-with-gluster/
>
> --
> Prasanna
>
> >
> > Niels
> >
> >
> >> We also need to document the exact workflow to enable a file in
> >> a Gluster volume to be exposed as a block device. This will help
> >> determine where the creation of the file could take place.
> >>
> >> We can capture our decisions from these discussions in the
> >> following page:
> >>
> >> https://github.com/heketi/heketi/wiki/Proposed-Changes
> >>
> >> - Luis
> >>
> >>
> >> ----- Original Message -----
> >> From: "Humble Chirammal" <hchiramm at redhat.com>
> >> To: "Raghavendra Talur" <rtalur at redhat.com>
> >> Cc: "Prasanna Kalever" <pkalever at redhat.com>, "gluster-devel" <gluster-devel at gluster.org>, "Stephen Watt" <swatt at redhat.com>, "Luis Pabon" <lpabon at redhat.com>, "Michael Adam" <madam at redhat.com>, "Ramakrishna Yekulla" <rreddy at redhat.com>, "Mohamed Ashiq Liyazudeen" <mliyazud at redhat.com>
> >> Sent: Tuesday, September 13, 2016 2:23:39 AM
> >> Subject: Re: [Gluster-devel] [Heketi] Block store related API design discussion
> >>
> >>
> >>
> >>
> >>
> >> ----- Original Message -----
> >> | From: "Raghavendra Talur" <rtalur at redhat.com>
> >> | To: "Prasanna Kalever" <pkalever at redhat.com>
> >> | Cc: "gluster-devel" <gluster-devel at gluster.org>, "Stephen Watt" <swatt at redhat.com>, "Luis Pabon" <lpabon at redhat.com>,
> >> | "Michael Adam" <madam at redhat.com>, "Humble Chirammal" <hchiramm at redhat.com>, "Ramakrishna Yekulla"
> >> | <rreddy at redhat.com>, "Mohamed Ashiq Liyazudeen" <mliyazud at redhat.com>
> >> | Sent: Tuesday, September 13, 2016 11:08:44 AM
> >> | Subject: Re: [Gluster-devel] [Heketi] Block store related API design discussion
> >> |
> >> | On Mon, Sep 12, 2016 at 11:30 PM, Prasanna Kalever <pkalever at redhat.com>
> >> | wrote:
> >> |
> >> | > Hi all,
> >> | >
> >> | > This mail is open for discussion on gluster block store integration with
> >> | > heketi and its REST API interface design constraints.
> >> | >
> >> | >
> >> | > ___ Volume Request ...
> >> | > |
> >> | > |
> >> | > PVC claim -> Heketi --->|
> >> | > |
> >> | > |
> >> | > |
> >> | > |
> >> | > | __ BlockCreate
> >> | > | |
> >> | > | |__ BlockInfo
> >> | > | |
> >> | > |___ Block Request (APIS)-> |__ BlockResize
> >> | > |
> >> | > |__ BlockList
> >> | > |
> >> | > |__ BlockDelete
> >> | >
> >> | > Heketi will have block API and volume API, when user submit a Persistent
> >> | > volume claim, Kubernetes provisioner based on the storage class(from PVC)
> >> | > talks to heketi for storage, heketi intern calls block or volume API's
> >> | > based on request.
> >> | >
> >> |
> >> | This is probably wrong. It won't be Heketi calling block or volume APIs. It
> >> | would be Kubernetes calling block or volume API *of* Heketi.
> >> |
> >> |
> >> | > With my limited understanding, heketi currently creates clusters from
> >> | > provided nodes, creates volumes and handover them to the user.
> >> | > For block related API's, it has to deal with files right ?
> >> | >
> >> | > Here is how block API's look like in short-
> >> | > Create: heketi has to create file in the volume and export it as a iscsi
> >> | > target device and hand it over to user.
> >> | > Info: show block stores information across all the clusters, connection
> >> | > info, size etc.
> >> | > resize: resize the file in the volume, refresh connections from initiator
> >> | > side
> >> | > List: List the connections
> >> | > Delete: logout the connections and delete the file in the gluster volume
> >> | >
> >> | > Couple of questions:
> >> | > 1. Should Block API have sub API's such as FileCreate, FileList,
> >> | > FileResize, File delete and etc then get it used in Block API as they
> >> | > mostly deal with files.
> >> | >
> >> |
> >> | IMO, Heketi should not expose any File related API. It should only have
> >> | APIs to service request for block devices; how the block devices are
> >> | created and modified is an implementation detail.
> >> |
> >> |
> >> | > 2. How do we create the actual file in the volume, meaning using FUSE
> >> | > mount (which may involve an extra process running) or gfapi, again if gfapi
> >> | > should we go with c API's, python bindings or go bindings ?
> >> | >
> >> | 3. Should we get targetcli related (LUN exporting) setup done from heketi
> >> | > or do we seek help from gdeploy for this ?
> >> | >
> >> |
> >> | I would prefer to either have it in Heketi or in Kubernetes. If the API in
> >> | Heketi promises just the creation of block device, then the rest of the
> >> | implementation should be in Kubernetes(the export part). If the API in
> >> | Heketi promises create and export both, I would say Heketi should have the
> >> | implementation within itself.
> >> |
> >> |
> >>
> >> IMO, we should not think about how the clients ( ex: k8s) use it, because there may be different clients.
> >> We should concentrate mainly on 'in/out' of block API in Heketi. Regardless of which client, the API should act the same way.
> >>
> >> --Humble
> >> _______________________________________________
> >> Gluster-devel mailing list
> >> Gluster-devel at gluster.org
> >> http://www.gluster.org/mailman/listinfo/gluster-devel
> >
> > _______________________________________________
> > Gluster-devel mailing list
> > Gluster-devel at gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-devel
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: not available
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20160919/29e99a92/attachment.sig>
More information about the Gluster-devel
mailing list