<html><head></head><body>Personally, I'd like to see the glusterd service replaced by a k8s native controller (named "kluster").<br>
<br>
I'm hoping to use this vacation I'm currently on to write up a design doc.<br><br><div class="gmail_quote">On August 23, 2018 12:58:03 PM PDT, Michael Adam <obnox@samba.org> wrote:<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<pre class="k9mail">On 2018-07-25 at 06:38 -0700, Vijay Bellur wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;"> Hi all,<br></blockquote><br>Hi Vijay,<br><br>Thanks for announcing this to the public and making everyone<br>more aware of Gluster's focus on container storage!<br><br>I would like to add an additional perspective to this,<br>giving some background about the history and origins:<br><br>Integrating Gluster with kubernetes for providing<br>persistent storage for containerized applications is<br>not new. We have been working on this since more than<br>two years now, and it is used by many community users<br>and and many customers (of Red Hat) in production.<br><br>The original software stack used heketi<br>(<a href="https://github.com/heketi/heketi)">https://github.com/heketi/heketi)</a> as a high level service<br>interface for gluster to facilitate the easy self-service for<br>provisioning volumes in kubernetes. Heketi implemented some ideas<br>that were originally part of the glusterd2 plans already in a<br>separate, much more narrowly scoped project to get us started<br>with these efforts in the first place, and also went beyond those<br>original ideas. These features are now being merged into<br>glusterd2 which will in the future replace heketi in the<br>container storage stack.<br><br>We were also working on kubernetes itself, writing the<br>privisioners for various forms of gluster volumes in kubernets<br>proper (<a href="https://github.com/kubernetes/kubernetes)">https://github.com/kubernetes/kubernetes)</a> and also the<br>external storage repo<br>(<a href="https://github.com/kubernetes-incubator/external-storage).">https://github.com/kubernetes-incubator/external-storage).</a><br>Those provisioners will eventually be replaced by the mentioned<br>csi drivers. The expertise of the original kubernetes<br>development is now flowing into the CSI drivers.<br><br>The gluster-containers repository was created and used<br>for this original container-storage effort already.<br><br>The mentioned <a href="https://github.com/gluster/gluster-kubernetes">https://github.com/gluster/gluster-kubernetes</a><br>repository was not only the place for storing the deployment<br>artefacts and tools, but it was actually intended to be the<br>upstream home of the gluster-container-storage project.<br><br>In this view, I see the GCS project announced here<br>as a GCS version 2. The original first version,<br>even though never officially announced that widely in a formal<br>introduction like this, and never given a formal release<br>or version number (let me call it version one), was the<br>software stack described above and homed at the<br>gluster-kubernetes repository. If you look at this project<br>(and heketi), you see that it has a nice level of popularity.<br><br>I think we should make use of this traction instead of<br>ignoring the legacy, and turn gluster-kubernetes into the<br>home of GCS (v2). In my view, GCS (v2) will be about:<br><br>* replacing some of the components with newer, i.e.<br> - i.e. glusterd2 instead of the heketi and glusterd1 combo<br> - csi drivers (the new standard) instead of native<br> kubernetes plugins<br>* adding the operator feature,<br> (even though we are currently also working on an operator<br> for the current stack with heketi and traditional gluster,<br> since this will become important in production before<br> this v2 will be ready.)<br><br>These are my 2cents on this topic.<br>I hope someone finds them useful.<br><br>I am very excited to (finally) see the broader gluster<br>community getting more aligned behind the idea of bringing<br>our great SDS system into the space of containers! :-)<br><br>Cheers - Michael<br><br><br><br><br><br><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;"> We would like to let you know that some of us have started focusing on an<br> initiative called ‘Gluster for Container Storage’ (in short GCS). As of<br> now, one can already use Gluster as storage for containers by making use of<br> different projects available in github repositories associated with gluster<br> <https://github.com/gluster> & Heketi <https://github.com/heketi/heketi>.<br> The goal of the GCS initiative is to provide an easier integration of these<br> projects so that they can be consumed together as designed. We are<br> primarily focused on integration with Kubernetes (k8s) through this<br> initiative.<br> <br> Key projects for GCS include:<br> Glusterd2 (GD2)<br> <br> Repo: <a href="https://github.com/gluster/glusterd2">https://github.com/gluster/glusterd2</a><br> <br> The challenge we have with current management layer of Gluster (glusterd)<br> is that it is not designed for a service oriented architecture. Heketi<br> overcame this limitation and made Gluster consumable in k8s by providing<br> all the necessary hooks needed for supporting Persistent Volume Claims.<br> <br> Glusterd2 provides a service oriented architecture for volume & cluster<br> management. Gd2 also intends to provide many of the Heketi functionalities<br> needed by Kubernetes natively. Hence we are working on merging Heketi with<br> gd2 and you can follow more of this action in the issues associated with<br> the gd2 github repository.<br> gluster-block<br> <br> Repo: <a href="https://github.com/gluster/gluster-block">https://github.com/gluster/gluster-block</a><br> <br> This project intends to expose files in a gluster volume as block devices.<br> Gluster-block enables supporting ReadWriteOnce (RWO) PVCs and the<br> corresponding workloads in Kubernetes using gluster as the underlying<br> storage technology.<br> <br> Gluster-block is intended to be consumed by stateful RWO applications like<br> databases and k8s infrastructure services like logging, metrics etc.<br> gluster-block is more preferred than file based Persistent Volumes in K8s<br> for stateful/transactional workloads as it provides better performance &<br> consistency guarantees.<br> anthill / operator<br> <br> Repo: <a href="https://github.com/gluster/anthill">https://github.com/gluster/anthill</a><br> <br> This project aims to add an operator for Gluster in Kubernetes., Since it<br> is relatively new, there are areas where you can contribute to make the<br> operator experience better (please refer to the list of issues). This<br> project intends to make the whole Gluster experience in k8s much smoother<br> by automatic management of operator tasks like installation, rolling<br> upgrades etc.<br> gluster-csi-driver<br> <br> Repo: <a href="http://github.com/gluster/gluster-csi-driver">http://github.com/gluster/gluster-csi-driver</a><br> <br> This project will provide CSI (Container Storage Interface) compliant<br> drivers for GlusterFS & gluster-block in k8s.<br> gluster-kubernetes<br> <br> Repo: <a href="https://github.com/gluster/gluster-kubernetes">https://github.com/gluster/gluster-kubernetes</a><br> <br> This project is intended to provide all the required installation and<br> management steps for getting gluster up and running in k8s.<br> GlusterFS<br> <br> Repo: <a href="https://github.com/gluster/glusterfs">https://github.com/gluster/glusterfs</a><br> <br> GlusterFS is the main and the core repository of Gluster. To support<br> storage in container world, we don’t need all the features of Gluster.<br> Hence, we would be focusing on a stack which would be absolutely required<br> in k8s. This would allow us to plan and execute tests well, and also<br> provide users with a setup which works without too many options to tweak.<br> <br> Notice that glusterfs default volumes would continue to work as of now, but<br> the translator stack which is used in GCS will be much leaner and geared to<br> work optimally in k8s.<br> Monitoring<br> Repo: <a href="https://github.com/gluster/gluster-prometheus">https://github.com/gluster/gluster-prometheus</a><br> <br> As k8s ecosystem provides its own native monitoring mechanisms, we intend<br> to have this project be the placeholder for required monitoring plugins.<br> The scope of this project is currently WIP and we welcome your inputs to<br> shape the project.<br> <br> More details on this can be found at:<br> <a href="https://lists.gluster.org/pipermail/gluster-users/2018-July/034435.html">https://lists.gluster.org/pipermail/gluster-users/2018-July/034435.html</a><br> <br> Gluster-Containers<br> <br> *Repo: <a href="https://github.com/gluster/gluster-containers">https://github.com/gluster/gluster-containers</a><br> <https://github.com/gluster/gluster-containers>This repository provides<br> container specs / Dockerfiles that can be used with a container runtime<br> like cri-o & docker.Note that this is not an exhaustive or final list of<br> projects involved with GCS. We will continue to update the project list<br> depending on the new requirements and priorities that we discover in this<br> journey.*<br> <br> *We welcome you to join this journey by looking up the repositories and<br> contributing to them. As always, we are happy to hear your thoughts about<br> this initiative and please stay tuned as we provide periodic updates about<br> GCS here!Regards,*<br> <br> *Vijay*<br> <br> *(on behalf of Gluster maintainers @ Red Hat)*<br></blockquote><br><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;"><hr><br> Gluster-devel mailing list<br> Gluster-devel@gluster.org<br> <a href="https://lists.gluster.org/mailman/listinfo/gluster-devel">https://lists.gluster.org/mailman/listinfo/gluster-devel</a><br></blockquote><br></pre></blockquote></div></body></html>