<div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;font-size:small"><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Mar 21, 2019 at 6:31 PM Xiubo Li <<a href="mailto:xiubli@redhat.com">xiubli@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<div class="gmail-m_8724098580848434547moz-cite-prefix">On 2019/3/21 18:09, Prasanna Kalever
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, Mar 21, 2019 at
9:00 AM Xiubo Li <<a href="mailto:xiubli@redhat.com" target="_blank">xiubli@redhat.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote">
<div>
<p class="gmail-m_8724098580848434547gmail-m_-8427338770979236914part">All,</p>
<p class="gmail-m_8724098580848434547gmail-m_-8427338770979236914part">I am one of
the contributor for<span> </span><a href="https://github.com/gluster/gluster-block" rel="noopener" target="_blank">gluster-block</a>[1] project,
and also I contribute to linux kernel and<span> </span><a href="https://github.com/open-iscsi" rel="noopener" target="_blank">open-iscsi</a>
project.[2]</p>
<p class="gmail-m_8724098580848434547gmail-m_-8427338770979236914part">NBD was
around for some time, but in recent time, linux
kernel’s Network Block Device (NBD) is enhanced and
made to work with more devices and also the option to
integrate with netlink is added. So, I tried to
provide a glusterfs client based NBD driver recently.
Please refer<span> </span><a href="https://github.com/gluster/glusterfs/issues/633" rel="noopener" target="_blank">github issue #633</a>[3], and
good news is I have a working code, with most basic
things @<span> </span><a href="https://github.com/gluster/nbd-runner" rel="noopener" target="_blank">nbd-runner project</a>[4].</p>
<p class="gmail-m_8724098580848434547gmail-m_-8427338770979236914part">While this
email is about announcing the project, and asking for
more collaboration, I would also like to discuss more
about the placement of the project itself. Currently
nbd-runner project is expected to be shared by our
friends at Ceph project too, to provide NBD driver for
Ceph. I have personally worked with some of them
closely while contributing to open-iSCSI project, and
we would like to take this project to great success.</p>
<p class="gmail-m_8724098580848434547gmail-m_-8427338770979236914part">Now few
questions:</p>
<ol class="gmail-m_8724098580848434547gmail-m_-8427338770979236914part">
<li>Can I continue to use<span> </span><a href="http://github.com/gluster/nbd-runner" rel="noopener" target="_blank">http://github.com/gluster/nbd-runner</a><span> </span>as
home for this project, even if its shared by other
filesystem projects?</li>
</ol>
<ul class="gmail-m_8724098580848434547gmail-m_-8427338770979236914part">
<li>I personally am fine with this.</li>
</ul>
<ol start="2" class="gmail-m_8724098580848434547gmail-m_-8427338770979236914part">
<li>Should there be a separate organization for this
repo?</li>
</ol>
<ul class="gmail-m_8724098580848434547gmail-m_-8427338770979236914part">
<li>While it may make sense in future, for now, I am
not planning to start any new thing?</li>
</ul>
<p class="gmail-m_8724098580848434547gmail-m_-8427338770979236914part">It would be
great if we have some consensus on this soon as
nbd-runner is a new repository. If there are no
concerns, I will continue to contribute to the
existing repository.</p>
</div>
</blockquote>
<div><br>
</div>
<div>
<div><span></span>Thanks Xiubo Li,
for finally sending this email out. Since this email is
out on gluster mailing list, I would like to take a
stand from gluster community point of view *only* and
share my views.</div>
<div><br>
</div>
<div>My honest answer is "If we want
to maintain this within gluster org, then 80% of the
effort is common/duplicate of what we did all these days
with gluster-block",</div>
<div><br>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<p>The great idea came from Mike Christie days ago and the
nbd-runner project's framework is initially emulated from
tcmu-runner. This is why I name this project as nbd-runner, which
will work for all the other Distributed Storages, such as
Gluster/Ceph/Azure, as discussed with Mike before. <br>
</p>
<p>nbd-runner(NBD proto) and tcmu-runner(iSCSI proto) are almost the
same and both are working as lower IO(READ/WRITE/...) stuff, not
the management layer like ceph-iscsi-gateway and gluster-block
currently do.<br>
</p>
<p>Currently since I only implemented the Gluster handler and also
using the RPC like glusterfs and gluster-block, most of the other
code (about 70%) in nbd-runner are for the NBD proto and these are
very different from tcmu-runner/glusterfs/gluster-block projects,
and there are many new features in NBD module that not yet
supported and then there will be more different in future.</p>
<p>
The framework coding has been done and the nbd-runner project is
already stable and could already work well for me now.</p>
<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<div>
<div>like:</div>
<div>* rpc/socket code</div>
<div>* cli/daemon parser/helper
logics</div>
<div>* gfapi util functions</div>
<div>* logger framework</div>
<div>* inotify & dyn-config
threads</div>
</div>
</div>
</div>
</div>
</blockquote>
<p>Yeah, these features were initially from tcmu-runner project,
Mike and I coded two years ago. Currently nbd-runner also has
copied them from tcmu-runner.<br></p></div></blockquote><div><br></div><div><div class="gmail_default"><span style="font-family:arial,helvetica,sans-serif;font-size:small"></span><font face="arial, helvetica, sans-serif">I don't think tcmu-runner has any of,<br><br></font></div><div class="gmail_default"><font face="arial, helvetica, sans-serif">-> cli/daemon approach routines</font></div><div class="gmail_default"><font face="arial, helvetica, sans-serif">-> rpc low-level clnt/svc routines</font></div><div class="gmail_default"><font face="arial, helvetica, sans-serif">-> gfapi level file create/delete util functions </font></div><div class="gmail_default"><font face="arial, helvetica, sans-serif">-> Json parser support</font></div><div class="gmail_default"><font face="arial, helvetica, sans-serif">-> socket bound/listener related functionalities</font></div><div class="gmail_default"><font face="arial, helvetica, sans-serif">-> autoMake build frame-work, and </font></div><div class="gmail_default"><font face="arial, helvetica, sans-serif">-> many other maintenance files</font></div><div class="gmail_default"><font face="arial, helvetica, sans-serif"><br></font></div><div class="gmail_default"><font face="arial, helvetica, sans-serif">I actually can go in detail and furnish a long list of reference made here and you cannot deny the fact, but its **all okay** to take references from other alike projects. But my intention was not to point about the copy made here, but rather saying we are just wasting our efforts rewriting, copy-pasting, maintaining and fixing the same functionality framework.</font></div><div class="gmail_default"><font face="arial, helvetica, sans-serif"><br></font></div><div class="gmail_default"><font face="arial, helvetica, sans-serif">Again all I'm trying to make is, if at all you want to maintain nbd client as part of <a href="http://gluster.org">gluster.org</a>, why not use gluster-block itself ? which is well tested and stable enough.</font></div><div class="gmail_default"><font face="arial, helvetica, sans-serif"><br></font></div><div class="gmail_default"><font face="arial, helvetica, sans-serif">Apart from all the examples I have mentioned in my previous thread, there are other great </font><span style="font-family:arial,helvetica,sans-serif">advantages from</span><font face="arial, helvetica, sans-serif"> user perspective as-well,</font><span style="font-family:arial,helvetica,sans-serif"> like:</span></div><div class="gmail_default"><span style="font-family:arial,helvetica,sans-serif"><br></span></div><div class="gmail_default"><span style="font-family:arial,helvetica,sans-serif">* The top layers such as heketi consuming gluster's block storage really don't have to care whether the backend provider is tcmu-runner or nbd-runner or qemu-tcmu or kernel loopback or fileIO or something else ...</span><br></div><div class="gmail_default"><font face="arial, helvetica, sans-serif">They simply call gluster-block and get a block device out there.</font></div><div class="gmail_default"><span style="font-family:arial,helvetica,sans-serif"><br></span></div><div class="gmail_default"><span style="font-family:arial,helvetica,sans-serif">* We can reuse the existing gluster-block's rest api interface too.</span><br></div><div class="gmail_default"><font face="arial, helvetica, sans-serif"><br></font></div><div class="gmail_default"><font face="arial, helvetica, sans-serif"><br></font></div><div class="gmail_default"><font face="arial, helvetica, sans-serif">** Believe me, over the years I have learned it from my experience and </font><span style="font-family:arial,helvetica,sans-serif">its a very fact </span><span style="font-family:arial,helvetica,sans-serif">that, we can save lot of energy and time by reusing existing stable framework rather than building a new one</span><span style="font-family:arial,helvetica,sans-serif"> from scratch **</span></div><div class="gmail_default"><font face="arial, helvetica, sans-serif"><br></font></div><div class="gmail_default"><font face="arial, helvetica, sans-serif"><br></font></div><div class="gmail_default"><font face="arial, helvetica, sans-serif">I will try to spend few hours over my weekends and send a nbd client application PR for gluster-block (IMO this shouldn't exceed ~200 lines), will request your review there.</font></div><div class="gmail_default"><br></div><div class="gmail_default"><font face="arial, helvetica, sans-serif"><br></font></div><div class="gmail_default"><font face="arial, helvetica, sans-serif">Cheers!</font></div><div class="gmail_default"><font face="arial, helvetica, sans-serif">--</font></div><div class="gmail_default"><font face="arial, helvetica, sans-serif">Prasanna</font></div></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF"><p>
</p>
<p>Very appreciated for you great ideas here Prasanna and hope
nbd-runner could be more generically and successfully used in
future.</p>
<p>BRs</p>
<p>Xiubo Li<br>
</p>
<p><br>
</p>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<div>
<div>* configure/Makefile/specfiles</div>
<div>* docsAboutGluster and etc ..</div>
<div><br>
</div>
<div>The repository gluster-block is
actually a home for all the block related stuff within
gluster and its designed to accommodate alike
functionalities, if I was you I would have simply copied
nbd-runner.c into <a href="https://github.com/gluster/gluster-block/tree/master/daemon/" target="_blank">https://github.com/gluster/gluster-block/tree/master/daemon/</a>
just like ceph plays it here <a href="https://github.com/ceph/ceph/blob/master/src/tools/rbd_nbd/rbd-nbd.cc" target="_blank">https://github.com/ceph/ceph/blob/master/src/tools/rbd_nbd/rbd-nbd.cc</a>
and be done.</div>
<div><br>
</div>
<div>Advantages of keeping nbd
client within gluster-block:</div>
<div>-> No worry about
maintenance code burdon</div>
<div>-> No worry about monitoring
a new component</div>
<div>-> shipping packages to
fedora/centos/rhel is handled</div>
<div>-> This helps improve and
stabilize the current gluster-block framework</div>
<div>-> We can build a common CI</div>
<div>-> We can use reuse common
test framework and etc ..</div>
<div><br>
</div>
<div>If you have an impression that
gluster-block is for management, then I would really
want to correct you at this point.</div>
<div><br>
</div>
<div>Some of my near future plans
for gluster-block:</div>
<div>* Allow exporting blocks with
FUSE access via fileIO backstore to improve large-file
workloads, draft: <a href="https://github.com/gluster/gluster-block/pull/58" target="_blank">https://github.com/gluster/gluster-block/pull/58</a></div>
<div>* Accommodate kernel loopback
handling for local only applications</div>
<div>* The same way we can
accommodate nbd app/client, and IMHO this effort
shouldn't take 1 or 2 days to get it merged with in
gluster-block and ready for a go release.</div>
<div><br>
</div>
<div><br>
</div>
<div>Hope that clarifies it.</div>
<div><br>
</div>
<div><br>
</div>
<div>Best Regards,</div>
<div>--</div>
<div>Prasanna<span></span></div>
</div>
<div> </div>
<blockquote class="gmail_quote">
<div>
<p class="gmail-m_8724098580848434547gmail-m_-8427338770979236914part">Regards,<br>
Xiubo Li (@lxbsz)</p>
<p class="gmail-m_8724098580848434547gmail-m_-8427338770979236914part">[1] -<span> </span><a href="https://github.com/gluster/gluster-block" rel="noopener" target="_blank">https://github.com/gluster/gluster-block</a><br>
[2] -<span> </span><a href="https://github.com/open-iscsi" rel="noopener" target="_blank">https://github.com/open-iscsi</a><br>
[3] -<span> </span><a href="https://github.com/gluster/glusterfs/issues/633" rel="noopener" target="_blank">https://github.com/gluster/glusterfs/issues/633</a><br>
[4] -<span> </span><a href="https://github.com/gluster/nbd-runner" rel="noopener" target="_blank">https://github.com/gluster/nbd-runner</a></p>
</div>
_______________________________________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@gluster.org" target="_blank">Gluster-devel@gluster.org</a><br>
<a href="https://lists.gluster.org/mailman/listinfo/gluster-devel" rel="noreferrer" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-devel</a></blockquote>
</div>
</div>
</div>
</blockquote>
<p><br>
</p>
</div>
</blockquote></div></div></div></div>