[Gluster-devel] Pluggable interface for erasure coding?

[Xavier Hernandez]

March 8th also works for me.

On 03/03/17 06:18, Pranith Kumar Karampuri wrote:
> 8th March works for me. 9th I am travelling. Xavi is well versed with
> this part of the code, so even if I am not available, you guys should
> pick the time which is convenient for Xavi.
>
> The following may be of help for this discussion.
> https://github.com/gluster/glusterfs/blob/master/doc/developer-guide/ec-implementation.md
>
> If my memory serves right, Xavi's implementation outperformed
> liberasure, that is the reason we didn't integrate with liberasure sooner.
>
>
> On Fri, Mar 3, 2017 at 7:01 AM, Per Simonsen <per.simonsen at memoscale.com
> <mailto:per.simonsen at memoscale.com>> wrote:
>
>     Hi,
>
>     I forgot to add the time zone: the suggested time is 10 am (GMT+1).
>
>     Best,
>     Per
>
>     On Thu, Mar 2, 2017 at 5:19 PM, Per Simonsen
>     <per.simonsen at memoscale.com <mailto:per.simonsen at memoscale.com>> wrote:
>
>         Hi,
>
>         I suggest that we set up an online meeting next week to discuss
>         the erasure coding features as well as possible implementations
>         of a plugin architecture. We also have some experience
>         integrating with the liberasure library mentioned by Prashant
>         which we can share.
>
>         Does 10 am on Wednesday(8th of March) or Thursday(9th of March)
>         next week work for you guys?
>
>         Best,
>         Per Simonsen
>         CEO
>         MemoScale
>
>
>         On Thu, Mar 2, 2017 at 12:00 AM, Xavier Hernandez
>         <xhernandez at datalab.es <mailto:xhernandez at datalab.es>> wrote:
>
>             Hi Niels,
>
>             On 02/03/17 07:58, Niels de Vos wrote:
>
>                 Hi guys,
>
>                 I think this is a topic/question that has come up
>                 before, but I can not
>                 find any references or feature requests related to it.
>                 Because there are
>                 different libraries for Erasure Coding, it would be
>                 interesting to be
>                 able to select alternatives to the bundled
>                 implementation that Gluster
>                 has.
>
>
>             I agree.
>
>                 Are there any plans to make the current Erasure Coding
>                 implementation more pluggable?
>
>
>             Yes. I've had this in my todo list for a long time. Once I
>             even tried to implement the necessary infrastructure but
>             didn't finish and now the code has changed too much to reuse it.
>
>                 Would this be a possible feature request,
>                 or would it require a major rewrite of the current
>                 interface?
>
>
>             At the time I tried it, it required major changes. Now that
>             the code has been considerably restructured to incorporate
>             the dynamic code generation feature, maybe it doesn't
>             require so many changes, though I'm not sure.
>
>
>                 Here at FAST [0] I have briefly spoken to Per Simonsen
>                 from MemoScale
>                 [1]. This company offers a (proprietary) library for
>                 Erasure Coding,
>                 optimized for different architectures, and  with some
>                 unique(?) features
>                 for recovering a failed fragment/disk. If Gluster allows
>                 alternative
>                 implementations for the encoding, it would help
>                 organisations and
>                 researchers to get results of their work in a
>                 distributed filesystem.
>                 And with that, spread the word about how easy to adapt
>                 and extend
>                 Gluster is :-)
>
>
>             That could be interesting. Is there any place where I can
>             find additional information about the features of this library ?
>
>             Xavi
>
>
>
>                 Thanks,
>                 Niels
>
>
>                 0. https://www.usenix.org/conference/fast17
>                 <https://www.usenix.org/conference/fast17>
>                 1. https://memoscale.com/
>
>
>
>
>
>
>
> --
> Pranith