[Gluster-devel] GlusterD 2.0 status updates

[Avra Sengupta]

On 09/07/2015 05:50 PM, Kaushal M wrote:
> Hi Richard,
> Thanks a lot for you feedback. I've done my replies inline.
>
> On Sat, Sep 5, 2015 at 5:46 AM, Richard Wareing <rwareing at fb.com> wrote:
>> Hey Atin (and the wider community),
>>
>> This looks interesting, though I have a couple questions:
>>
>> 1. Language choice - Why the divergence from Python (which I'm no fan of) which is already heavily used in GlusterFS?  It seems a bit strange to me to introduce yet another language into the GlusterFS code base.  Doing this will make things less cohesive, harder to test, make it more difficult for contributors to understand the code base and improve their coding skills to be effective contributors.  I'm a bit concerned we are setting a precedent that development will switch to the new flavor of the day.  If a decision has been made to shift away from Python for the portions of GlusterFS where performance isn't a concern, will the portions currently written in Python be re-written as well?  I also question the wisdom of a language will a shallow developer pool, and a less development process (somewhat of an ironic choice IMHO).
>>
> One of our aims for GlusterD-2.0 was to switch to a higher level
> language. While C is good for solving lower level performance critical
> problems, it isn't best suited for the kind of management tasks we
> want GlusterD to focus on. The choice of Go over Python as the higher
> level language, was mainly driven by the following
> - Go is easier to a hang of for a developer coming from a C
> background. IMO for a developer new to both Go and Python, it's easier
> to start producing working code in Go. The structure and syntax of the
> language and the related tools make it easier.
> - Go has built into the language support (think goroutines, channels)
> for easily implementing the concurrency patterns that we have in the
> current GlusterD codebase. This makes it easier for us think about
> newer designs based on our understanding of the existing
> implementation.
> - We have concerns about the concurrency and threading capabilities of
> Python. We have faced a lot of problems with doing concurrency and
> threading in GlusterD (though this is mostly down to bad design).
> Python has known issues with threading, which doesn't give us
> confidence as python novices.
> - Go has a pretty good standard library (possibly the best standard
> library), which provides us with almost everything required. This
> reduces the number of dependencies that we pull in.
>
> That's not to say Go doesn't have it's drawbacks. The major drawback
> that I see currently with Go is that it doesn't have a way to do
> plugins (dynamically load and execute binaries). But there have been
> recent developments in this area. Go 1.5 landed support for building
> Go packages as dynamic libraries. There is design ready to support
> runtime loadability (plugins) ready, which is targeted for inclusion
> in Go 1.6. As Go follows a 6 month release cycle, 1.6 is scheduled for
> a Feb 2016 release, we need not wait too long for this to land. In the
> meantime, we plan to build up the rest of the infrastructure required.
[AVRA]: There are other components like snapshot, geo-rep management 
which are closely tied with the glusterd code base today. In order to 
scale glusterd we have to change the current design and most of these 
components might have to re-write some parts of it, to play well with 
the new glusterd. This is completely acceptable, as it brings along 
solution to a lot of pain points these components have been suffering from.

But we also have to look at the developer base currently contributing to 
these components too. Most of the snapshot and geo-rep code base (going 
with these examples as I myself have contributed code in them), is 
written in C and python, and the developers contributing to the same are 
well versed in those languages. For them to be expected to pick up a new 
language, and re-writing the components in that language, might be a bit 
far fetched. I would assume, glusterd would be providing an interface, 
for other components to interact with it, without enforcing the choice 
of language. And if so, would such an interface have any impact on the 
design or performance of these components.