[Gluster-devel] GlusterD 2.0 status updates

[Atin Mukherjee]

On 09/08/2015 12:31 PM, Avra Sengupta wrote:
> On 09/07/2015 05:50 PM, Kaushal M wrote:
>> Hi Richard,
>> Thanks a lot for you feedback. I've done my replies inline.
>>
>> On Sat, Sep 5, 2015 at 5:46 AM, Richard Wareing <rwareing at fb.com> wrote:
>>> Hey Atin (and the wider community),
>>>
>>> This looks interesting, though I have a couple questions:
>>>
>>> 1. Language choice - Why the divergence from Python (which I'm no fan
>>> of) which is already heavily used in GlusterFS?  It seems a bit
>>> strange to me to introduce yet another language into the GlusterFS
>>> code base.  Doing this will make things less cohesive, harder to
>>> test, make it more difficult for contributors to understand the code
>>> base and improve their coding skills to be effective contributors. 
>>> I'm a bit concerned we are setting a precedent that development will
>>> switch to the new flavor of the day.  If a decision has been made to
>>> shift away from Python for the portions of GlusterFS where
>>> performance isn't a concern, will the portions currently written in
>>> Python be re-written as well?  I also question the wisdom of a
>>> language will a shallow developer pool, and a less development
>>> process (somewhat of an ironic choice IMHO).
>>>
>> One of our aims for GlusterD-2.0 was to switch to a higher level
>> language. While C is good for solving lower level performance critical
>> problems, it isn't best suited for the kind of management tasks we
>> want GlusterD to focus on. The choice of Go over Python as the higher
>> level language, was mainly driven by the following
>> - Go is easier to a hang of for a developer coming from a C
>> background. IMO for a developer new to both Go and Python, it's easier
>> to start producing working code in Go. The structure and syntax of the
>> language and the related tools make it easier.
>> - Go has built into the language support (think goroutines, channels)
>> for easily implementing the concurrency patterns that we have in the
>> current GlusterD codebase. This makes it easier for us think about
>> newer designs based on our understanding of the existing
>> implementation.
>> - We have concerns about the concurrency and threading capabilities of
>> Python. We have faced a lot of problems with doing concurrency and
>> threading in GlusterD (though this is mostly down to bad design).
>> Python has known issues with threading, which doesn't give us
>> confidence as python novices.
>> - Go has a pretty good standard library (possibly the best standard
>> library), which provides us with almost everything required. This
>> reduces the number of dependencies that we pull in.
>>
>> That's not to say Go doesn't have it's drawbacks. The major drawback
>> that I see currently with Go is that it doesn't have a way to do
>> plugins (dynamically load and execute binaries). But there have been
>> recent developments in this area. Go 1.5 landed support for building
>> Go packages as dynamic libraries. There is design ready to support
>> runtime loadability (plugins) ready, which is targeted for inclusion
>> in Go 1.6. As Go follows a 6 month release cycle, 1.6 is scheduled for
>> a Feb 2016 release, we need not wait too long for this to land. In the
>> meantime, we plan to build up the rest of the infrastructure required.
> [AVRA]: There are other components like snapshot, geo-rep management
> which are closely tied with the glusterd code base today. In order to
> scale glusterd we have to change the current design and most of these
> components might have to re-write some parts of it, to play well with
> the new glusterd. This is completely acceptable, as it brings along
> solution to a lot of pain points these components have been suffering from.
> 
> But we also have to look at the developer base currently contributing to
> these components too. Most of the snapshot and geo-rep code base (going
> with these examples as I myself have contributed code in them), is
> written in C and python, and the developers contributing to the same are
> well versed in those languages. For them to be expected to pick up a new
> language, and re-writing the components in that language, might be a bit
> far fetched. I would assume, glusterd would be providing an interface,
> for other components to interact with it, without enforcing the choice
> of language. And if so, would such an interface have any impact on the
> design or performance of these components.
We are not restricting ourselves to a specific language as far as
features pluggability is concerned and that's the end goal of it.  The
overall idea is to come up with a framework where each features
(independent of language) can be plugged in dynamically.

HTH,
Atin
> 
> 
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel