[Gluster-devel] GlusterD 2.0 status updates

[Kaushal M]

On Tue, Sep 8, 2015 at 4:53 PM, Atin Mukherjee <amukherj at redhat.com> wrote:
>
>
> On 09/08/2015 12:31 PM, Avra Sengupta wrote:
>> On 09/07/2015 05:50 PM, Kaushal M wrote:
>>> Hi Richard,
>>> Thanks a lot for you feedback. I've done my replies inline.
>>>
>>> On Sat, Sep 5, 2015 at 5:46 AM, Richard Wareing <rwareing at fb.com> wrote:
>>>> Hey Atin (and the wider community),
>>>>
>>>> This looks interesting, though I have a couple questions:
>>>>
>>>> 1. Language choice - Why the divergence from Python (which I'm no fan
>>>> of) which is already heavily used in GlusterFS?  It seems a bit
>>>> strange to me to introduce yet another language into the GlusterFS
>>>> code base.  Doing this will make things less cohesive, harder to
>>>> test, make it more difficult for contributors to understand the code
>>>> base and improve their coding skills to be effective contributors.
>>>> I'm a bit concerned we are setting a precedent that development will
>>>> switch to the new flavor of the day.  If a decision has been made to
>>>> shift away from Python for the portions of GlusterFS where
>>>> performance isn't a concern, will the portions currently written in
>>>> Python be re-written as well?  I also question the wisdom of a
>>>> language will a shallow developer pool, and a less development
>>>> process (somewhat of an ironic choice IMHO).
>>>>
>>> One of our aims for GlusterD-2.0 was to switch to a higher level
>>> language. While C is good for solving lower level performance critical
>>> problems, it isn't best suited for the kind of management tasks we
>>> want GlusterD to focus on. The choice of Go over Python as the higher
>>> level language, was mainly driven by the following
>>> - Go is easier to a hang of for a developer coming from a C
>>> background. IMO for a developer new to both Go and Python, it's easier
>>> to start producing working code in Go. The structure and syntax of the
>>> language and the related tools make it easier.
>>> - Go has built into the language support (think goroutines, channels)
>>> for easily implementing the concurrency patterns that we have in the
>>> current GlusterD codebase. This makes it easier for us think about
>>> newer designs based on our understanding of the existing
>>> implementation.
>>> - We have concerns about the concurrency and threading capabilities of
>>> Python. We have faced a lot of problems with doing concurrency and
>>> threading in GlusterD (though this is mostly down to bad design).
>>> Python has known issues with threading, which doesn't give us
>>> confidence as python novices.
>>> - Go has a pretty good standard library (possibly the best standard
>>> library), which provides us with almost everything required. This
>>> reduces the number of dependencies that we pull in.
>>>
>>> That's not to say Go doesn't have it's drawbacks. The major drawback
>>> that I see currently with Go is that it doesn't have a way to do
>>> plugins (dynamically load and execute binaries). But there have been
>>> recent developments in this area. Go 1.5 landed support for building
>>> Go packages as dynamic libraries. There is design ready to support
>>> runtime loadability (plugins) ready, which is targeted for inclusion
>>> in Go 1.6. As Go follows a 6 month release cycle, 1.6 is scheduled for
>>> a Feb 2016 release, we need not wait too long for this to land. In the
>>> meantime, we plan to build up the rest of the infrastructure required.
>> [AVRA]: There are other components like snapshot, geo-rep management
>> which are closely tied with the glusterd code base today. In order to
>> scale glusterd we have to change the current design and most of these
>> components might have to re-write some parts of it, to play well with
>> the new glusterd. This is completely acceptable, as it brings along
>> solution to a lot of pain points these components have been suffering from.
>>
>> But we also have to look at the developer base currently contributing to
>> these components too. Most of the snapshot and geo-rep code base (going
>> with these examples as I myself have contributed code in them), is
>> written in C and python, and the developers contributing to the same are
>> well versed in those languages. For them to be expected to pick up a new
>> language, and re-writing the components in that language, might be a bit
>> far fetched. I would assume, glusterd would be providing an interface,
>> for other components to interact with it, without enforcing the choice
>> of language. And if so, would such an interface have any impact on the
>> design or performance of these components.
> We are not restricting ourselves to a specific language as far as
> features pluggability is concerned and that's the end goal of it.  The
> overall idea is to come up with a framework where each features
> (independent of language) can be plugged in dynamically.

I'd like to clarify this a little bit. Yes, we are aiming to provide
an interface which the feature developers will make use of to
integrate with GlusterD. This interface would most likely be language
agnostic (it'd be awesome if we could stick to just Go here as it'd
reduce our maintenance effort). But this interface would be targeted
mainly towards filesystem features that would not require a huge deal
of specialized management intelligence. This kind of features would
require the some or all of following from GlusterD,
- A way to insert xlators into graph without messing with GlusterD
code (any xlator basically)
- A way to be able validate and to set options on a xlator (volume set)
- A way to create customized graphs (for example for rebalance daemon,
shd, tiering daemon, quotad etc)
- A way to define/manage daemons (the same as above)
- A way to create a cli commands (for example with quota, or the heal
command etc.)
- A way to hook into existing commands (for example for new cluster
xlators into the volume create command)
(This is most of the requirements I expect. Some more could be added here.)

This interface will allow new features to become integrated into
GlusterD without requiring the developer to write a lot of code. For
some of the above (inserting xlators into graph and volume set),
interfaces could be implemented where developers can get away without
writing code at all. Ideally the developers would only need to write
code only for specific intelligence requirements, like the replicate
xlator would probably need to write code to validate enough bricks are
available during volume create. This interface should be enough for
most cases.

But for something like snapshot, I don't think this interface will
suffice. Snapshot in it's current implementation, contains a huge
amount logic within GlusterD and an integral part. I expect this to
remain the same for GlusterD-2.0 as well. If all this logic were to be
still written in C, I think it would defeat the purpose of moving to a
higher level language for the management code. I've already given the
reasons for the choice of Go over Python. Also, for most of the team
who work on GlusterD, both Go and Python were new. So we are learning
Go on the way as well. From what we've observed till now, we've
started producing working code quickly. So I believe the concern over
learning Go is not really a large one.

Also, I don't expect the a large amount of code to be written for
snapshot in GlusterD-2.0. Most of the code for written for
implementing snapshot in the current GlusterD, was to prepare the
infrastructure to get snapshot to work, as GlusterD didn't provide
what was required. With GlusterD-2.0, this wouldn't be required as we
are using the experiences gained during the development of the
snapshot infra as the basis for GlusterD-2.0. The snapshot
contributors would only need to work on the snapshot features.

I'll also add that we don't expect any contributor to work on
integrating their features into GlusterD on their own. We will be
around to help.

> HTH,
> Atin
>>
>>
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-devel