[Gluster-devel] Early review/feedback for glusterfs 4.0 plan

[James]

On Thu, 2013-12-12 at 15:33 -0800, Anand Avati wrote:
> > 2b) If someone can help with the algorithms I mentioned in:
> >
> https://lists.gnu.org/archive/html/gluster-devel/2013-11/msg00135.html
> >   I think I'll be able to provide a convincing case that Gluster
> > management isn't as bad as you might be alluding to in your 4.0
> > comments. I think a lot of the GlusterFS core team are allergic to
> > Puppet. I think this is quite normal, because Gluster core is super
> > low level C, where as Puppet (a mostly declarative language) is
> super
> > high level and on the opposite side of this spectrum. However, I
> think
> > most people are discounting the importance of having something like
> > this. The future is _all_ configuration management. The early
> adopters
> > are almost mostly there.
> >
> 
> That's probably not entirely the case. I don't think there is anybody
> who
> disagrees on the need/importance of puppet and puppet-gluster. I also
> see
> how it becomes almost necessary on a 10k node cluster, and why it is
> important to make gluster integrate nicely with the puppet ecosystem.
> 
> At the same time, the reason why puppet-gluster module is having to
> solve
> complex issues is because gluster management isn't doing the right
> job.
Agreed.

>  The
> algorithms you are working for puppet-gluster are not puppet specific
> and
> their right "location" is in the core of gluster.
I agree that building some of these things into Gluster core makes
sense. At some point you end up building configuration management into
Gluster core. Somewhere along that road, I'd argue that it makes more
sense to have that logic externally in a declarative language, but I'm
sure I get there earlier than you do ;)

It doesn't mean that I won't like seeing some of it in Gluster, but I
think there are some benefits (LOC, reasoning about logic) that Puppet
can provide.

One problem is that doing this "blesses" Puppet as _the_ tool, when
someone else might prefer Chef. The good news there is that it would be
easy for someone to port between the two.

>  It doesn't feel right to
> see such "dense" logic in the puppet layer while that intelligence is
> needed pretty much for every one.
I try to "Push Puppet" to see how far it can go before it falls down :)
There are cracks, but I'm not done yet!
> 
> I'm hoping 4.0 will make the gluster-puppet module simple and elegant,
> and
> not "unnecessary".
I think it's elegant now, but I don't see it becoming simple.
Distributed things are by their very nature, quite complex. I'm sure you
probably know this more than I do.
> 
> 3) Between early Gluster x.y and 3.x I had to completely change
> > Puppet-Gluster to support the new management style. The current
> > Puppet-Gluster stuff is easily one of the most complicated Puppet
> > modules that exists _anywhere_! This is partly due to the fact that
> > while Gluster might be easy to manage (and even logical) for a
> human,
> > it is _not_ from an automation point of view.
> 
> 
> That's precisely my point as well. The complexity in the puppet module
> is
> for a problem which has to be fixed somewhere. And I think the right
> place
> for that extra logic to reside is within the gluster management layer.
> Puppet module should not need to know how gluster is distributing and
> replicating data - it is too low level/internal knowledge to be
> exposed up
> to a puppet-like layer.
> 
Don't users want to be able to decide these things?

Well I'm happy to try and work on these things. Since 4.0 isn't due for
some time, and also because current Puppet-Gluster could be seen as a
"research project", if you're able to help contribute some of the
algorithms that I'm trying to work on:

1) "Legacy" Gluster users (when 4.0 comes out) will still have awesome
management support. I think RHS might appreciate this.

2) We'll be able to test some of the algorithms well in advance before
all the Gluster code gets written.

Code is here:
https://lists.gnu.org/archive/html/gluster-devel/2013-11/txtdAJcI6dR34.txt

Nice having this chat with you,

Cheers!
James

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part
URL: <http://supercolony.gluster.org/pipermail/gluster-devel/attachments/20131213/8179a6d3/attachment-0001.sig>