[Gluster-devel] Performance Translators' Stability and Usefulness - Regression test outline

[Anand Avati]

>   I hope my extremely long winded rant here :) has explained adequately what
> I feel GlusterFS needs to have in a regression testing system.
>
> Geoff.

Geoff, and others, we hear you. Loud and clear. We acknowledge that
the project could have fared better if the QA processes were already
in place. Regarding the wiki page, which apparently has caused
confusion about misrepresenting the presence of QA processes, I would
like to clarify a few points. Even though the reasons involve some
details about the company (Z Research Inc.) I would still disclose it
just to clarify to the community that there were no evil intentions to
fool anyone.

First of all, in Sept 08, we did indeed want to kick in everything
what the document speaks about. We did make our QA recruitment, and
that person did start off with this very initiative. And as very first
line the page speaks, "processes to be followed at", they were in the
planning phase. That person (also the author of the first version of
the page) had to quit soon after that for personal reasons. And the
page was left stale -- yes, we acknowledge this was a mistake. Since
then we have had some restructuring within (and still recruiting) and
now, we do have a more dedicated QA team. And in the last month, as
Shehjar has mentioned earlier in this thread, a unit test and
regression test framework has indeed been under development, and we
intend to have the framework and test cases published in a couple of
months for public review. I was hoping we would not reach a stage to
discuss things at such degree of detail to clarify the situation.

But it is more important to make clear the actual reasons and events
to maintain a good relationship with the community. The summary of
what I intend to convey is that, what has happened (as described
above), is NOT a justification from the devs, but only to convey that
nothing was done on purpose as an eyewash, and did not have any
malicious intent of fooling the community whatsoever -- as alleged. We
acknowledge that the documentation is out of sync with the codebase,
including the QA page, but that is only due to shortage of resources
(than any kind of intentions.) The team here is spread thin and
everybody plays multiple roles (there is the whole commercial support
happening under the hood which drains out a lot of our time). We, more
than anyone acknowledge that the project would be nowhere close to
where it is, if not for the wonderful support, inputs, debugging and
help from you guys.

I would like to say that works for bringing about a regression suite
is actually under progress. We ourselves would have wanted it to be
out earlier than it is scheduled, but we are limited by the situation
we are in. There are reasons for the delay (our current QA lead is to
return from a long leave before the development on the framework
continues full scale.) And the fact that GlusterFS being this complex
system does not help in terms for getting help from non full-time
contributors, or having new recruits be productive from an early
stage.

There is definitely a concentration within the team towards stability.
For 2.1 we are bringing about a comprehensive internal resource
accounting to detect and corner leaks and features to dump internal
structures of the process for debugging -- all of this apart from the
testing suite under development. Moving forward we promise to keep the
release-X.X branches 'stable' and keep the master only "as stable as
possible". So, people who not want to be on the bleeding edge, please
switch to the release-2.0 branch for now.

Once again, thank you all for being patient and I hope to wind up this
thread with no sour feelings around! :)

Avati