[automated-testing] [Gluster-Maintainers] seeking an update on the plans around automated tests and test framework for Gluster

[sankarshan]

I wanted to follow up on this.

At the last community meeting we talked about setting up the CI
environment (on CentOS CI?) so that there are regular runs and the
detail is available. I wanted to check if all the teething issues
around the topic have been addressed and we now have complete runs to
look at. Additionally, the Glusto developers have to plan for capacity
(time, effort) - the deployment of a new system inside the Gluster
infrastructure requires the team developing the application to be
available for addressing topics related to failures; debugging
failures and then proposing how those can be fixed.

While we are now informed that there are quite a few patches and new
tests, the questions raised in the original email are not completely
answered. I'd have preferred to put 7.3 through a test run prior to
release, but I am guessing that ship has sailed. This brings up the
need to have more specific and granular planning around the inclusion
of Glusto in the software supply chain. Also, the topic of upgrade
testing - is that still an OPEN item with Glusto or a solved problem?

If you'd prefer to have another set of meetings so that there is
better understanding of the expectations, I'm available today/Tue and
tomorrow/Wed. The next earliest would be the Tuesday in the next week.

On Wed, 5 Feb 2020 at 21:42, Amar Tumballi <amar at kadalu.io> wrote:

> Some updates inline.
>
> On Tue, Feb 4, 2020 at 12:11 PM sankarshan <sankarshan at kadalu.io> wrote:
>>
>> This is a good set of updates. If we wanted to discuss the impact of
>> this activity in terms of how they have been improving the user
>> experience for (a) specific workflows or, (b) specific workloads - how
>> can we measure the robustness of the current test library (or, batches
>> of tests) in Glusto? I'd like to see a conversation - perhaps at an
>> upcoming Community Meeting - from this perspective. The kind of
>> workloads for which Gluster is used is more or less well known by now.
>> So, being able to validate the readiness and extensive consumption of
>> Glusto tests from that perspective would be good to see.
>>
>
> Agree! I do see a lot of activity on Glusto-tests repository, but doesn't see where to find the run, or the result of the run. Also, haven't seen any bugs/issues raised from the testing using glusto.  While the number of patches is great, the value we need is surely through the ability to find use-case specific bugs which we couldn't test through regression framework in glusterfs repository.




-- 
sankarshan at kadalu.io | TZ: UTC+0530
kadalu.io : Making it easy to provision storage in k8s!