[Gluster-devel] Proposal for improving throughput for regression test

[Pranith Kumar Karampuri]

On 05/08/2015 08:54 PM, Justin Clift wrote:
> On 8 May 2015, at 10:02, Mohammed Rafi K C <rkavunga at redhat.com> wrote:
>> Hi All,
>>
>> As we all know, our regression tests are killing us. An average, one
>> regression will take approximately two and half hours to complete the
>> run. So i guess this is the right time to think about enhancing our
>> regression.
>>
>> Proposal 1:
>>
>> Create a new option for the daemons to specify that it is running as
>> test mode, then we can skip fsync calls used for data durability.
>>
>> Proposal 2:
>>
>> Use ip address instead of host name, because it takes some good amount
>> of time to resolve from host name, and even some times causes spurious
>> failure.
>>
>>
>> Proposal 3:
>> Each component has a lot of .t files and there is redundancy in tests,
>> We can do a rework to reduce the .t files and make less number of tests
>> that covers unit testing for a component , and run regression runs once
>> in a day (nightly) .
>>
>> Please provide your inputs for the proposed ideas , and feel free to add
>> a new idea.
> Proposal 4:
>
> Break the regression tests into parts that can be run in parallel.
>
> So, instead of the regression testing for a particular CR going from the
> first test to the last in a serial sequence, we break it up into a number
> of chunks (dir based?) and make each of these a task.
>
> That won't reduce the overall number of tests, but it should get the time
> down for the result to be finished.
>
> Caveat : We're going to need more VM's, as once we get into things
> queueing up it's not going to help. :/
Raghavendra Talur(CCed) did some work on this earlier by using more 
docker isntances on a single VM to get the running time under an hour.

Pranith
>
> + Justin
>
> --
> GlusterFS - http://www.gluster.org
>
> An open source, distributed file system scaling to several
> petabytes, and handling thousands of clients.
>
> My personal twitter: twitter.com/realjustinclift
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel