[Gluster-devel] Proposal for improving throughput for regression test

[Justin Clift]

On 8 May 2015, at 17:37, Atin Mukherjee <amukherj at redhat.com> wrote:
On 05/08/2015 08:54 PM, Justin Clift wrote:
>> On 8 May 2015, at 10:02, Mohammed Rafi K C <rkavunga at redhat.com> wrote:
>>> Hi All,
>>> 
>>> As we all know, our regression tests are killing us. An average, one
>>> regression will take approximately two and half hours to complete the
>>> run. So i guess this is the right time to think about enhancing our
>>> regression.
>>> 
>>> Proposal 1:
>>> 
>>> Create a new option for the daemons to specify that it is running as
>>> test mode, then we can skip fsync calls used for data durability.
>>> 
>>> Proposal 2:
>>> 
>>> Use ip address instead of host name, because it takes some good amount
>>> of time to resolve from host name, and even some times causes spurious
>>> failure.
>>> 
>>> 
>>> Proposal 3:
>>> Each component has a lot of .t files and there is redundancy in tests,
>>> We can do a rework to reduce the .t files and make less number of tests
>>> that covers unit testing for a component , and run regression runs once
>>> in a day (nightly) .
>>> 
>>> Please provide your inputs for the proposed ideas , and feel free to add
>>> a new idea.
>> 
>> Proposal 4:
>> 
>> Break the regression tests into parts that can be run in parallel.
>> 
>> So, instead of the regression testing for a particular CR going from the
>> first test to the last in a serial sequence, we break it up into a number
>> of chunks (dir based?) and make each of these a task.
>> 
>> That won't reduce the overall number of tests, but it should get the time
>> down for the result to be finished.
>> 
>> Caveat : We're going to need more VM's, as once we get into things
>> queueing up it's not going to help. :/
> This could be really effective and I've been thinking about it for quite
> a long time :)

Yeah, the idea wasn't first thought of by me.  This seems like a good place
to suggest it though. :)

+ Justin

--
GlusterFS - http://www.gluster.org

An open source, distributed file system scaling to several
petabytes, and handling thousands of clients.

My personal twitter: twitter.com/realjustinclift