[Gluster-devel] [Gluster-users] Need testers for GlusterFS 3.4.4

[Ben Turner]

----- Original Message -----
> From: "BGM" <bernhard.glomm at ecologic.eu>
> To: "Vijay Bellur" <vbellur at redhat.com>
> Cc: "Ben Turner" <bturner at redhat.com>, "Justin Clift" <justin at gluster.org>, "Pranith Kumar Karampuri"
> <pkarampu at redhat.com>, gluster-users at gluster.org, "Gluster Devel" <gluster-devel at gluster.org>
> Sent: Wednesday, June 4, 2014 2:43:23 PM
> Subject: Re: [Gluster-users] [Gluster-devel] Need testers for GlusterFS 3.4.4
> 
> ain't got the time to participate right now,
> but could the test setup
> go into the proposed new gluster manual?
> like:
> - which tests (grouped into: build from source, running from .rpm/.deb
> binaries)
> - which layout (distributed, mirrored, ... )
> - which platform (hardware specs, virtualized on what)
> - ???
> then I would eventually catch up later on.
> we might get a cfengine/puppet framework to easily
> install and performance-test each release...
> ... a wiki/webform to upload the results?

I have a good setup with 10G NW + 10TB over a 12 drive RAID 6 that I use for gluster perf regression testing.  I usually run on a 2x2 volume with 4 clients but the automation is scalable.  It will be really easy for me to point at upstream bits just like I do for downstream.  I store the results on my jenkins server but also shoot out an email with a summary of the current run vs baseline.  I can't put my jenkins or the test systems on the net but I could point the report email and gluster-devel to report status?  Here is an example:

Performance regression results, logging package info:
**Servers are running**
Kernel: kernel-2.6.32-431.17.1.el6.x86_64
Gluster: glusterfs-3.6.0.11-1.el6rhs.x86_64
Server tuning - tuned-adm profile rhs-high-throughput

**Clients are running**
Kernel: kernel-2.6.32-431.el6.x86_64
Gluster: glusterfs-3.6.0.11-1.el6.x86_64
Mount type = glusterfs

*Starting tests*
Sequential write regression test complete, the return code was: 0

The results are:
decision parameters:
  sample type = throughput
  confidence threshold =  95.00 %
  max. pct. deviation =  11.00 %
  regression threshold =  10.00 % 
sample stats for baseline:
  min = 1492903.020000
  max = 1654043.810000
  mean = 1600066.944000
  sd = 63601.772075
  pct.dev. =  3.97 %
sample stats for current:
  min = 1629821.840000
  max = 1703702.990000
  mean = 1659824.372500
  sd = 31380.985345
  pct.dev. =  1.89 %
current mean improvement over baseline is  3.73 percent
magnitude of change is less than standard deviation of samples
t-test t-statistic = -1.703805 probability = 0.132195
t-test says that mean of two sample sets differs with probability  86.78%
probability that sample sets have same mean = 0.1322
sample sets are statistically indistinguishable for specified confidence level

Will this work for perf regression stuff?

-b
 
> best regards
> 
> Bernhard
> 
> 
> 
> > On 04.06.2014, at 18:57, Vijay Bellur <vbellur at redhat.com> wrote:
> > 
> >> On 06/04/2014 07:44 PM, Ben Turner wrote:
> >> ----- Original Message -----
> >>> From: "Justin Clift" <justin at gluster.org>
> >>> To: "Pranith Kumar Karampuri" <pkarampu at redhat.com>
> >>> Cc: "Ben Turner" <bturner at redhat.com>, gluster-users at gluster.org,
> >>> "Gluster Devel" <gluster-devel at gluster.org>
> >>> Sent: Wednesday, June 4, 2014 9:35:47 AM
> >>> Subject: Re: [Gluster-users] [Gluster-devel] Need testers for GlusterFS
> >>> 3.4.4
> >>> 
> >>>> On 04/06/2014, at 6:33 AM, Pranith Kumar Karampuri wrote:
> >>>> On 06/04/2014 01:35 AM, Ben Turner wrote:
> >>>>>> Sent: Thursday, May 29, 2014 6:12:40 PM
> >>> <snip>
> >>>>> FSSANITY_TEST_LIST: arequal bonnie glusterfs_build compile_kernel
> >>>>> dbench
> >>>>> dd ffsb fileop fsx fs_mark iozone locks ltp multiple_files
> >>>>> posix_compliance postmark read_large rpc syscallbench tiobench
> >>>>> 
> >>>>> I am starting on NFS now, I'll have results tonight or tomorrow
> >>>>> morning.
> >>>>> I'll look updating the component scripts to work and run them as well.
> >>>> Thanks a lot for this ben.
> >>>> 
> >>>> Justin, Ben,
> >>>>     Do you think we can automate running of these scripts without a lot
> >>>>     of
> >>>>     human intervention? If yes, how can I help?
> >>>> 
> >>>> We can use that just before making any release in future :-).
> >>> 
> >>> 
> >>> It's a decent idea.  :)
> >>> 
> >>> Do you have time to get this up and running?
> >> 
> >> Yep, can do.  I'll see what else I can get going as well, I'll start with
> >> the sanity tests I mentioned above and go from there.  How often do we
> >> want these run?  Daily?  Weekly?  On GIT checkin?  Only on RC?
> > 
> > Daily would be great.
> > 
> > -Vijay
> > 
> > _______________________________________________
> > Gluster-users mailing list
> > Gluster-users at gluster.org
> > http://supercolony.gluster.org/mailman/listinfo/gluster-users
>