[Gluster-devel] Perf regression for beta2.

Benjamin Turner bennyturns at gmail.com
Tue Jun 17 20:20:34 UTC 2014 

All performance regression tests passed.  I am reworking the results email
to be less wordy and point to the current / historical results in a nice
graph / web page.  This will be a summary in the future instead of all the
details like below.  Oh and I compared the beta2 bits
to glusterfs-3.4.0.59rhs-1.el6rhs.x86_64.rpm as I didn't have time to rerun
baselines on community 3.4 bits.

Performance regression results, logging package info:
**Servers are running**
Kernel: kernel-2.6.32-431.el6.x86_64
Gluster: glusterfs-3.5.1-0.3.beta2.el6.x86_64
Server tuning - tuned-adm profile rhs-high-throughput

**Clients are running**
Kernel: kernel-2.6.32-431.el6.x86_64
Gluster: glusterfs-3.5.1-0.3.beta2.el6.x86_64
Mount type = glusterfs

*Starting tests*
Sequential write regression test complete, the return code was: 0

The results are:
decision parameters:
  sample type = throughput
  confidence threshold =  95.00 %
  max. pct. deviation =  11.00 %
  regression threshold =  10.00 %
sample stats for baseline:
  min = 1540953.410000 <callto:1540953.410000>
  max = 1605080.590000 <callto:1605080.590000>
  mean = 1587339.478000 <callto:1587339.478000>
  sd = 26755.449462 <callto:26755.449462>
  pct.dev. =  1.69 %
sample stats for current:
  min = 1468745.380000 <callto:1468745.380000>
  max = 1525744.160000 <callto:1525744.160000>
  mean = 1500096.920000 <callto:1500096.920000>
  sd = 27208.783338 <callto:27208.783338>
  pct.dev. =  1.81 %
current mean improvement over baseline is -5.50 percent
magnitude of change is at least  3.68%
t-test t-statistic = 4.825614 probability = 0.001909
t-test says that mean of two sample sets differs with probability  99.81%
probability that sample sets have same mean = 0.0019

Sequential read regression test complete, the return code was: 0

The results are:
decision parameters:
  sample type = throughput
  confidence threshold =  95.00 %
  max. pct. deviation =  13.00 %
  regression threshold =  10.00 %
sample stats for baseline:
  min = 2901850.550000 <callto:2901850.550000>
  max = 2929673.440000 <callto:2929673.440000>
  mean = 2919766.100000 <callto:2919766.100000>
  sd = 13669.228331 <callto:13669.228331>
  pct.dev. =  0.47 %
sample stats for current:
  min = 2860390.300000 <callto:2860390.300000>
  max = 3046184.410000 <callto:3046184.410000>
  mean = 2959399.192500 <callto:2959399.192500>
  sd = 89189.147258 <callto:89189.147258>
  pct.dev. =  3.01 %
current mean improvement over baseline is  1.36 percent
magnitude of change is less than standard deviation of samples
t-test t-statistic = -0.996395 probability = 0.352247
t-test says that mean of two sample sets differs with probability  64.78%
probability that sample sets have same mean = 0.3522
sample sets are statistically indistinguishable for specified confidence
level

Random read regression test complete, the return code was: 0
The results are:
decision parameters:
  sample type = throughput
  confidence threshold =  95.00 %
  max. pct. deviation =  12.00 %
  regression threshold =  10.00 %
sample stats for baseline:
  min = 254828.820000 <callto:254828.820000>
  max = 274380.550000 <callto:274380.550000>
  mean = 263317.932500 <callto:263317.932500>
  sd = 9267.903360
  pct.dev. =  3.52 %
sample stats for current:
  min = 277584.450000 <callto:277584.450000>
  max = 292256.150000 <callto:292256.150000>
  mean = 284253.572500 <callto:284253.572500>
  sd = 6573.110614
  pct.dev. =  2.31 %
current mean improvement over baseline is  7.95 percent
magnitude of change is at least  4.43%
t-test t-statistic = -3.685135 probability = 0.010268
t-test says that mean of two sample sets differs with probability  98.97%
probability that sample sets have same mean = 0.0103
current sample set is statistically better than baseline sample set

Random write regression test complete, the return code was: 0

The results are:
decision parameters:
  sample type = throughput
  confidence threshold =  95.00 %
  max. pct. deviation =  40.00 %
  regression threshold =  10.00 %
sample stats for baseline:
  min = 248391.310000 <callto:248391.310000>
  max = 378684.010000 <callto:378684.010000>
  mean = 315515.882000 <callto:315515.882000>
  sd = 61903.795184 <callto:61903.795184>
  pct.dev. = 19.62 %
sample stats for current:
  min = 347379.690000 <callto:347379.690000>
  max = 445939.370000 <callto:445939.370000>
  mean = 409822.955000 <callto:409822.955000>
  sd = 44118.399188 <callto:44118.399188>
  pct.dev. = 10.77 %
current mean improvement over baseline is 29.89 percent
magnitude of change is at least 10.27%
t-test t-statistic = -2.556531 probability = 0.037742
t-test says that mean of two sample sets differs with probability  96.23%
probability that sample sets have same mean = 0.0377
current sample set is statistically better than baseline sample set

Sequential write NFS regression test complete, the return code was: 0

The results are:
decision parameters:
  sample type = throughput
  confidence threshold =  95.00 %
  max. pct. deviation =  13.00 %
  regression threshold =  10.00 %
sample stats for baseline:
  min = 866772.180000 <callto:866772.180000>
  max = 979631.270000 <callto:979631.270000>
  mean = 910415.654000 <callto:910415.654000>
  sd = 53605.065583 <callto:53605.065583>
  pct.dev. =  5.89 %
sample stats for current:
  min = 882409.120000 <callto:882409.120000>
  max = 965392.330000 <callto:965392.330000>
  mean = 935562.377500 <callto:935562.377500>
  sd = 38435.561104 <callto:38435.561104>
  pct.dev. =  4.11 %
current mean improvement over baseline is  2.76 percent
magnitude of change is less than standard deviation of samples
t-test t-statistic = -0.785909 probability = 0.457694
t-test says that mean of two sample sets differs with probability  54.23%
probability that sample sets have same mean = 0.4577
sample sets are statistically indistinguishable for specified confidence
level

Sequential read NFS regression test complete, the return code was: 0

The results are:
decision parameters:
  sample type = throughput
  confidence threshold =  95.00 %
  max. pct. deviation =  13.00 %
  regression threshold =  10.00 %
sample stats for baseline:
  min = 1771215.420000 <callto:1771215.420000>
  max = 1783981.700000 <callto:1783981.700000>
  mean = 1780125.490000 <callto:1780125.490000>
  sd = 5241.213206
  pct.dev. =  0.29 %
sample stats for current:
  min = 1962247.520000 <callto:1962247.520000>
  max = 1975594.330000 <callto:1975594.330000>
  mean = 1971027.137500 <callto:1971027.137500>
  sd = 6292.201659
  pct.dev. =  0.32 %
current mean improvement over baseline is 10.72 percent
magnitude of change is at least 10.40%
t-test t-statistic = -49.792107 probability = 0.000000
t-test says that mean of two sample sets differs with probability 100.00%
probability that sample sets have same mean = 0.0000
current sample set is statistically better than baseline sample set

Random read NFS regression test complete, the return code was: 0
The results are:
decision parameters:
  sample type = throughput
  confidence threshold =  95.00 %
  max. pct. deviation =  12.00 %
  regression threshold =  10.00 %
sample stats for baseline:
  min = 471409.350000 <callto:471409.350000>
  max = 486627.100000 <callto:486627.100000>
  mean = 478061.648000 <callto:478061.648000>
  sd = 5948.858016
  pct.dev. =  1.24 %
sample stats for current:
  min = 542448.920000 <callto:542448.920000>
  max = 548563.640000 <callto:548563.640000>
  mean = 544958.915000 <callto:544958.915000>
  sd = 2878.456606
  pct.dev. =  0.53 %
current mean improvement over baseline is 13.99 percent
magnitude of change is at least 12.75%
t-test t-statistic = -20.453073 probability = 0.000000
t-test says that mean of two sample sets differs with probability 100.00%
probability that sample sets have same mean = 0.0000
current sample set is statistically better than baseline sample set

Random write NFS regression test complete, the return code was: 0

The results are:
decision parameters:
  sample type = throughput
  confidence threshold =  95.00 %
  max. pct. deviation =  11.00 %
  regression threshold =  10.00 %
sample stats for baseline:
  min = 136566.050000 <callto:136566.050000>
  max = 162768.380000 <callto:162768.380000>
  mean = 146064.294000 <callto:146064.294000>
  sd = 10565.636623 <callto:10565.636623>
  pct.dev. =  7.23 %
sample stats for current:
  min = 135400.510000 <callto:135400.510000>
  max = 149097.090000 <callto:149097.090000>
  mean = 140298.877500 <callto:140298.877500>
  sd = 6068.116248
  pct.dev. =  4.33 %
current mean improvement over baseline is -3.95 percent
magnitude of change is less than standard deviation of samples
t-test t-statistic = 0.963490 probability = 0.367397
t-test says that mean of two sample sets differs with probability  63.26%
probability that sample sets have same mean = 0.3674
sample sets are statistically indistinguishable for specified confidence
level
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-devel/attachments/20140617/c3944706/attachment-0001.html>

More information about the Gluster-devel mailing list