[Gluster-infra] [Gluster-devel] NetBSD tests not running to completion.
jdarcy at redhat.com
Thu Jan 7 09:45:29 UTC 2016
> If you do not
> prevent integration of patches that break NetBSD regression, that will get
> in, and tests will break one by one over time.
On the other hand, if patch A starts blocking all merges because of
NetBSD failures, then all platforms - including NetBSD - are denied the
benefit of fixes in patches B through Z. The real problem is that our
tests are so non-deterministic, so that they pass once and a patch gets
merged, but then fail every time after that. The signal-to-noise ratio
is really low, and for some reason this problem seems even worse on
NetBSD than on Linux. The cost of mandatory NetBSD tests is unbounded,
sometimes small enough to be a good investment (of our time) but
sometimes totally out of proportion to that benefit. That's not just
frustrating for developers; it's also a disservice to the vast majority
of our users who might be waiting on fixes. I'd prefer a "defined level
of effort" approach which *might* reduce the benefit we derive from
NetBSD testing but *definitely* keeps the cost under control.
More information about the Gluster-infra