[CentOS-devel] a post-build test harness
mail-lists at karan.org
Fri Nov 28 15:56:15 UTC 2008
Jeff Johnson wrote:
> What is "Rspec"? I'm unfamiliar with the reference.
Essentially a unit testing framework that works on specifications rather
than tests. Let me see if I can come up with a semi functional example
to demonstrate what I have in mind.
> Per-pkg unit tests are likely best done by running rpmlint with a custom
I've looked at rpmlint a few times, let me look again. Perhaps most of
this might be plumbable via there.
> That introduces a 3rd layer, in addition to unit & integration testing,
> that indeed, a package appears in distribution URI's as intended.
I think some / most of this info might be extractable from createrep'd
metadata. Will need to dig more into that, at the moment just looking at
individual package testing, and some collection stuff. Eg. making sure
we get the right multilib stuff into the right place.
> One way to avoid the test for higher is to automate R++ when building.
> The problem these days is how to recognize templates for Release: so
> that an autoincrement might be performed reliably. I suspect that
> EVR autoincrement is an easier problem to solve than detecting
> failure after the build.
I agree with the auto bump, or check at the buildsystem level to make
sure that package-to-build is indeed higher. Its whats in place at the
moment here. However, its not easy to enforce on the main distro level
stuff, since we need to 'inhert' Rel: Ver: from upstream and match it
>> And having this harness pass or fail depending on the test outcome.
> The "fail" case is often painful, requiring human intervention.
> A more automated "fail" proof process flow is perhaps a better
> approach than a "fail" test.
> OTOH, detection is better than not testing at all, even if automated
> or prevention (through processes) is better than "fail" detection.
I think you put it better than I was able to, the whole aim is to catch
these things before it breaks or causes issues on production machines
out there. We dont really churn out that many packages per day that
would make such reporting get noisy and irritating, as yet.
: except packages we change, those get a .centos in the Release tag
and we can control it using homebrew scripting.
More information about the CentOS-devel