As a part of the contrib acceptance process, can we spread the load a bit by saying that anyone who wants to submit a package, must also find someone else ( other than their sponsor ) to write a set of acceptance tests[1] for their package.
I realise that in some cases, its going to be non-trivial writing these sort of tests, eg when looking at a new kernel module that needs specific hardware to be present for it to do something testable. However in many cases its really very trivial ( eg. writing tests for bind are extremely easy, just need to get a set of configs into place and run a dns lookup or a dozen ).
How I imagine this to work would be that a post-build testing harness could do whatever environment/teardown is required, get a clean instance installed, get the fresh packages into there and run the tests. Stage1 could be just a standard test harness that ensures the quality of the rpms being built and how they work as a collection, tools like repomanage etc could be part of Stage1.
Follow that up with the acceptance tests written specifically for the package. Dont care what testing framework is used, even bash will work in most cases ( shunit2 ? ), simpletest, rspec, python-unit. Whatever. As long as it can be called from the command line, it should work.
Just wanted to get the idea out there and see what the general feeling is about this. If people are seriously keen, we cn even rip out unit and integration tests from inside package %build sections and run them post-install :) Although as something to start with, lets not go that far.
[1]: http://en.wikipedia.org/wiki/Acceptance_testing