[CentOS-devel] functional testing as a contrib requirement
Karanbir Singh
mail-lists at karan.org
Thu Aug 13 00:09:28 UTC 2009
As a part of the contrib acceptance process, can we spread the load a
bit by saying that anyone who wants to submit a package, must also find
someone else ( other than their sponsor ) to write a set of acceptance
tests[1] for their package.
I realise that in some cases, its going to be non-trivial writing these
sort of tests, eg when looking at a new kernel module that needs
specific hardware to be present for it to do something testable. However
in many cases its really very trivial ( eg. writing tests for bind are
extremely easy, just need to get a set of configs into place and run a
dns lookup or a dozen ).
How I imagine this to work would be that a post-build testing harness
could do whatever environment/teardown is required, get a clean instance
installed, get the fresh packages into there and run the tests. Stage1
could be just a standard test harness that ensures the quality of the
rpms being built and how they work as a collection, tools like
repomanage etc could be part of Stage1.
Follow that up with the acceptance tests written specifically for the
package. Dont care what testing framework is used, even bash will work
in most cases ( shunit2 ? ), simpletest, rspec, python-unit. Whatever.
As long as it can be called from the command line, it should work.
Just wanted to get the idea out there and see what the general feeling
is about this. If people are seriously keen, we cn even rip out unit and
integration tests from inside package %build sections and run them
post-install :) Although as something to start with, lets not go that far.
[1]: http://en.wikipedia.org/wiki/Acceptance_testing
--
Karanbir Singh : http://www.karan.org/ : 2522219 at icq
More information about the CentOS-devel
mailing list