[CentOS-devel] a post-build test harness

Wed Nov 26 17:48:47 UTC 2008
Seth Vidal <skvidal at fedoraproject.org>


On Wed, 26 Nov 2008, Karanbir Singh wrote:

> Hi,
>
> One of the things that I'd like to get working sometime is a proper 
> post-build test harness for rpm's. At the moment, what we do is quite basic, 
> along with rpmdiff and some comparison tools we do a string of bash-written 
> script tests. There must be another way of doing this, a sort of rpm::unit, 
> for the lack of a better name, but modelled on Rspec. To illustrate :
>
> For Pkg in PackageList: {
>  ensure Pkg.SIGGPG => present;
> };
>
> Makes sure that Packages we are testing are signed, I guess it could also be 
> ensure=> 'blahblah'; to make sure its signed with a specific key.
>
> A more complex test might be to make sure that content in multilib pkgs match 
> for overlapping content + multilib acceptable content.
>
> An even more complex test might be to make sure any rpm that drops an init 
> script also has a chkconfig stanza in the %post / %pre ( as an example ).
>
> The first step would, of course, be to look at doing this per package, and 
> then growing it to work with multiple packages ( eg. doing things like :
>
> TestPackage: { ensure latest in -> 
> http://mirror.centos.org/centos/updates/i386/ } ; to make sure that this new 
> package does indeed provide an higher EVR than whatever is in the repo at the 
> end of that url.
>
> And having this harness pass or fail depending on the test outcome.
>
> Perhaps this is not the right list, but most of the people who would end up 
> using this or having to live with me forcing them to use it are on this 
> list... so I thought this would be a good place to start.
>
> Comments ? Am I wasting my time ?
>


I'm working on a tool for doing some of these same things for fedora right 
now.

Here's the list of things we wanted to check in fedora for a tree of pkgs:

#~ Checks:
  #~ Package/repo checks:
   #~ metadata matches
   #~ comps correct
   #~ file conflicts
   #~ normal conflicts
   #~ obsoleted pkgs in tree
   #~ circular obsoletes
   #~ self obsoleting pkgs
   #~ unresolveable provides
  #~ Tree/Distro
   #~ verify all files in .treeinfo exist/match
   #~ check for sha1sums if it finds .iso files

#~ Notifications:
  #~ Package:
   #~ report all triggers
   #~ look for all postrans/pretrans things
   #~ self-provided filedeps
   #~ self-provided normal deps
   #~ soname provides conflicts
     #~ in general list out all multiple providers

I'm doing the work in the 'verifytree' tool in yum-utils.

will definitely work on centos 5.3 :)

-sv