[CentOS-devel] QA effort

[Karanbir Singh]

On 04/04/2011 06:46 PM, Dennis Gerasimov wrote:
> Thank you, I think I know just the part I want to tackle: the cluster
> suite: cman, clvm and rgmanager (excluding gfs* for now).
> Where do we keep the tests? Are there examples of tests for other
> packages that I can look at?

http://www.karan.org/blog/index.php/2010/03/30/autmated-install-testing-for-centos 
has something to start with, its very basic, but its something. There 
have been a few posts about that here on the list as well. But I'll give 
you enough to get started with :

The tests themselves can be of two types :

a) installer based ( where one would write a kickstart, that sets up the 
machine does its tests or sets up the tests in %post ). Kickstarts are 
well, kickstarts. Look here for some examples if you need to : 
https://nazar.karan.org/cgit/bluecain/tree/

b) post-install scripts. These are the ones where one would do 
functional tests. Each of these tests is a single script which must - at 
this time - be self contained. And the only real concern for the testing 
system is what the exit code was ( 0 = pass, 1 = fail ). They can be 
written in any language you like. Here is a very simple example:

------------------------------------
#!/bin/sh
echo -n 'Test that all updates can be applied to this machine cleanly'
yum -d0 -y upgrade > /dev/null 2>&1
if [ $? -eq 0 ]; then
   echo ': PASS'
else
   echo ': Fail'
   exit 1
fi
------------------------------------

here is another which does something similar :
------------------------------------
#!/bin/sh
echo -n 'Test that all 32 rpms can be removed'
# only run this test on x86_64 machines!
is64=$(uname -m|grep x86_64)
if [ "$?" -ne '0' ]; then
   echo ' Skip'
   exit 1
fi
yum -d0 -y erase *.i?86 > /dev/null 2>&1
if [ $? -eq 0 ]; then
   echo ' PASS'
else
   echo ' Fail'
   exit 1
fi

# note, this does not imply the machine is still usable after
# the remove! need to test that independently

------------------------------------

Things that you *do* need to be concerned about :

- kickstarts should make no assumptions about storage ( since we run 
them on xen and kvm hosts ). If you need specific names and labels, use lvm.

- Getting the tests into the machine post install is not something you 
need to worry about, thats done by the test harness.

- ideally, tests will be in filenames that reflect the package or the 
nature of the test they are hoping to work against. eg: 
yum_CanBlindUpgrade.sh and yum_CanRemove32bit.sh

- if you are not writing a kickstart to cover the scope of tests, then 
you need to assume packages being tested are not installed. eg. php 
tests should start with a 'yum -d0 install php'

- test real functionality first, then work your way down if you want to. 
Eg:

------------------------------------
#!/bin/sh
echo -n 'Test to see if dns works'
echo 'nameserver 8.8.8.8'  > /etc/resolv.conf
# its important we dont hit a dns record with a wildcard like centos.org
ping -c 1 www.google.com > /dev/null 2>&1
if [ $? -eq 0 ]; then
   echo ' PASS'
else
   echo ' Fail'
   exit 1
fi

# implied results:
# - network works
# - default route is really honoured
# - atleast one network link on the machine is working
# - ipv4 is functional

------------------------------------

So in this case, its testing if DNS is functional ( something a user 
would care about ) and you get some free test wins ( network works, 
route etc ).

Questions ? ask away. Maybe a few people can get together and even start 
working on a set of wiki pages for this stuff!

The tests are all inside a git repo, but for the first few that you 
write - how about posting them here to the list and we can take it from 
there.

- KB