[CentOS-devel] FreeIPA functional testing

[James Hogarth]

On 29 July 2013 17:25, James Hogarth <james.hogarth at gmail.com> wrote:

> Hi all,
>
> The initial commit I made (not carried out pull request just yet) does the
> following:
>  * Installs IPA packages
>  * Does automated basic configuration of server
>  * Verifies kerberos tickets being issued for host/admin user
>  * Tests adding a user (and checks that the initial password is expired
> plus the change)
>
> In the next few days I'm intending to add:
>  * Test adding a service and getting a keytab and certificate for that
> service
>  * Test adding a host
>  * Test adding DNS zone and records
>  * Test reloading bind (regression test for RHBA-2013-0739)
>  * Test sudo rules configured via IPA
>  * Test deleting all the stuff added above
>
> After that I'll issue the pull request...
>
> Next stage subsequent to the server tests working would be client side to
> tests registration against the IPA server itself...
>
> Is there anything else that anyone can think of that would be useful to
> have in the IPA test suite?
>
>

Well it was a 6 months that went by too fast and apologies for the delay...

The suite defined above has been pushed to gitorious and a merge request
made (with tigalch having kindly carried out a fully successful run against
my branch yesterday - thanks for that!).

Whilst finishing off the test suite the biggest hurdle I encountered was
that runtests.sh runs all tests sequentially and the system they are run on
is not cleaned between tests - so one can affect another.

Specifically the bits breaking a full run with just p_ipa-server passing in
a local VM:

1) The earlier httpd/mod_ssl/php testsuite that installed an SSL enabled
site when IPA requires mod_nss - these cannot be installed at the same time
and just remove mod_ssl leaves SSL config in place that then fails to parse
;)
2) The hostname of localhost.localdomain was problematic due to a valid
non-local host and an IP not on lo was needed
3) A later ntp test for centos servers in ntp.conf when the ipa install
puts the rhel ntp servers in the config (incidentally should we note a
packaging bug on this just as the centos ntp.conf that is installed does
not use rhel ntp servers or doesn't it matter now in light of the recent
activities?)

Looking at the final commit I ended up using a 0-preclean.sh script to get
the system into a known clean-ish (enough for the tests to pass) condition
and a 99-postclean.sh script to restore backed up config and make use of
yum history rollback to revert the VM to the pre-ipa install (in terms of
packages in place) so as to cause minimum disruption to the rest.

Without going for individual LXC containers, spawned VMs per test suite, or
similar complication might I suggest that each test suite gets a similar
pre/post to at least get the system to kind of a reverted state at the end
of the test and minimise ordering or unclean issues/dependencies that then
might crop up in future?

In principle the test suite should work for the el7 beta but that's
something I plan to test in the next few days (and make modifications for
el7 if it does not)...

After that i'm happy to put in some grunt work on pre/post cleanup scripts
if people think it's a good idea ... it's easier to do on el6 than el5 (due
to the existence of yum history) but I'm sure a similar accomdation could
be worked out - and later after that it should make parallelising the tests
easier to improve run time as the suite grows into el7 or is adapated for
use by SIGs.

 Cheers,

James
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.centos.org/pipermail/centos-devel/attachments/20140201/383f06d2/attachment.html>