[CentOS] Configuration Compliance auditing for many CentOS 5.x boxes

Wed Feb 1 19:54:08 UTC 2012
Tom H <tom at limepepper.co.uk>

Hi CentOS experts,*

Short Version*

I would like to produce a weekly report in HTML for each CentOS 5.x 
server we have indicating configuration compliance with some industry 
benchmark. I am looking for a tool or tools to implement this, I am 
happy to use 3rd party proprietary stuff if necessary.
Long(er) Version*

    Current Situation.. I have a client with many (200x) CentOS 5.x 
servers deployed in various web, mail, database and file server roles, 
and these boxes have been variously administrated to a lessor or greater 

All the boxes have EPEL repository included as part of their 
base-install, and all boxes have cron jobs for "yum -y update" running 
frequently, and are rebooted when kernels are available. (so they are 
not in a terrible state)

For network, local and external vulnerabilities - We use a 3rd party 
firm, who use WebInspect to monitor for external facing ports and 
vulnerable services and produce various regular reports to my boss. 
(hence am not looking at Nessus, OpenVAS or network based scanning tools 
right now, or indeed any vulnerability tools)

However we now have a New Big Boss in Town - who is an ex security 
compliance dude. The new rules are; that if its not being regularly 
tested, then its not in compliance, even if it is in compliance etc. (to 
be honest, I quite like that rule)

So now I am looking for a way to generate a report of server compliance 
with some compliance standard for all the boxes regularly.

We have a basic list of configuration settings, that is a weaker form of 
various compliance recommendations, so I am confident that most 
compliance benchmarks like CIS, EAL3 or the linux web STIG level would 
be sufficient.

We have chef installed on the CentOS instances, hence I can push out yum 
based packages, (and I can install from source tarballs, but it will 
make me cry, on these instances)

I Would like to have...  a tool that runs locally on each CentOS box and 
produces a reasonably comprehensive html report regarding configuration 

(and a massive bonus would be to send email alert for severe problems, 
but I can script that if required)

Ideally I could generate a weekly report that indicates compliance with 
1 or more of the recognised linux server benchmarks. I am happy to pay 
for a subscription for the checklist, but I suspect the kind per 
instance 100 USD licenses I see are going to blow my budget.

Current progress is...

I see that OPENSCAP and OVAL have tools in CentOS-base or EPEL, such as

     ovaldi - oval reference interpreter

Which can be used to create reports. However they seem a little unrefined.

For SCAP and OVAL content I have found the following.

1. NIST provide SCAP content for RHEL desktop, which is kinda close;
2. http://usgcb.nist.gov/usgcb/rhel_content.html
3. There is a tool called sectool in the fedora repos, but I can't get 
it to run on CentOS due to a missing python-slip module.

Any suggestions on functioning stacks for this problem would be helpful.