On 07/11/07 12:38, Johnny Hughes wrote: ..

I have pulled the upstream binary and compared it to the CentOS binary ... here is the result

Verifying cachefilesd-0.7-6.el5.i386.rpm ... cachefilesd-0.7-6.el5.i386.rpm OK-PERFECT ref:44133 +/-:0 %:0

That means it is linked to all the same things AND, the OK-PERFECT means that even the Size reported for the RPM is identical.

Interesting tool - could you tell me more about this? TIA Jan

On Nov 7, 2007, at 8:12 AM, Johnny Hughes wrote:

Sure ... it is here:

http://mirror.centos.org/centos/4/build/distro/tmverifyrpms

Would you like a better comparison tool? Nothing wrong with the existing, but better, and more detailed, could easily be done.

73 de Jeff

Jeff Johnson wrote:

...

Sure ... it is here:

http://mirror.centos.org/centos/4/build/distro/tmverifyrpms

Would you like a better comparison tool? Nothing wrong with the existing, but better, and more detailed, could easily be done.

absolutely :)

we do rpmcompares and various other bits of fluffery within the buildsystem and adding tools that help us put out a better result would always be welcome!

Sure ... it is here:

http://mirror.centos.org/centos/4/build/distro/tmverifyrpms

You can use it to compare RPMS of the same name in a CentOS and RHEL directory that reside in the same directory as the script.

Thanks. The idea sounds very useful - I've started on modifying your script (attached) to cope with other directory structures and take signatures etc into account (should still largely behave as before, unless you give it arguments..).

I am not sure whether a "common" version for CentOS/SL/SLC/.. makes sense (the Changelog seem to document several generalization+simplification/specialization cycles already..), but if Jeff now adds new checks to your version, I'd be happy to benefit from them as well.

Best regards jan

#!/bin/bash # tmverifyrpms.sh # http://mirror.centos.org/centos/4/build/distro/tmverifyrpms # Check a set of rpms against a reference. # Ideas from David Cox's perl script ported to bash # creates tmverifyrpms.log in repository where it is run in the form of: # rpmname-cmp OK|OK-PERFECT|FAIL ref:refsize +/-:sizediffabs %:sizediff% # Where # - refsize is the rpm -q --qf %{SIZE} of the ref rpm # - sizediff% is the % difference between cmp and ref # - sizediffabs is the difference in bytes # If there are library, or file name/perm differences, create # reports/rpm.out # # Version 4.1.5-CentOS # 2007-02-27 Johnny Hughes johnny@centos.org # - Modified so that it will match some of the standard file # extensions we use (.centos, .c4. .el4.centos) with their # reference counterparts w/o those extensions # 2005-07-13 Johnny Hughes johnny@centos.org # - Modified David's great script to run standalone # for using in CentOS compares. # - Removed many of the features (run in background, # srpms maps, etc.) which are not needed in this # implementation of the script # # You can get all of David's tmtools as a group # from the /tmtools directory of any taolinux # mirror. See http://www.taolinux.com for details # # Thanks to David for a great script :) # # Version 4.1.4 # 2004-12-23 David L. Parsley parsley@linuxjedi.org # - Sort rpm --requires output # 2004-12-21 parsley@linuxjedi.org # - Findrepo & cd there first (to run in subdirs) # - Put error checking before the fork-to-background # 2004-12-03 parsley@linuxjedi.org # - Discard rpm errors # - Fix sign error in SZDIFF # 2004-11-30 parsley@linuxjedi.org # - Strip (xxx) from end (ld-linux.so mainly) # - Add checking of rpm --requires # 2004-09-30 David L. Parsley parsley@linuxjedi.org # - use mktemp -u, or files will be mode 600 # 2004-05-16 David L. Parsley parsley@linuxjedi.org # - Add user & group to fingerprinting # - Strip right-hand-side of library linkages (after =>) # - dump stderr by default (can override locally) # 2004-05-01 David L. Parsley parsley@linuxjedi.orgj # - use rpm --dump for files, can run unpriv #set -x # # This has been modified to become a standalone compare tool # by Johnny Hughes for CentOS compares on 7/13/2005 #

# more modifications: # take some input arguments to cope with different layouts, allow for several directories to compare with # remove dependency on current directory, quote everything to cope with whitespace in path # use $TMP iff set # make "workdir" temporary. # extend patterns for modified RPM names to include SL

STARTDIR=`pwd`

#This is where the reference files are REFDIRS=$STARTDIR/RHEL

#This is the directory with files to be compared CMPDIR=$STARTDIR/CentOS #Reporting goes here RPTDIR=$STARTDIR/reports

# optional: want signed RPMs RPMNEEDSIGN= # optional: insist on this signing key RPMGPGKEY=

WORKDIR=`mktemp -t -d rpmverify-work.XXXXX`

# indicator that something bad was found. GLOBALFAIL=0

help() { cat <<EOFhelp Usage: tmverifyrpms [-h] [-s] [-g keyid] [-r refdir,refdir] [-c cmpdir] [-l logdir] compare RPMs in cmpdir (default:$CMPDIR) to those in refdirs (default:$REFDIRS), list differences into logdir (default:$RPTDIR). -h: help -s: want signed RPMs -g: like -s, but accept only this keyid EOFhelp exit 0 }

while getopts "hsvg:r:c:l:w:" flag do [ "$flag" == "h" ] && help [ "$flag" == "s" ] && RPMNEEDSIGN=1 [ "$flag" == "g" ] && RPMGPGKEY="${OPTARG##0x%}" [ "$flag" == "r" ] && REFDIRS="${OPTARG//,/ /}" [ "$flag" == "c" ] && CMPDIR="${OPTARG%%/%}" [ "$flag" == "l" ] && RPTDIR="${OPTARG%%/%}" [ "$flag" == "w" ] && WORKDIR="${OPTARG%%/%}" done

[ -n "$RPMGPGKEY" ] && RPMNEEDSIGN=1

if [ -d "$WORKDIR" ] then rm -rf "$WORKDIR" fi

mkdir -p "$RPTDIR" "$WORKDIR"

exec 6>&1 exec 1>"$RPTDIR/tmverifyrpms.log"

rpmarch(){ STRIPPED=${1%.rpm} echo ${STRIPPED##*.} }

genfilelist(){ rpm -qp --dump "$1" 2>/dev/null | LC_ALL=C sort | while read DUMPLINE do DUMPLINE=${DUMPLINE/ [01] [01] /TMLINESEP} FSYMLINK=${DUMPLINE#*TMLINESEP* } DUMPLINE=${DUMPLINE%TMLINESEP*} FILENAME="${DUMPLINE% * * * * * *}" REST=${DUMPLINE#$FILENAME } set $REST echo "$FILENAME mode:$4 owner:$5 group:$6 linkto:$FSYMLINK" done }

comprpms(){ CMPRPM="$1" REFRPM="$2" RPM=`basename "$CMPRPM"` PKGOK="OK" CMPRPMDIR=`mktemp -t -d cmprpm.XXXXXX` REFRPMDIR=`mktemp -t -d refrpm.XXXXXX` rm -f "$RPTDIR/$RPM.out" rpm2cpio "$CMPRPM" | ( cd "$CMPRPMDIR"; cpio -id --no-absolute-filenames --quiet ) rpm2cpio "$REFRPM" | ( cd "$REFRPMDIR"; cpio -id --no-absolute-filenames --quiet ) cd "$CMPRPMDIR" find . ! -type d | while read FILENAME do if [ -x "$FILENAME" -a -f "$FILENAME" ] then echo $FILENAME >> $WORKDIR/CMPDIR-libs ldd "$FILENAME" 2>&1 | LC_ALL=C sort >> "$WORKDIR/CMPDIR-libs" fi done cd "$REFRPMDIR" find . ! -type d | while read FILENAME do if [ -x "$FILENAME" -a -f "$FILENAME" ] then echo "$FILENAME" >> "$WORKDIR/REFDIR-libs" ldd "$FILENAME" 2>&1 | LC_ALL=C sort >> "$WORKDIR/REFDIR-libs" fi done

genfilelist "$CMPRPM" > "$WORKDIR/CMPDIR-list" genfilelist "$REFRPM" > "$WORKDIR/REFDIR-list" diff -u "$WORKDIR/CMPDIR-list" "$WORKDIR/REFDIR-list" > "$WORKDIR/rpmdiff" if [ $? -eq 1 ] then PKGOK="FAIL" echo "Differing file fingerprints:">> "$RPTDIR/$RPM.out" cat "$WORKDIR/rpmdiff" >> "$RPTDIR/$RPM.out" fi rpm -qp --requires "$CMPRPM" 2>/dev/null | LC_ALL=C sort >"$WORKDIR/CMPDIR-req" rpm -qp --requires "$REFRPM" 2>/dev/null | LC_ALL=C sort >"$WORKDIR/REFDIR-req" diff -u "$WORKDIR/CMPDIR-req" "$WORKDIR/REFDIR-req" > "$WORKDIR/reqdiff" if [ $? -eq 1 ] then PKGOK="FAIL" echo "Differing package requirements:">> "$RPTDIR/$RPM.out" cat "$WORKDIR/reqdiff" >> "$RPTDIR/$RPM.out" fi if [ -e "$WORKDIR/REFDIR-libs" ] then # Only want what it's looking for, not what it actually gets perl -pi -e 's/ =>.*//' "$WORKDIR/CMPDIR-libs" perl -pi -e 's/ =>.*//' "$WORKDIR/REFDIR-libs" perl -pi -e 's/ (.*//' "$WORKDIR/CMPDIR-libs" perl -pi -e 's/ (.*//' "$WORKDIR/REFDIR-libs" diff -u "$WORKDIR/CMPDIR-libs" "$WORKDIR/REFDIR-libs" > "$WORKDIR/libdiff" if [ $? -eq 1 ] then PKGOK="FAIL" echo "Differing libraries:">> "$RPTDIR/$RPM.out" cat "$WORKDIR/libdiff" >> "$RPTDIR/$RPM.out" echo "ldd for CMPDIR-rpm:">> "$RPTDIR/$RPM.out" cat "$WORKDIR/CMPDIR-libs" >> "$RPTDIR/$RPM.out" echo "ldd for REFDIR-rpm:">> "$RPTDIR/$RPM.out" cat "$WORKDIR/REFDIR-libs" >> "$RPTDIR/$RPM.out" fi fi if [ -e "$WORKDIR/CMPDIR-libs" ] then grep -q 'not found' "$WORKDIR/CMPDIR-libs" if [ $? -eq 0 ] then PKGOK="FAIL" echo "Missing libraries:" \ >> "$RPTDIR/$RPM.out" cat "$WORKDIR/CMPDIR-libs" >> "$RPTDIR/$RPM.out" fi fi

rm -f $WORKDIR/libdiff $WORKDIR/rpmdiff CMPDIR-libs REFDIR-libs

# MORE TESTS HERE CMPSIZE=`rpm -qp --qf %{SIZE} $CMPRPM 2>/dev/null` REFSIZE=`rpm -qp --qf %{SIZE} $REFRPM 2>/dev/null` SZDIFF=$(($CMPSIZE - $REFSIZE)) ABSDIFF=$((SZDIFF<0?-$SZDIFF:$SZDIFF)) if [ $REFSIZE -ne 0 ] then PCTDIFF=$(($ABSDIFF * 100 / $REFSIZE)) else PCTDIFF=-0- fi if [ $PKGOK = "OK" -a $SZDIFF -eq 0 ] then PKGOK="OK-PERFECT" fi rm -rf $CMPRPMDIR $REFRPMDIR

[ "$PKGOK" = "FAIL" ] && GLOBALFAIL=$(( GLOBALFAIL++ )) RESULTS="$RPM $PKGOK ref:$REFSIZE +/-:$SZDIFF %:$PCTDIFF" }

# Get list of rpms to compare

RPMLIST=`echo ${CMPDIR}/*.rpm`

for CMPRPM in $RPMLIST do CMPRPM="${CMPRPM##${CMPDIR}/}"

echo -n "Verifying $CMPRPM ..."

# tests done directly on the RPM, i.e. not comparing with something # readable? if [ ! -r "${CMPDIR}/$CMPRPM" ] then echo "" echo "$CMPRPM FAIL - unreadable" echo "-------------------------" GLOBALFAIL=$(($GLOBALFAIL++)) continue fi

# corrupt, signed etc - try to run rpmk only once... verify=`rpm -Kv "${CMPDIR}/$CMPRPM" 2>&1` if ! echo "$verify" | grep -q "Header SHA1 digest: OK" then echo "" echo "$CMPRPM FAIL - corrupted header or not an RPM" echo "corrupted header or not an RPM:" >> "$RPTDIR/$CMPRPM.out" echo "$verify" >> "$RPTDIR/$CMPRPM.out" echo "-------------------------" GLOBALFAIL=$((GLOBALFAIL++)) continue fi echo -n "."

if ! echo "$verify" | grep -q "MD5 digest: OK" then echo "" echo "$CMPRPM FAIL - corrupted payload" echo "corrupted payload:" >> "$RPTDIR/$CMPRPM.out" echo "$verify" >> "$RPTDIR/$CMPRPM.out" echo "-------------------------" GLOBALFAIL=$((GLOBALFAIL++)) continue fi echo -n "."

# do we insist on signed RPMs? if [ -n "$RPMNEEDSIGN" ] then if ! echo "$verify" | grep -q "Header V3 DSA signature" ||\ ! echo "$verify" | grep -q "V3 DSA signature" then echo "" echo "$CMPRPM FAIL - not signed" echo "not signed:" >> "$RPTDIR/$CMPRPM.out" echo "$verify" >> "$RPTDIR/$CMPRPM.out" echo "-------------------------" GLOBALFAIL=$((GLOBALFAIL++)) continue fi echo -n "."

if ! echo "$verify" | grep -q "V3 DSA signature: OK" ||\ ! echo "$verify" | grep -q "Header V3 DSA signature: OK" then echo "" echo "$CMPRPM FAIL - signed with unknown key or bad signature" echo "signed with unknown key or bad signature:" >> "$RPTDIR/$CMPRPM.out" echo "$verify" >> "$RPTDIR/$CMPRPM.out" echo "-------------------------" GLOBALFAIL=$((GLOBALFAIL++)) continue fi echo -n "."

# do we insist on a specific key? if [ -n "$RPMGPGKEY" ] then if ! echo "$verify" | grep -q "V3 DSA signature: OK, key ID $RPMGPGKEY" ||\ ! echo "$verify" | grep -q "Header V3 DSA signature: OK, key ID $RPMGPGKEY" then echo "" echo "$CMPRPM FAIL - not signed with distro key $RPMGPGKEY" echo "not signed with distro key $RPMGPGKEY:" >> "$RPTDIR/$CMPRPM.out" echo "$verify" >> "$RPTDIR/$CMPRPM.out" echo "-------------------------" GLOBALFAIL=$((GLOBALFAIL++)) continue fi echo -n "." fi fi # comparisons to some upstream RPM

for REFDIR in $REFDIRS do # remove trailing slashes REFDIR="${REFDIR%%/%}"

# prefer to look for exactly-named RPMs if [ -e "$REFDIR/$CMPRPM" ]; then comprpms "$REFDIR/$CMPRPM" "$CMPDIR/$CMPRPM" echo "" echo $RESULTS echo "-------------------------" continue 2 fi # try with the original-non-distro name no_changed="$CMPRPM" for pat in .el4.centos .el5.centos .centos4 .centos .c4 .SL4 .cern .slc4 .slc; do # should be able to use {,pattern,pattern,..} but doesn't work with bash-2 REFRPM="${no_changed//${pat}/}" done

if [ -e "$REFDIR/$no_changed" ]; then echo -n "(distro tag)... " comprpms "$REFDIR/CMPRPM" "$CMPDIR/$REFRPM" echo "" echo $RESULTS echo "-------------------------" continue 2 fi done

echo "no match found for $CMPRPM" echo "-------------------------" done

echo "Finished with, $GLOBALFAIL errors"

exec 1>&6

if [ "$GLOBALFAIL" -gt 0 ] echo "There were $GLOBALFAIL errors - see $RPTDIR/tmverifyrpms.log" >2 exit 1 fi