[CentOS] 5.5->5.6 upgrade hiccough

Tue Apr 12 22:20:43 UTC 2011
Devin Reade <gdr at gno.org>

I spoke a bit too soon about the upgrade being flawless. It turns out 
that one subsystem started failing after the upgrade, although it is
not an out-of-the-box one.

I've included the description here in case someone runs into something
similarly weird (especially with scripts calling MySQL?) and because
although I have a work-around, I'm not sure (but can guess) as to the
actual cause of the change.

This subsystem is on a pacemaker/corosync cluster, which as one 
resource group is running a resource agent (RA) for a bacula-fd
instance.  (The bacula-fd RA takes care of backing up those
DRBD-mounted filesystems that are part of the resource group.)
The bacula-fd was failing because as a run-before command on 
the backup job, it was executing the mysqlblasy.pl script to 
back up some MySQL databases.  After the upgrade, mysqlblasy.pl
started failing when running from bacula-fd, even though it 
could be run directly as root.

However, *none* of these components were upgraded during the 5.5->5.6
OS upgrade:
   mysql server & client
   mysqlblasy.pl
   bacula daemons/clients
   bacula RA
   pacemaker/corosync

After suitable diagnosis, I was able to determine that with CentOS 5.5,
the environment provided to the RA had $HOME set, but that with 
CentOS 5.6 $HOME was *not* set.  In particular, since this RA runs
as root, in 5.5 HOME was set to "/" (as opposed to "/root").  Since
there was no change to pacemaker, I suspect that there was either a
change in the way daemons are started up, or a change to the fork
or set[re]uid model (or something similar).

Ensuring that HOME was (re)set properly before invoking mysqlblasy.pl
caused the backups to work again as expected.

Devin