[CentOS] SOS: Production VM not starting!

Mon Dec 10 22:57:09 UTC 2012
Nikolaos Milas <nmilas at noa.gr>

I am using a VM with CentOS 5.8 x86_64 under KVM. I only have console 
access to the VM through a virtual console (web based).

Tonight, after a routine "yum update", I did a "shutdown -r now" due to 
kernel update and the VM won't start. See console screenshot vm1.png:

https://vmail.noa.gr/files/vm1.png

There is an error (which I haven't seen before):

type=1404 audit (...): selinux=0 auid=...  ses= ...

(see vm1.png above)

(The system cannot load even with the old kernel; the same error occurs.)

Note that SElinux is disabled on this system. I booted in rescue mode, 
and auto mount was unsuccessful (see 
https://vmail.noa.gr/files/vm2.png). The log of the rescue process shows 
some error (see https://vmail.noa.gr/files/vm4.png)

However, later I successfully mounted it using:

mount -t ext3 /dev/vda3 /mnt/sysimage

I then did a umount and:

fsck.ext3 /dev/vda3 which found it clean (see 
https://vmail.noa.gr/files/vm3.png).

What is wrong there?

Can you please guide me on how to make it work again?

This is a production ftp machine. Please help to revive.

Thanks,
Nick