[CentOS] Superblock Problem

Thu Jul 1 21:25:34 UTC 2010
Guy Boisvert <boisvert.guy at videotron.ca>

  Hi all,

After rebooting my CentOS 5.5 server, i have the following message:

==================================
Red Hat nash version 5.1.19.6 starting
EXT3-fs: unable to read superblock
mount: error mounting /dev/root on /sysroot as ext3: invalid argument
setuproot: moving /root failed: No such file or directory
setuproot: error mounting /proc: No such file or directory
setuproot: error mounting /sys: No such file or directory
switchroot: mount failed: no such file or directory
Kernel panic - not syncing: Attempted to kill init!


I booted the server (rescue mode) with CentOS 5.4 x64 i have used to 
initially install it.  It recognizes all the RAID1 partitions i have.

cat /mnt/sysimage/proc/mdstat
============================
md3 : active raid1 sdb2[1] sda2[0]
       2048192 blocks [2/2] [UU]

md1 : active raid1 sdb3[1] sda3[0]
       486134848 blocks [2/2] [UU]

md0 : active raid1 sdb1[0] sda1[1]
       200704 blocks [2/2] [UU]

unused devices: <none>


cat /mnt/sysimage/boot/grub/menu.lst
===================================
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE:  You have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /boot/, eg.
#          root (hd0,0)
#          kernel /vmlinuz-version ro root=/dev/md1
#          initrd /initrd-version.img
#boot=/dev/md0
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title CentOS (2.6.18-164.10.1.el5xen)
         root (hd0,0)
         kernel /xen.gz-2.6.18-164.10.1.el5
         module /vmlinuz-2.6.18-164.10.1.el5xen ro root=/dev/md1 rhgb quiet
         module /initrd-2.6.18-164.10.1.el5xen.img
title CentOS (2.6.18-164.10.1.el5)
         root (hd0,0)
         kernel /vmlinuz-2.6.18-164.10.1.el5 ro root=/dev/md1 rhgb quiet
         initrd /initrd-2.6.18-164.10.1.el5.img
title CentOS (2.6.18-164.6.1.el5)
         root (hd0,0)
         kernel /vmlinuz-2.6.18-164.6.1.el5 ro root=/dev/md1 rhgb quiet
         initrd /initrd-2.6.18-164.6.1.el5.img
title CentOS (2.6.18-164.6.1.el5xen)
         root (hd0,0)
         kernel /xen.gz-2.6.18-164.6.1.el5
         module /vmlinuz-2.6.18-164.6.1.el5xen ro root=/dev/md1 rhgb quiet
         module /initrd-2.6.18-164.6.1.el5xen.img
title CentOS (2.6.18-164.el5xen)
         root (hd0,0)
         kernel /xen.gz-2.6.18-164.el5
         module /vmlinuz-2.6.18-164.el5xen ro root=/dev/md1 rhgb quiet
         module /initrd-2.6.18-164.el5xen.img
title CentOS-base (2.6.18-164.el5)
         root (hd0,0)
         kernel /vmlinuz-2.6.18-164.el5 ro root=/dev/md1 rhgb quiet
         initrd /initrd-2.6.18-164.el5.img



./getinfo.sh disk
==============
== BEGIN uname -rmi ==
2.6.18-164.el5 x86_64 x86_64
== END   uname -rmi ==

== BEGIN rpm -q centos-release ==
centos-release-5-5.el5.centos
== END   rpm -q centos-release ==

== BEGIN getenforce ==
Permissive
== END   getenforce ==

== BEGIN cat /etc/fstab ==
/dev/md1                /                       ext3    defaults,acl    1 1
/dev/md0                /boot                   ext3    defaults        1 2
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
/dev/md3                swap                    swap    defaults        0 0
== END   cat /etc/fstab ==

== BEGIN df -h ==
Filesystem            Size  Used Avail Use% Mounted on
/dev/md1              450G   72G  355G  17% /
/dev/md0              190M   45M  136M  25% /boot
== END   df -h ==

== BEGIN fdisk -l ==
Disk /dev/md0 doesn't contain a valid partition table
Disk /dev/md1 doesn't contain a valid partition table
Disk /dev/md3 doesn't contain a valid partition table

Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

    Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          25      200781   fd  Linux raid 
autodetect
/dev/sda2              26         280     2048287+  fd  Linux raid 
autodetect
/dev/sda3             281       60801   486134932+  fd  Linux raid 
autodetect

Disk /dev/sdb: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

    Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1          25      200781   fd  Linux raid 
autodetect
/dev/sdb2              26         280     2048287+  fd  Linux raid 
autodetect
/dev/sdb3             281       60801   486134932+  fd  Linux raid 
autodetect

Disk /dev/md0: 205 MB, 205520896 bytes
2 heads, 4 sectors/track, 50176 cylinders
Units = cylinders of 8 * 512 = 4096 bytes


Disk /dev/md1: 497.8 GB, 497802084352 bytes
2 heads, 4 sectors/track, 121533712 cylinders
Units = cylinders of 8 * 512 = 4096 bytes


Disk /dev/md3: 2097 MB, 2097348608 bytes
2 heads, 4 sectors/track, 512048 cylinders
Units = cylinders of 8 * 512 = 4096 bytes

== END   fdisk -l ==

== BEGIN blkid ==
/dev/md3: TYPE="swap"
/dev/md0: UUID="085a2fe2-a28b-4c82-aae8-d4711a73f824" TYPE="ext3"
/dev/md1: UUID="4e26e66d-b4f3-45e3-8307-e13358b13610" TYPE="ext3"
/dev/sdb3: UUID="4e26e66d-b4f3-45e3-8307-e13358b13610" TYPE="ext3"
/dev/sdb2: TYPE="swap"
/dev/sdb1: UUID="085a2fe2-a28b-4c82-aae8-d4711a73f824" TYPE="ext3"
/dev/sda3: UUID="4e26e66d-b4f3-45e3-8307-e13358b13610" TYPE="ext3"
/dev/sda2: TYPE="swap"
/dev/sda1: UUID="085a2fe2-a28b-4c82-aae8-d4711a73f824" TYPE="ext3"
== END   blkid ==


Tried to boot from different kernels, same result.  I searched Google 
and read something about superblock being "repeted" to many places on 
the filesystem but didn't tried anything because of bad explanations and 
fear of srewing up the partition.

I already backed up the important stuff to another server.  I'd like to 
know if it's possible to fix that problem quickly, if not i'll 
re-install it.  But i'd like to know the fix anyway so if it appends 
again, would know what to do!

Thanks!

Guy Boisvert
IngTegration inc.


-- 
Guy Boisvert, ing.
IngTegration inc.
http://www.ingtegration.com

AVIS DE CONFIDENTIALITÉ : ce message peut contenir des
renseignements confidentiels appartenant exclusivement à
IngTegration Inc. ou à ses filiales. Si vous n'êtes pas
le destinataire indiqué ou prévu dans ce  message (ou
responsable de livrer ce message à la personne indiquée ou
prévue) ou si vous pensez que ce message vous a été adressé
par erreur, vous ne pouvez pas utiliser ou reproduire ce
message, ni le livrer à quelqu'un d'autre. Dans ce cas, vous
devez le détruire et vous êtes prié d'avertir l'expéditeur
en répondant au courriel.

CONFIDENTIALITY NOTICE : Proprietary/Confidential Information
belonging to IngTegration Inc. and its affiliates may be
contained in this message. If you are not a recipient
indicated or intended in this message (or responsible for
delivery of this message to such person), or you think for
any reason that this message may have been addressed to you
in error, you may not use or copy or deliver this message to
anyone else. In such case, you should destroy this message
and are asked to notify the sender by reply email.