[CentOS] fsck problems

Tue Mar 23 14:12:14 UTC 2010
Todd Denniston <Todd.Denniston at tsb.cranrdte.navy.mil>

m.roth at 5-cent.us wrote, On 03/23/2010 09:49 AM:
>> sync wrote:
>>> Thanks for all replies .
>>>
>>> Today, I did the following things,and also met the other errror message:
> <snip>
>>> Then,  reboot the server and then boot it from the hard disk.
>>>
>>> This time, the screen came up with these:
>>>
>>> Checking root filesystem:
>>> ----------------------------------------------
>>> EXT3-fs error (device dm-0) :
>>> ext3_get_inode_loc: unable to read  inode block -inode=7473432,
>>> block=14942282
> <snip>
>>> It is obvious that the fsck command maybe not to use .
>>>
>>> What could i do now ?
>>>
>>>  Maybe it is the only method to reinstall the CentOS operating system.
>>> isn't it?
>> That looks like you either have a physically bad hard disk, or
>> directory/inode corruption in the entry needed to load fsck.  Try running
> <snip>
> Looks to me as though you definitely have a hard drive going bad. The
> thing that says that to me is "unable to read", not "error reading". I'd
> reboot from a DVD with linux rescue, do *not* mount the filesystems, and
> do an fsck -c (and any other options) on all the partitions that are
> formatted. This will check for bad blocks. If there's only a few, say, 1,
> or 10, note how many of them there are, and let it take its default to
> mark, and then fix everything else. If there's a *bunch* of them (100+),
> you need a new hard disk, now.
> 
>         mark "done this too much recently"

Side question: is there encryption in use on the volumes?

And assuming you fsck ... considering  "Checking root filesystem" is where the error shows up, I
would be

a) reading `man rpm`
and
b) running some variation of
for i in `all the packages installed`; \
do \
  echo "looking at $i"; \
  rpm -V $i;\
done > howbadismysystem.txt 2>&1

or

rpm -Va > confusinghowbadismysystem.txt 2>&1


and look for things that are not config files that have changed.
of course on CentOS 5.x prelinking makes a mess of things.

And then there is the implicit assumption that the rpm db was not one of the things that got
hammered. :)

-- 
Todd Denniston
Crane Division, Naval Surface Warfare Center (NSWC Crane)
Harnessing the Power of Technology for the Warfighter