[CentOS] Root-filesystem remounts as read-only during 5.2 upgrade (system completely shoot)

Wed Jun 25 16:34:26 UTC 2008
Bernhard Gschaider <bgschaid_lists at ice-sf.at>

Judging from the frequency of my messages here one could think that
I'm too stupid to upgrade a workstation to 5.2 (but the servers I've
tried work without problem)

OK. The problem: I've tried to upgrade a 5.1-x86_64-workstation to 5.2.
During the upgrade immidiatly after (according to the
/var/log/messages) upgrading the two (64 & 32-bit) libgcc-packages an
EXT3-error occurs and the root-partition gets remounted in
read-only-mode. Consequently ALL following package upgrades throw
errors saying this or that can't be done because he can't write to
/etc, /usr but in the end yum says everything is OK.

(Side info: /var is on a different partition, so it is still
writeable)

At reboot the machine wants a manual fsck which throws a lot of
errors. 
Then the machine reboots with with a lot of error messages (basically
because it can't find /lib64/libgcc_s.so.1 (nothing a symlink can't
fix). After that I try to fix things by manually upgrading the
libgcc-packages (otherwise yum won'T run). "rpm -q" for selected
packages shows that the majority of the packages is still in 5.1-state.
Trying to "yum upgrade" again fail because the machine can't find any
servers (but network functionality seems OK)

So I try to reboot. And now the really strange thing happens: A simple
"rpm -q rpm" says that no rpm is installed. Rebuilding the
rpm-database doesn't help.

So I'm a bit stuck here with a machine that is in limbo.

My questions: what could have been the cause for this (the machine was
working OK before that). The only thing I feel a little guilty about
was deinstalling the nvidia-kernel-driver and not reboot, but this
can't f### up the file-system, can it?

Any suggestions what I can do to get the machine into a normal state
again (apart from reinstall from scratch)