[CentOS] USB key installed OS; file system goes read-only randomly...

Fri May 7 02:26:08 UTC 2010
Jason Pyeron <jpyeron at pdinc.us>

This happened a few times now, I (soft) reboot and do the fsck thing and all is
fine until it happens again.

Any ideas?

[root at test-dhcp ~]# yum remove bluez-libs bluez-utils
Loaded plugins: fastestmirror
Setting up Remove Process
Resolving Dependencies
<snip/>
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Erasing        : bluez-utils
1/3
  Erasing        : bluez-libs
2/3
Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/yum/rpmtrans.py", line 376, in callback
    self._unInstStop( bytes, total, h )
  File "/usr/lib/python2.4/site-packages/yum/rpmtrans.py", line 479, in
_unInstStop
    self.ts_done(h, action)
  File "/usr/lib/python2.4/site-packages/yum/rpmtrans.py", line 307, in ts_done
    misc.unlink_f(self.ts_done_fn)
  File "/usr/lib/python2.4/site-packages/yum/misc.py", line 777, in unlink_f
    os.unlink(filename)
OSError: [Errno 30] Read-only file system:
'/var/lib/yum/transaction-done.2010-05-06.15:18.05'
error: python callback <bound method RPMTransaction.callback of
<yum.rpmtrans.RPMTransaction instance at 0x9fddeac>> failed, aborting!
[root at test-dhcp ~]# > /tmp/test
-bash: /tmp/test: Read-only file system
[root at test-dhcp ~]# mount -n -o remount,rw /
mount: block device /dev/VolGroup00/LogVol00 is write-protected, mounting
read-only
[root at test-dhcp ~]# fdisk -l /dev/sda

Disk /dev/sda: 4105 MB, 4105175040 bytes
255 heads, 63 sectors/track, 499 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      104391   83  Linux
/dev/sda2              14         499     3903795   8e  Linux LVM
[root at test-dhcp ~]# > /boot/foo
[root at test-dhcp ~]# rm /boot/foo
rm: remove regular empty file `/boot/foo'? y
[root at test-dhcp ~]# > /tmp/foo
-bash: /tmp/foo: Read-only file system
[root at test-dhcp ~]#




Was tailing logs when it happened:

[root at test-dhcp ~]# tail -f /var/log/dmesg /var/log/secure /var/log/messages
/var/log/faillog
==> /var/log/dmesg <==
md: ... autorun DONE.
device-mapper: multipath: version 1.0.5 loaded
EXT3 FS on dm-0, internal journal
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sda1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
SELinux: initialized (dev sda1, type ext3), uses xattr
SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
Adding 917496k swap on /dev/VolGroup00/LogVol01.  Priority:-1 extents:1
across:917496k
SELinux: initialized (dev binfmt_misc, type binfmt_misc), uses genfs_contexts

==> /var/log/secure <==
May  6 14:51:08 test-dhcp sshd[2237]: Server listening on :: port 22.
May  6 14:51:08 test-dhcp sshd[2237]: error: Bind to port 22 on 0.0.0.0 failed:
Address already in use.
May  6 14:53:07 test-dhcp login: pam_unix(login:session): session opened for
user root by (uid=0)
May  6 14:53:09 test-dhcp login: ROOT LOGIN ON tty1
May  6 15:04:05 test-dhcp sshd[2258]: Server listening on :: port 22.
May  6 15:04:05 test-dhcp sshd[2258]: error: Bind to port 22 on 0.0.0.0 failed:
Address already in use.
May  6 15:04:28 test-dhcp sshd[2385]: Connection closed by 192.168.4.12
May  6 15:04:39 test-dhcp sshd[2387]: Accepted password for root from
192.168.4.12 port 4794 ssh2
May  6 15:04:40 test-dhcp sshd[2387]: pam_unix(sshd:session): session opened for
user root by (uid=0)

==> /var/log/messages <==
May  6 15:05:04 test-dhcp kernel: Bluetooth: Core ver 2.10
May  6 15:05:04 test-dhcp kernel: NET: Registered protocol family 31
May  6 15:05:04 test-dhcp kernel: Bluetooth: HCI device and connection manager
initialized
May  6 15:05:04 test-dhcp kernel: Bluetooth: HCI socket layer initialized
May  6 15:05:04 test-dhcp kernel: Bluetooth: L2CAP ver 2.8
May  6 15:05:04 test-dhcp kernel: Bluetooth: L2CAP socket layer initialized
May  6 15:05:04 test-dhcp kernel: Bluetooth: RFCOMM socket layer initialized
May  6 15:05:04 test-dhcp kernel: Bluetooth: RFCOMM TTY layer initialized
May  6 15:05:04 test-dhcp kernel: Bluetooth: RFCOMM ver 1.8
May  6 15:05:05 test-dhcp kernel: Bluetooth: HIDP (Human Interface Emulation)
ver 1.1

==> /var/log/faillog <==

==> /var/log/secure <==
May  6 15:05:51 test-dhcp sshd[2504]: Accepted password for root from
192.168.4.12 port 4795 ssh2
May  6 15:05:52 test-dhcp sshd[2504]: pam_unix(sshd:session): session opened for
user root by (uid=0)

==> /var/log/messages <==
May  6 15:06:46 test-dhcp yum: Erased: redhat-lsb
May  6 15:07:41 test-dhcp yum: Erased: cups
May  6 15:09:23 test-dhcp yum: Erased: avahi-compat-libdns_sd
May  6 15:10:29 test-dhcp avahi-daemon[2383]: Got SIGTERM, quitting.
May  6 15:10:29 test-dhcp avahi-daemon[2383]: Leaving mDNS multicast group on
interface eth0.IPv6 with address fc00::30:18ae:6b84.
May  6 15:10:29 test-dhcp avahi-daemon[2383]: Leaving mDNS multicast group on
interface eth0.IPv4 with address 192.168.22.23.
May  6 15:10:29 test-dhcp yum: Erased: avahi
May  6 15:18:06 test-dhcp sdpd[2073]: terminating...
May  6 15:18:06 test-dhcp hcid[2069]: Unregister path:/org/bluez
May  6 15:18:06 test-dhcp hcid[2069]: Exit
May  6 15:18:06 test-dhcp hidd[2210]: Exit
May  6 15:18:07 test-dhcp kernel: Bluetooth: BNEP (Ethernet Emulation) ver 1.2
May  6 15:18:07 test-dhcp kernel: Bluetooth: BNEP filters: protocol multicast
May  6 15:18:07 test-dhcp yum: Erased: bluez-utils


--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
-                                                               -
- Jason Pyeron                      PD Inc. http://www.pdinc.us -
- Principal Consultant              10 West 24th Street #100    -
- +1 (443) 269-1555 x333            Baltimore, Maryland 21218   -
-                                                               -
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
This message is copyright PD Inc, subject to license 20080407P00.