[CentOS] Debugging I/O Errors that abort the journal

james

james at wintercastle.net
Thu Oct 18 15:49:37 UTC 2012


I got a I/O error on one of my disks that caused the journal to abort 
(error below) and be mounted in read only mode. I rebooted and the file 
system check ran automatically and resolved the error. I did some searching 
around and couldn't find a likely cause (the disk is a virtual disk, shared 
from a SAN for VMWare ESXi5.0).

An hour after bringing the system back up, we got another one. Same sort of 
error, different inode. Went to run level 1, and ran fsck manually (running 
now).

I don't see any obvious errors in the vmware-log file for the VM. Or on the 
SAN. All the other VMs are running fine and I don't see any issues there.

Is there a good process to follow when trying to debug an I/O error like 
this? There are so many variables in this case. Wasn't sure if there were 
any ext3 utilities that I don't know about but should be using

Error:
Oct 18 02:52:38 shiva clamd[3978]: SelfCheck: Database status OK.
Oct 18 03:00:03 shiva kernel: end_request: I/O error, dev sdb, sector 
51083840
Oct 18 03:00:03 shiva kernel: Aborting journal on device sdb10.
Oct 18 03:00:03 shiva kernel: ext3_abort called.
Oct 18 03:00:03 shiva kernel: EXT3-fs error (device sdb10): 
ext3_journal_start_sb: Detected aborted journal
Oct 18 03:00:03 shiva kernel: Remounting filesystem read-only
Oct 18 03:00:03 shiva kernel: EXT3-fs error (device sdb10) in 
ext3_reserve_inode_write: Journal has aborted
Oct 18 03:00:03 shiva kernel: EXT3-fs error (device sdb10) in 
ext3_reserve_inode_write: Journal has aborted
Oct 18 03:02:38 shiva clamd[3978]: SelfCheck: Database status OK.




More information about the CentOS mailing list