On Mon, Aug 30, 2010 at 09:13:07PM -0700, Gordon Messmer wrote:
On 08/30/2010 07:24 PM, fred smith wrote:
Aug 30 22:09:08 fcshome kernel: md: created md1
...
Aug 30 22:09:08 fcshome kernel: md: kicking non-fresh sda2 from array!
...
Aug 30 22:09:08 fcshome kernel: md: created md0
...
Aug 30 22:09:08 fcshome kernel: md: kicking non-fresh sda1 from array!
Yep, your arrays are broken. mdmonitor should have emailed you about this. Make sure that you receive and read mail to the root user.
/sbin/mdadm /dev/md1 --fail /dev/sda2 --remove /dev/sda2 /sbin/mdadm /dev/md1 --add /dev/sda2
/sbin/mdadm /dev/md0 --fail /dev/sda1 --remove /dev/sda1 /sbin/mdadm /dev/md0 --add /dev/sda1
Thanks to Gordon, Robert, and all the others who contributed to my learning experience!
The problem was that /dev/sda had dropped out of the raid array, and that sdb remained. while sdb remained, it is the one that was being updated by yum updates, but sda is the one that grub was booting. hence the out- of-sync files/kernels/etc.
following the instructions above has solved the problem, and the array is now rebuilding.
I found some references online in several places (newegg commentson the specific WD drive I have, as well as other places) to a drive "feature" called LTER that allows setting a timeout for slow reads (?? maybe read errors??? I'd have to go back and re-read 'cause my memory is gone), and the default setting lets it delay for very long periods, causing the raid controller to think the drive has died and to drop it from the array. Apparently Linux software raid is subject to the same issue.
These online sources go on to mention that older versions of the specific drive model can have this setting changed (with the WDTLER.exe utility), but that WD, in its infinite wisdom, has removed that capability from "newer" drives. I tried the utility on my system and it reports "no drives found" so mine, even though they're over a year old, must be of the "newer" category. I may end up replacing them if they continue to do this to me. After all, the purpose and intent in building a desktop system with RAID-1 was redundancy, NOT HASSLE.
Thanks again to all who advised.
Fred