[CentOS] Software raid Oddity

Wed Dec 16 21:18:22 UTC 2020
Christopher Wensink <cwensink at five-star-plastics.com>

I had an issue similar to this years ago where I helped out a former 
employer on a Dell Poweredge System with a RAID 5 array (Windows). The 
system Refused to Boot, but there were lights on the front of the 
backplane were the drives slid in, indicating drive fault (amber) or 
drive ok (green).  One of the tests I did was I re-arranged the drives 
where they were inserted into the backplane. When I did that the same 
lights (slots) went amber after re-arranging the drives.

The problem wasn't the drives at all, the problem was the controller 
card going bad.  The IT guy that was there full time ended up shipping 
the drives off to a recovery service depo, and they recovered the data 
there, no problem.

When I worked for Sage we had SCSI RAID Controller cards that had 
similar functions, where the RAID card config was backed up in the 
drives, and the Drive configuration was stored in the RAID controller, 
so they backed up the config of each other.

In the event of a failure of the controller card, the same model card 
could be put back into the system, and the config data pulled off the 
recovery location in the drives, then the system was back up and going 
again.

Perhaps that is what's happening to your system.

I would take several full bare metal backups right now (and test restore 
the data onto a new system) there may be looming hardware failure around 
the corner.

Chris

On 12/16/2020 3:10 PM, Frank Cox wrote:
> On Wed, 16 Dec 2020 13:57:13 -0700
> Paul R. Ganci via CentOS wrote:
>
>> My gut suggests that the raid array was never degraded and that my
>> system (i.e. cat /proc/mdstat) was lying to me. Any Opinions?
> I wonder if it's a ram failure in either the main computer or the drive controller.  An intermittent ram failure (or cold solder joint or something equally hard to track down) could cause all manner of un-repeatable weirdness.
>

-- 
Christopher Wensink
IS Administrator
Five Star Plastics, Inc
1339 Continental Drive
Eau Claire, WI 54701
Office:  715-831-1682
Mobile:  715-563-3112
Fax:  715-831-6075
cwensink at five-star-plastics.com
www.five-star-plastics.com