[CentOS] Race condition with mdadm at bootup?

Hello folks,

I am experiencing a weird problem at bootup with large RAID-6 arrays. 
After Googling around (a lot) I find that others are having the same 
issues with CentOS/RHEL/Ubuntu/whatever.  In my case it's Scientific 
Linux-6 which should behave the same way as CentOS-6.  I had the same 
problem with the RHEL-6 evaluation version.  I'm posting this question 
to the SL mailing list as well.

For some reason, each time I boot the server a random number of RAID 
arrays will come up with the hot-spare missing.  This occurs with 
hot-spare components only, never with the active components.  Once in a 
while I'm lucky enough to have all components come up correctly when the 
system boots.  Which hot spares fail to be configured is completely random.

I have 12 2TB drives, each divided into 4 primary partitions, and 
configured as 8 partitionable MD arrays.  All drives are partitioned 
exactly the same way.  Each R6 array consists of 5 components 
(partitions) plus a hot-spare.  The small RAID-1 host OS array never has 
a problem with its hot spare.

The predominant theory via Google is that there's a race condition at 
boot time between full enumeration of all disk partitions and mdadm 
assembling the arrays.

Does anyone know of a way to have mdadm delay its assembly until all 
partitions are enumerated?  Even if it's simply to insert a 
several-second wait time, that would probably work.  My knowledge of the 
internal workings of the boot process isn't good enough to know where to 
look.

I tried to issue 'mdadm -A -s /dev/md/md_dXX' after booting, but all it 
does is complain about "No suitable drives found for /dev....."

Here is the mdadm.conf file:
-------------------------------------

MAILADDR root
PROGRAM /root/bin/record_md_events.sh

DEVICE partitions
##DEVICE /dev/sd*    <<---- this didn't help.
AUTO +imsm +1.x -all

## Host OS root arrays:
ARRAY /dev/md0
    metadata=1.0 num-devices=2 spares=1
    UUID=75941adb:33e8fa6a:095a70fd:6fe72c69
ARRAY /dev/md1
    metadata=1.1 num-devices=2 spares=1
    UUID=7a96d82d:bd6480a2:7433f1c2:947b84e9
ARRAY /dev/md2
    metadata=1.1 num-devices=2 spares=1
    UUID=ffc6070d:e57a675e:a1624e53:b88479d0

## Partitionable arrays on LSI controller:
ARRAY /dev/md/md_d10
    metadata=1.2 num-devices=5 spares=1
    UUID=135f0072:90551266:5d9a126a:011e3471
ARRAY /dev/md/md_d11
    metadata=1.2 num-devices=5 spares=1
    UUID=59e05755:5b3ec51e:e3002cfd:f0720c38
ARRAY /dev/md/md_d12
    metadata=1.2 num-devices=5 spares=1
    UUID=7916eb13:cd5063ba:a1404cd7:3b65a438
ARRAY /dev/md/md_d13
    metadata=1.2 num-devices=5 spares=1
    UUID=9a767e04:e4e56a9d:c369d25c:9d333760

## Partitionable arrays on Tempo controllers:
ARRAY /dev/md/md_d20
    metadata=1.2 num-devices=5 spares=1
    UUID=1d5a3c32:eb9374ac:eff41754:f8a176c1
ARRAY /dev/md/md_d21
    metadata=1.2 num-devices=5 spares=1
    UUID=38ffe8c9:f3922db9:60bb1522:80fea016
ARRAY /dev/md/md_d22
    metadata=1.2 num-devices=5 spares=1
    UUID=ebb4ea67:b31b2105:498d81af:9b4f45d3
ARRAY /dev/md/md_d23
    metadata=1.2 num-devices=5 spares=1
    UUID=da07407f:deeb8906:7a70ae82:6b1d8c4a

-------------------------------------

Your suggestions are most welcome ... thanks.

Chuck