[CentOS] RAID5 suddenly broken

Mathieu Baudier

mbaudier at argeo.org
Wed Aug 17 22:40:24 UTC 2011


Hello,

I have a RAID5 array on my CentOS 5.6 x86_64 workstation which
"suddenly" failed to work (actually after the system could not resume
from a suspend).

I had recently issues after moving the workstation to another office,
where one of the disks got accidently unplugged. But the RAID was
working and it had reconstructed (as far as I can tell) the data.
After I replugged the disk, it was working normally over the last two
days (enough to get back to a proper state I guess/hoped)

This RAID is used as an LVM volume group for all my important data,
among them the root of the operating system(s).
It based on four partitions on four separate disks (the third
partition of each disk, 3 active, one spare)

When booting, I get an error message similar to:

raid5 failed: No md superblock detected on /dev/md0.

and the LVM volume group does not come up.

I then booted using the CentOS 5.6 LiveCD and tried to run a few mdadm
command (see just below).
It seems that there are some data still lying around, but I'm not very
experienced with RAID and I thought that I would ask for advice before
trying commands which may impact the data such as assemble, etc.

Thanks in advance for any help!

Mathieu

[root at livecd ~]# mdadm --misc -E /dev/md0
mdadm: No md superblock detected on /dev/md0.
[root at livecd ~]# mdadm --misc -Q /dev/md0
/dev/md0: is an md device which is not active
/dev/md0: No md super block found, not an md component.
[root at livecd ~]# mdadm --misc -D /dev/md0
mdadm: md device /dev/md0 does not appear to be active.
[root at livecd ~]# mdadm -E /dev/sd*3
/dev/sda3:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 7533411a:f066a145:1e89d48e:1a8374a3
  Creation Time : Tue Dec  1 12:01:05 2009
     Raid Level : raid5
  Used Dev Size : 204796416 (195.31 GiB 209.71 GB)
     Array Size : 409592832 (390.62 GiB 419.42 GB)
   Raid Devices : 3
  Total Devices : 2
Preferred Minor : 0

    Update Time : Wed Aug 17 14:47:36 2011
          State : active
 Active Devices : 2
Working Devices : 2
 Failed Devices : 1
  Spare Devices : 0
       Checksum : ed6d5dcd - correct
         Events : 38857

         Layout : left-symmetric
     Chunk Size : 256K

      Number   Major   Minor   RaidDevice State
this     0       8        3        0      active sync   /dev/sda3

   0     0       8        3        0      active sync   /dev/sda3
   1     1       0        0        1      faulty removed
   2     2       8       51        2      active sync   /dev/sdd3
/dev/sdb3:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 7533411a:f066a145:1e89d48e:1a8374a3
  Creation Time : Tue Dec  1 12:01:05 2009
     Raid Level : raid5
  Used Dev Size : 204796416 (195.31 GiB 209.71 GB)
     Array Size : 409592832 (390.62 GiB 419.42 GB)
   Raid Devices : 3
  Total Devices : 4
Preferred Minor : 0

    Update Time : Wed Aug 10 11:52:15 2011
          State : clean
 Active Devices : 3
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 1
       Checksum : ed63a948 - correct
         Events : 9022

         Layout : left-symmetric
     Chunk Size : 256K

      Number   Major   Minor   RaidDevice State
this     1       8       19        1      active sync   /dev/sdb3

   0     0       8        3        0      active sync   /dev/sda3
   1     1       8       19        1      active sync   /dev/sdb3
   2     2       8       51        2      active sync   /dev/sdd3
   3     3       8       35        3      spare   /dev/sdc3
/dev/sdc3:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 7533411a:f066a145:1e89d48e:1a8374a3
  Creation Time : Tue Dec  1 12:01:05 2009
     Raid Level : raid5
  Used Dev Size : 204796416 (195.31 GiB 209.71 GB)
     Array Size : 409592832 (390.62 GiB 419.42 GB)
   Raid Devices : 3
  Total Devices : 3
Preferred Minor : 0

    Update Time : Tue Aug 16 17:48:59 2011
          State : clean
 Active Devices : 3
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 0
       Checksum : ed6bf2b0 - correct
         Events : 10670

         Layout : left-symmetric
     Chunk Size : 256K

      Number   Major   Minor   RaidDevice State
this     1       8       35        1      active sync   /dev/sdc3

   0     0       8        3        0      active sync   /dev/sda3
   1     1       8       35        1      active sync   /dev/sdc3
   2     2       8       51        2      active sync   /dev/sdd3
/dev/sdd3:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 7533411a:f066a145:1e89d48e:1a8374a3
  Creation Time : Tue Dec  1 12:01:05 2009
     Raid Level : raid5
  Used Dev Size : 204796416 (195.31 GiB 209.71 GB)
     Array Size : 409592832 (390.62 GiB 419.42 GB)
   Raid Devices : 3
  Total Devices : 2
Preferred Minor : 0

    Update Time : Wed Aug 17 14:47:36 2011
          State : active
 Active Devices : 2
Working Devices : 2
 Failed Devices : 1
  Spare Devices : 0
       Checksum : ed6d5e01 - correct
         Events : 38857

         Layout : left-symmetric
     Chunk Size : 256K

      Number   Major   Minor   RaidDevice State
this     2       8       51        2      active sync   /dev/sdd3

   0     0       8        3        0      active sync   /dev/sda3
   1     1       0        0        1      faulty removed
   2     2       8       51        2      active sync   /dev/sdd3
[root at livecd ~]#



More information about the CentOS mailing list