Hello,
yesterday night I had a problem with my server located at a hoster (strato.de). I couldn't ssh to it and over the remote serial console I saw "out of memory" errors (sorry, don't have the text).
Then I had reinstall CentOS 5.5/64 bit + all my setup (2h work), because I have a contract with a social network and they will shut down my little card game if it is not reponding.
Now the server seems to work ok, but I'm worried about those /var/log/message:
kernel: INFO: task md1_resync:9770 blocked for more than 120 seconds. kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. kernel: md1_resync D ffff81011edba820 0 9770 55 9768 (L-TLB) kernel: ffff810083259d70 0000000000000046 0000000000000000 ffff81011ea66e0c kernel: ffff81011ea66c0c 000000000000000a ffff81011ccc27a0 ffff81011edba820 kernel: 000011d56f2a0b28 000000000000100f ffff81011ccc2988 000000028008b4d7 kernel: Call Trace: kernel: [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 kernel: [<ffffffff8021af66>] md_do_sync+0x1d8/0x833 kernel: [<ffffffff8008ca47>] enqueue_task+0x41/0x56 kernel: [<ffffffff8008cab2>] __activate_task+0x56/0x6d kernel: [<ffffffff8008c897>] dequeue_task+0x18/0x37 kernel: [<ffffffff80062ff8>] thread_return+0x62/0xfe kernel: [<ffffffff800a0b5f>] autoremove_wake_function+0x0/0x2e kernel: [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 kernel: [<ffffffff8021b93a>] md_thread+0xf8/0x10e kernel: [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 kernel: [<ffffffff8021b842>] md_thread+0x0/0x10e kernel: [<ffffffff8003296e>] kthread+0xfe/0x132 kernel: [<ffffffff8005dfb1>] child_rip+0xa/0x11 kernel: [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 kernel: [<ffffffff80032870>] kthread+0x0/0x132 kernel: [<ffffffff8005dfa7>] child_rip+0x0/0x11 kernel:
The /var/log/mcelog is empty.
# df -h Filesystem Size Used Avail Use% Mounted on /dev/md1 20G 1.4G 18G 8% / /dev/md3 176G 754M 166G 1% /var /dev/md0 993M 30M 913M 4% /boot /dev/md2 263G 352M 250G 1% /home tmpfs 2.0G 0 2.0G 0% /dev/shm
Does anybody please have an advice? :-(
(Besides "contact or change" your hoster, because it doesn't work).
I have 2 disks in "software raid" at that machine, but not much experience with RAID. My dmesg is below.
I could reboot the machine in a "rescue mode", but I'm not sure which commands to try there.
Thank you Alex
Linux version 2.6.18-194.32.1.el5 (mockbuild@builder10.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-48)) #1 SMP Wed Jan 5 17:52:25 EST 2011 Command line: root=/dev/md1 console=tty0 console=ttyS0,57600 BIOS-provided physical RAM map: BIOS-e820: 0000000000010000 - 000000000009f000 (usable) BIOS-e820: 000000000009f000 - 00000000000a0000 (reserved) BIOS-e820: 00000000000e4000 - 0000000000100000 (reserved) BIOS-e820: 0000000000100000 - 00000000ddfb0000 (usable) BIOS-e820: 00000000ddfb0000 - 00000000ddfbe000 (ACPI data) BIOS-e820: 00000000ddfbe000 - 00000000ddfe0000 (ACPI NVS) BIOS-e820: 00000000ddfe0000 - 00000000ddfee000 (reserved) BIOS-e820: 00000000ddff0000 - 00000000de000000 (reserved) BIOS-e820: 00000000ff700000 - 0000000100000000 (reserved) BIOS-e820: 0000000100000000 - 0000000120000000 (usable) DMI present. ACPI: RSDP (v000 ACPIAM ) @ 0x00000000000faf80 ACPI: RSDT (v001 032510 RSDT1503 0x20100325 MSFT 0x00000097) @ 0x00000000ddfb0000 ACPI: FADT (v002 032510 FACP1503 0x20100325 MSFT 0x00000097) @ 0x00000000ddfb0200 ACPI: MADT (v001 032510 APIC1503 0x20100325 MSFT 0x00000097) @ 0x00000000ddfb0390 ACPI: MCFG (v001 032510 OEMMCFG 0x20100325 MSFT 0x00000097) @ 0x00000000ddfb0400 ACPI: OEMB (v001 032510 OEMB1503 0x20100325 MSFT 0x00000097) @ 0x00000000ddfbe040 ACPI: HPET (v001 032510 OEMHPET 0x20100325 MSFT 0x00000097) @ 0x00000000ddfb48c0 ACPI: SSDT (v001 A M I POWERNOW 0x00000001 AMD 0x00000001) @ 0x00000000ddfb4900 ACPI: DSDT (v001 A96B3 A96B3210 0x00000210 INTL 0x20051117) @ 0x0000000000000000 No NUMA configuration found Faking a node at 0000000000000000-0000000120000000 Bootmem setup node 0 0000000000000000-0000000120000000 Memory for crash kernel (0x0 to 0x0) notwithin permissible range disabling kdump On node 0 totalpages: 1022763 DMA zone: 2627 pages, LIFO batch:0 DMA32 zone: 890856 pages, LIFO batch:31 Normal zone: 129280 pages, LIFO batch:31 ACPI: PM-Timer IO Port: 0x808 ACPI: Local APIC address 0xfee00000 ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled) Processor #0 0:4 APIC version 16 ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled) Processor #1 0:4 APIC version 16 ACPI: LAPIC (acpi_id[0x03] lapic_id[0x02] enabled) Processor #2 0:4 APIC version 16 ACPI: LAPIC (acpi_id[0x04] lapic_id[0x03] enabled) Processor #3 0:4 APIC version 16 ACPI: IOAPIC (id[0x04] address[0xfec00000] gsi_base[0]) IOAPIC[0]: apic_id 4, version 33, address 0xfec00000, GSI 0-23 ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level) ACPI: IRQ0 used by override. ACPI: IRQ2 used by override. ACPI: IRQ9 used by override. Setting APIC routing to physical flat ACPI: HPET id: 0x8300 base: 0xfed00000 Using ACPI (MADT) for SMP configuration information Nosave address range: 000000000009f000 - 00000000000a0000 Nosave address range: 00000000000a0000 - 00000000000e4000 Nosave address range: 00000000000e4000 - 0000000000100000 Nosave address range: 00000000ddfb0000 - 00000000ddfbe000 Nosave address range: 00000000ddfbe000 - 00000000ddfe0000 Nosave address range: 00000000ddfe0000 - 00000000ddfee000 Nosave address range: 00000000ddfee000 - 00000000ddff0000 Nosave address range: 00000000ddff0000 - 00000000de000000 Nosave address range: 00000000de000000 - 00000000ff700000 Nosave address range: 00000000ff700000 - 0000000100000000 Allocating PCI resources starting at e0000000 (gap: de000000:21700000) SMP: Allowing 4 CPUs, 0 hotplug CPUs Built 1 zonelists. Total pages: 1022763 Kernel command line: root=/dev/md1 console=tty0 console=ttyS0,57600 Initializing CPU#0 PID hash table entries: 4096 (order: 12, 32768 bytes) Console: colour VGA+ 80x25 Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes) Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes) Checking aperture... CPU 0: aperture @ 4000000 size 32 MB Aperture too small (32 MB) No AGP bridge found Your BIOS doesn't leave a aperture memory hole Please enable the IOMMU option in the BIOS setup This costs you 64 MB of RAM Mapping aperture over 65536 KB of RAM @ 4000000 Nosave address range: 0000000004000000 - 0000000008000000 ACPI: DMAR not present Memory: 4016200k/4718592k available (2575k kernel code, 144564k reserved, 1304k data, 212k init) Calibrating delay loop (skipped), value calculated using timer frequency.. 5000.24 BogoMIPS (lpj=2500120) Security Framework v1.0.0 initialized SELinux: Initializing. SELinux: Starting in permissive mode selinux_register_security: Registering secondary module capability Capability LSM initialized as secondary Mount-cache hash table entries: 256 CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line) CPU: L2 Cache: 512K (64 bytes/line) CPU 0/0 -> Node 0 CPU: Physical Processor ID: 0 CPU: Processor Core ID: 0 SMP alternatives: switching to UP code ACPI: Core revision 20060707 Using local APIC timer interrupts. Detected 12.500 MHz APIC timer. SMP alternatives: switching to SMP code Booting processor 1/4 APIC 0x1 Initializing CPU#1 Calibrating delay using timer specific routine.. 5000.12 BogoMIPS (lpj=2500063) CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line) CPU: L2 Cache: 512K (64 bytes/line) CPU 1/1 -> Node 0 CPU: Physical Processor ID: 0 CPU: Processor Core ID: 1 Quad-Core AMD Opteron(tm) Processor 1381 stepping 02 SMP alternatives: switching to SMP code Booting processor 2/4 APIC 0x2 Initializing CPU#2 Calibrating delay using timer specific routine.. 5000.18 BogoMIPS (lpj=2500094) CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line) CPU: L2 Cache: 512K (64 bytes/line) CPU 2/2 -> Node 0 CPU: Physical Processor ID: 0 CPU: Processor Core ID: 2 Quad-Core AMD Opteron(tm) Processor 1381 stepping 02 SMP alternatives: switching to SMP code Booting processor 3/4 APIC 0x3 Initializing CPU#3 Calibrating delay using timer specific routine.. 4999.17 BogoMIPS (lpj=2499587) CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line) CPU: L2 Cache: 512K (64 bytes/line) CPU 3/3 -> Node 0 CPU: Physical Processor ID: 0 CPU: Processor Core ID: 3 Quad-Core AMD Opteron(tm) Processor 1381 stepping 02 Brought up 4 CPUs testing NMI watchdog ... OK. time.c: Using 14.318180 MHz WALL HPET GTOD HPET/TSC timer. time.c: Detected 2500.123 MHz processor. sizeof(vma)=176 bytes sizeof(page)=56 bytes sizeof(inode)=560 bytes sizeof(dentry)=216 bytes sizeof(ext3inode)=760 bytes sizeof(buffer_head)=96 bytes sizeof(skbuff)=248 bytes migration_cost=230 checking if image is initramfs... it is Freeing initrd memory: 2614k freed NET: Registered protocol family 16 ACPI: bus type pci registered PCI: BIOS Bug: MCFG area at e0000000 is not E820-reserved PCI: Not using MMCONFIG. PCI: Using configuration type 1 PCI: Using configuration type 1 for extended access ACPI: Interpreter enabled ACPI: Using IOAPIC for interrupt routing ACPI: No dock devices found. ACPI: PCI Root Bridge [PCI0] (0000:00) PCI: set SATA to AHCI mode PCI: Ignoring BAR0-3 of IDE controller 0000:00:14.1 PCI: Transparent bridge - 0000:00:14.4 ACPI: PCI Interrupt Routing Table [_SB_.PCI0._PRT] ACPI: PCI Interrupt Routing Table [_SB_.PCI0.P0P1._PRT] ACPI: PCI Interrupt Routing Table [_SB_.PCI0.PCE4._PRT] ACPI: PCI Interrupt Routing Table [_SB_.PCI0.PCE5._PRT] ACPI: PCI Interrupt Routing Table [_SB_.PCI0.P0PC._PRT] ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 *5 7 10 11 12 14 15) ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 7 10 11 12 14 *15) ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 7 *10 11 12 14 15) ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 7 *10 11 12 14 15) ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 7 10 11 12 14 15) *0, disabled. ACPI: PCI Interrupt Link [LNKF] (IRQs 9) *0, disabled. ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 7 10 *11 12 14 15) ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 7 10 11 12 14 15) *0, disabled. Linux Plug and Play Support v0.97 (c) Adam Belay pnp: PnP ACPI init pnp: PnP ACPI: found 14 devices usbcore: registered new driver usbfs usbcore: registered new driver hub PCI: Using ACPI for IRQ routing PCI: If a device doesn't work, try "pci=routeirq". If it helps, post a report PCI: Cannot allocate resource region 1 of device 0000:00:14.0 NetLabel: Initializing NetLabel: domain hash size = 128 NetLabel: protocols = UNLABELED CIPSOv4 NetLabel: unlabeled traffic allowed by default hpet0: at MMIO 0xfed00000 (virtual 0xffffffffff5fe000), IRQs 2, 8, 0, 0 hpet0: 4 32-bit timers, 14318180 Hz ACPI: DMAR not present PCI-DMA: Disabling AGP. PCI-DMA: aperture base @ 4000000 size 65536 KB PCI-DMA: using GART IOMMU. PCI-DMA: Reserving 64MB of IOMMU area in the AGP aperture pnp: 00:08: iomem range 0xfec00000-0xfec00fff has been reserved pnp: 00:08: iomem range 0xfee00000-0xfee00fff has been reserved pnp: 00:09: iomem range 0xffb80000-0xffbfffff could not be reserved pnp: 00:0b: ioport range 0xa00-0xa0f has been reserved pnp: 00:0b: ioport range 0xa10-0xa1f has been reserved pnp: 00:0c: iomem range 0xe0000000-0xefffffff has been reserved pnp: 00:0d: iomem range 0x0-0x9ffff could not be reserved pnp: 00:0d: iomem range 0xc0000-0xcffff could not be reserved pnp: 00:0d: iomem range 0xe0000-0xfffff could not be reserved pnp: 00:0d: iomem range 0x100000-0xddffffff could not be reserved PCI: Error while updating region 0000:00:14.0/1 (f0000004 != 8000a014) PCI: Bridge: 0000:00:01.0 IO window: c000-cfff MEM window: fe800000-fe9fffff PREFETCH window 0x00000000fc000000-0x00000000fdffffff PCI: Bridge: 0000:00:04.0 IO window: d000-dfff MEM window: fea00000-feafffff PREFETCH window: disabled. PCI: Bridge: 0000:00:05.0 IO window: e000-efff MEM window: feb00000-febfffff PREFETCH window: disabled. PCI: Bridge: 0000:00:14.4 IO window: disabled. MEM window: disabled. PREFETCH window: disabled. PCI: Setting latency timer of device 0000:00:04.0 to 64 PCI: Setting latency timer of device 0000:00:05.0 to 64 NET: Registered protocol family 2 IP route cache hash table entries: 131072 (order: 8, 1048576 bytes) TCP established hash table entries: 262144 (order: 10, 4194304 bytes) TCP bind hash table entries: 65536 (order: 8, 1048576 bytes) TCP: Hash tables configured (established 262144 bind 65536) TCP reno registered audit: initializing netlink socket (disabled) type=2000 audit(1300575339.480:1): initialized Total HugeTLB memory allocated, 0 VFS: Disk quotas dquot_6.5.1 Dquot-cache hash table entries: 512 (order 0, 4096 bytes) SELinux: Registering netfilter hooks Initializing Cryptographic API alg: No test for crc32c (crc32c-generic) ksign: Installing public key data Loading keyring - Added public key 1DF8C62584B4E7F2 - User ID: CentOS (Kernel Module GPG key) io scheduler noop registered io scheduler anticipatory registered io scheduler deadline registered io scheduler cfq registered (default) Boot video device is 0000:01:05.0 PCI: Setting latency timer of device 0000:00:04.0 to 64 PCI: Setting latency timer of device 0000:00:05.0 to 64 pci_hotplug: PCI Hot Plug PCI Core version: 0.5 ACPI: duty_cycle spans bit 4 ACPI: CPU0 (power states: C1[C1] C2[C2] C3[C3]) Real Time Clock Driver v1.12ac hpet_resources: 0xfed00000 is busy Non-volatile memory driver v1.2 Linux agpgart interface v0.101 (c) Dave Jones Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing enabled serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A 00:05: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A 00:06: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A brd: module loaded Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2 ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx SB600_PATA: IDE controller at PCI slot 0000:00:14.1 GSI 16 sharing vector 0xC1 and IRQ 16 ACPI: PCI Interrupt 0000:00:14.1[A] -> GSI 16 (level, low) -> IRQ 193 SB600_PATA: chipset revision 0 SB600_PATA: not 100% native mode: will probe irqs later ide0: BM-DMA at 0xff00-0xff07, BIOS settings: hda:pio, hdb:pio Probing IDE interface ide0... Probing IDE interface ide0... Probing IDE interface ide1... ide-floppy driver 0.99.newide usbcore: registered new driver hiddev usbcore: registered new driver usbhid drivers/usb/input/hid-core.c: v2.6:USB HID core driver PNP: PS/2 Controller [PNP0303:PS2K] at 0x60,0x64 irq 1 PNP: PS/2 controller doesn't have AUX irq; using default 12 serio: i8042 KBD port at 0x60,0x64 irq 1 serio: i8042 AUX port at 0x60,0x64 irq 12 mice: PS/2 mouse device common for all mice md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27 md: bitmap version 4.39 TCP bic registered Initializing IPsec netlink socket NET: Registered protocol family 1 NET: Registered protocol family 17 ACPI: (supports S0 S1 S3 S4 S5) Initalizing network drop monitor service Freeing unused kernel memory: 212k freed Write protecting the kernel read-only data: 505k GSI 17 sharing vector 0xC9 and IRQ 17 ACPI: PCI Interrupt 0000:00:13.5[D] -> GSI 19 (level, low) -> IRQ 201 ehci_hcd 0000:00:13.5: EHCI Host Controller ehci_hcd 0000:00:13.5: new USB bus registered, assigned bus number 1 ehci_hcd 0000:00:13.5: applying AMD SB600/SB700 USB freeze workaround ehci_hcd 0000:00:13.5: debug port 1 ehci_hcd 0000:00:13.5: irq 201, io mem 0xfe7ff000 ehci_hcd 0000:00:13.5: USB 2.0 started, EHCI 1.00, driver 10 Dec 2004 usb usb1: configuration #1 chosen from 1 choice hub 1-0:1.0: USB hub found hub 1-0:1.0: 10 ports detected ohci_hcd: 2005 April 22 USB 1.1 'Open' Host Controller (OHCI) Driver (PCI) ACPI: PCI Interrupt 0000:00:13.0[A] -> GSI 16 (level, low) -> IRQ 193 ohci_hcd 0000:00:13.0: OHCI Host Controller ohci_hcd 0000:00:13.0: new USB bus registered, assigned bus number 2 ohci_hcd 0000:00:13.0: irq 193, io mem 0xfe7fe000 usb usb2: configuration #1 chosen from 1 choice hub 2-0:1.0: USB hub found hub 2-0:1.0: 2 ports detected GSI 18 sharing vector 0xD1 and IRQ 18 ACPI: PCI Interrupt 0000:00:13.1[B] -> GSI 17 (level, low) -> IRQ 209 ohci_hcd 0000:00:13.1: OHCI Host Controller ohci_hcd 0000:00:13.1: new USB bus registered, assigned bus number 3 ohci_hcd 0000:00:13.1: irq 209, io mem 0xfe7fd000 usb usb3: configuration #1 chosen from 1 choice hub 3-0:1.0: USB hub found hub 3-0:1.0: 2 ports detected GSI 19 sharing vector 0xD9 and IRQ 19 ACPI: PCI Interrupt 0000:00:13.2[C] -> GSI 18 (level, low) -> IRQ 217 ohci_hcd 0000:00:13.2: OHCI Host Controller ohci_hcd 0000:00:13.2: new USB bus registered, assigned bus number 4 ohci_hcd 0000:00:13.2: irq 217, io mem 0xfe7fc000 usb usb4: configuration #1 chosen from 1 choice hub 4-0:1.0: USB hub found hub 4-0:1.0: 2 ports detected ACPI: PCI Interrupt 0000:00:13.3[B] -> GSI 17 (level, low) -> IRQ 209 ohci_hcd 0000:00:13.3: OHCI Host Controller ohci_hcd 0000:00:13.3: new USB bus registered, assigned bus number 5 ohci_hcd 0000:00:13.3: irq 209, io mem 0xfe7fb000 usb usb5: configuration #1 chosen from 1 choice hub 5-0:1.0: USB hub found hub 5-0:1.0: 2 ports detected ACPI: PCI Interrupt 0000:00:13.4[C] -> GSI 18 (level, low) -> IRQ 217 ohci_hcd 0000:00:13.4: OHCI Host Controller ohci_hcd 0000:00:13.4: new USB bus registered, assigned bus number 6 ohci_hcd 0000:00:13.4: irq 217, io mem 0xfe7fa000 usb usb6: configuration #1 chosen from 1 choice hub 6-0:1.0: USB hub found hub 6-0:1.0: 2 ports detected USB Universal Host Controller Interface driver v3.0 md: raid1 personality registered for level 1 SCSI subsystem initialized libata version 3.00 loaded. ahci 0000:00:12.0: version 3.0 GSI 20 sharing vector 0xE1 and IRQ 20 ACPI: PCI Interrupt 0000:00:12.0[A] -> GSI 22 (level, low) -> IRQ 225 ahci 0000:00:12.0: controller can't do 64bit DMA, forcing 32bit ahci 0000:00:12.0: AHCI 0001.0100 32 slots 4 ports 3 Gbps 0xf impl SATA mode ahci 0000:00:12.0: flags: ncq sntf ilck pm led clo pmp pio slum part scsi0 : ahci scsi1 : ahci scsi2 : ahci scsi3 : ahci ata1: SATA max UDMA/133 abar m1024@0xfe7ff800 port 0xfe7ff900 irq 225 ata2: SATA max UDMA/133 abar m1024@0xfe7ff800 port 0xfe7ff980 irq 225 ata3: SATA max UDMA/133 abar m1024@0xfe7ff800 port 0xfe7ffa00 irq 225 ata4: SATA max UDMA/133 abar m1024@0xfe7ff800 port 0xfe7ffa80 irq 225 ata1: softreset failed (device not ready) ata1: failed due to HW bug, retry pmp=0 ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300) ata1.00: ATA-8: Hitachi HDS721050CLA362, JP2OA39C, max UDMA/133 ata1.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 31/32) ata1.00: SB600 AHCI: limiting to 255 sectors per cmd ata1.00: SB600 AHCI: limiting to 255 sectors per cmd ata1.00: configured for UDMA/133 ata2: softreset failed (device not ready) ata2: failed due to HW bug, retry pmp=0 ata2: SATA link up 3.0 Gbps (SStatus 123 SControl 300) ata2.00: ATA-8: Hitachi HDS721050CLA362, JP2OA39C, max UDMA/133 ata2.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 31/32) ata2.00: SB600 AHCI: limiting to 255 sectors per cmd ata2.00: SB600 AHCI: limiting to 255 sectors per cmd ata2.00: configured for UDMA/133 ata3: SATA link down (SStatus 0 SControl 300) ata4: SATA link down (SStatus 0 SControl 300) Vendor: ATA Model: Hitachi HDS72105 Rev: JP2O Type: Direct-Access ANSI SCSI revision: 05 SCSI device sda: 976773168 512-byte hdwr sectors (500108 MB) sda: Write Protect is off sda: Mode Sense: 00 3a 00 00 SCSI device sda: drive cache: write back SCSI device sda: 976773168 512-byte hdwr sectors (500108 MB) sda: Write Protect is off sda: Mode Sense: 00 3a 00 00 SCSI device sda: drive cache: write back sda: sda1 sda2 sda3 sda4 < sda5 sda6 > sd 0:0:0:0: Attached scsi disk sda Vendor: ATA Model: Hitachi HDS72105 Rev: JP2O Type: Direct-Access ANSI SCSI revision: 05 SCSI device sdb: 976773168 512-byte hdwr sectors (500108 MB) sdb: Write Protect is off sdb: Mode Sense: 00 3a 00 00 SCSI device sdb: drive cache: write back SCSI device sdb: 976773168 512-byte hdwr sectors (500108 MB) sdb: Write Protect is off sdb: Mode Sense: 00 3a 00 00 SCSI device sdb: drive cache: write back sdb: sdb1 sdb2 sdb3 sdb4 < sdb5 sdb6 > sd 1:0:0:0: Attached scsi disk sdb device-mapper: uevent: version 1.0.3 device-mapper: ioctl: 4.11.5-ioctl (2007-12-12) initialised: dm-devel@redhat.com device-mapper: dm-raid45: initialized v0.2594l md: Autodetecting RAID arrays. md: autorun ... md: considering sdb6 ... md: adding sdb6 ... md: sdb5 has different UUID to sdb6 md: sdb3 has different UUID to sdb6 md: sdb1 has different UUID to sdb6 md: adding sda6 ... md: sda5 has different UUID to sdb6 md: sda3 has different UUID to sdb6 md: sda1 has different UUID to sdb6 md: created md3 md: bind<sda6> md: bind<sdb6> md: running: <sdb6><sda6> raid1: raid set md3 active with 2 out of 2 mirrors md: considering sdb5 ... md: adding sdb5 ... md: sdb3 has different UUID to sdb5 md: sdb1 has different UUID to sdb5 md: adding sda5 ... md: sda3 has different UUID to sdb5 md: sda1 has different UUID to sdb5 md: created md2 md: bind<sda5> md: bind<sdb5> md: running: <sdb5><sda5> raid1: raid set md2 active with 2 out of 2 mirrors md: considering sdb3 ... md: adding sdb3 ... md: sdb1 has different UUID to sdb3 md: adding sda3 ... md: sda1 has different UUID to sdb3 md: created md1 md: bind<sda3> md: bind<sdb3> md: running: <sdb3><sda3> raid1: raid set md1 active with 2 out of 2 mirrors md: considering sdb1 ... md: adding sdb1 ... md: adding sda1 ... md: created md0 md: bind<sda1> md: bind<sdb1> md: running: <sdb1><sda1> raid1: raid set md0 active with 2 out of 2 mirrors md: ... autorun DONE. kjournald starting. Commit interval 5 seconds EXT3-fs: mounted filesystem with ordered data mode. SELinux: Disabled at runtime. SELinux: Unregistering netfilter hooks type=1404 audit(1300575368.057:2): selinux=0 auid=4294967295 ses=4294967295 piix4_smbus 0000:00:14.0: Found 0000:00:14.0 device EDAC MC: Ver: 2.0.1 Jan 5 2011 EDAC amd64_edac: Ver: 3.2.0 Jan 5 2011 EDAC amd64: This node reports that Memory ECC is currently disabled, set F3x44[22] (0000:00:18.3). EDAC amd64: WARNING: ECC is disabled by BIOS. Module will NOT be loaded. Either Enable ECC in the BIOS, or set 'ecc_enable_override'. Also, use of the override can cause unknown side effects. amd64_edac: probe of 0000:00:18.2 failed with error -22 Floppy drive(s): fd0 is 1.44M e1000e: Intel(R) PRO/1000 Network Driver - 1.0.2-k3.1 e1000e: Copyright (c) 1999-2008 Intel Corporation. ACPI: PCI Interrupt 0000:02:00.0[A] -> GSI 16 (level, low) -> IRQ 193 PCI: Setting latency timer of device 0000:02:00.0 to 64 input: PC Speaker as /class/input/input0 e1000e 0000:02:00.0: Warning: detected ASPM enabled in EEPROM eth0: (PCI Express:2.5GB/s:Width x1) 40:61:86:ee:20:46 eth0: Intel(R) PRO/1000 Network Connection eth0: MAC: 2, PHY: 2, PBA No: ffffff-0ff ACPI: PCI Interrupt 0000:03:00.0[A] -> GSI 17 (level, low) -> IRQ 209 PCI: Setting latency timer of device 0000:03:00.0 to 64 sd 0:0:0:0: Attached scsi generic sg0 type 0 sd 1:0:0:0: Attached scsi generic sg1 type 0 e1000e 0000:03:00.0: Warning: detected ASPM enabled in EEPROM eth1: (PCI Express:2.5GB/s:Width x1) 40:61:86:ee:20:47 eth1: Intel(R) PRO/1000 Network Connection eth1: MAC: 2, PHY: 2, PBA No: ffffff-0ff shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 floppy0: no floppy controllers found Floppy drive(s): fd0 is 1.44M floppy0: no floppy controllers found lp: driver loaded but no devices found ACPI: Power Button (FF) [PWRF] ACPI: Power Button (CM) [PWRB] ACPI: Mapper loaded dell-wmi: No known WMI GUID found md: Autodetecting RAID arrays. md: autorun ... md: ... autorun DONE. device-mapper: multipath: version 1.0.5 loaded loop: loaded (max 8 devices) EXT3 FS on md1, internal journal kjournald starting. Commit interval 5 seconds EXT3 FS on md3, internal journal EXT3-fs: mounted filesystem with ordered data mode. kjournald starting. Commit interval 5 seconds EXT3 FS on md0, internal journal EXT3-fs: mounted filesystem with ordered data mode. kjournald starting. Commit interval 5 seconds EXT3 FS on md2, internal journal EXT3-fs: mounted filesystem with ordered data mode. Adding 3999736k swap on /dev/sda2. Priority:-1 extents:1 across:3999736k Adding 3999736k swap on /dev/sdb2. Priority:-2 extents:1 across:3999736k powernow-k8: Found 1 Quad-Core AMD Opteron(tm) Processor 1381 processors (4 cpu cores) (version 2.20.00) powernow-k8: 0 : fid 0x0 gid 0x0 (2500 MHz) powernow-k8: 1 : fid 0x0 gid 0x0 (1800 MHz) powernow-k8: 2 : fid 0x0 gid 0x0 (1300 MHz) powernow-k8: 3 : fid 0x0 gid 0x0 (800 MHz) ip_tables: (C) 2000-2006 Netfilter Core Team Netfilter messages via NETLINK v0.30. ip_conntrack version 2.4 (8192 buckets, 65536 max) - 304 bytes per conntrack NET: Registered protocol family 10 lo: Disabled Privacy Extensions IPv6 over IPv4 tunneling driver ADDRCONF(NETDEV_UP): eth0: link is not ready e1000e: eth0 NIC Link is Up 100 Mbps Full Duplex, Flow Control: None eth0: 10/100 speed: disabling TSO ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready ADDRCONF(NETDEV_UP): eth1: link is not ready md: syncing RAID array md0 md: minimum _guaranteed_ reconstruction speed: 1000 KB/sec/disc. md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for reconstruction. md: using 128k window, over a total of 1023936 blocks. md: delaying resync of md2 until md0 has finished resync (they share one or more physical units) md: delaying resync of md1 until md0 has finished resync (they share one or more physical units) md: delaying resync of md2 until md0 has finished resync (they share one or more physical units) md: md0: sync done. md: syncing RAID array md2 md: minimum _guaranteed_ reconstruction speed: 1000 KB/sec/disc. md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for reconstruction. md: using 128k window, over a total of 277728192 blocks. md: delaying resync of md1 until md2 has finished resync (they share one or more physical units) RAID1 conf printout: --- wd:2 rd:2 disk 0, wo:0, o:1, dev:sda1 disk 1, wo:0, o:1, dev:sdb1 INFO: task md1_resync:9770 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. md1_resync D ffff81011edba820 0 9770 55 9768 (L-TLB) ffff810083259d70 0000000000000046 0000000000000000 ffff81011ea66e0c ffff81011ea66c0c 000000000000000a ffff81011ccc27a0 ffff81011edba820 000011d56f2a0b28 000000000000100f ffff81011ccc2988 000000028008b4d7 Call Trace: [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021af66>] md_do_sync+0x1d8/0x833 [<ffffffff8008ca47>] enqueue_task+0x41/0x56 [<ffffffff8008cab2>] __activate_task+0x56/0x6d [<ffffffff8008c897>] dequeue_task+0x18/0x37 [<ffffffff80062ff8>] thread_return+0x62/0xfe [<ffffffff800a0b5f>] autoremove_wake_function+0x0/0x2e [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021b93a>] md_thread+0xf8/0x10e [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021b842>] md_thread+0x0/0x10e [<ffffffff8003296e>] kthread+0xfe/0x132 [<ffffffff8005dfb1>] child_rip+0xa/0x11 [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff80032870>] kthread+0x0/0x132 [<ffffffff8005dfa7>] child_rip+0x0/0x11
INFO: task md1_resync:9770 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. md1_resync D ffff81011edba820 0 9770 55 9768 (L-TLB) ffff810083259d70 0000000000000046 0000000000000000 ffff81011ea66e0c ffff81011ea66c0c 000000000000000a ffff81011ccc27a0 ffff81011edba820 000011d56f2a0b28 000000000000100f ffff81011ccc2988 000000028008b4d7 Call Trace: [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021af66>] md_do_sync+0x1d8/0x833 [<ffffffff8008ca47>] enqueue_task+0x41/0x56 [<ffffffff8008cab2>] __activate_task+0x56/0x6d [<ffffffff8008c897>] dequeue_task+0x18/0x37 [<ffffffff80062ff8>] thread_return+0x62/0xfe [<ffffffff800a0b5f>] autoremove_wake_function+0x0/0x2e [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021b93a>] md_thread+0xf8/0x10e [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021b842>] md_thread+0x0/0x10e [<ffffffff8003296e>] kthread+0xfe/0x132 [<ffffffff8005dfb1>] child_rip+0xa/0x11 [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff80032870>] kthread+0x0/0x132 [<ffffffff8005dfa7>] child_rip+0x0/0x11
INFO: task md1_resync:9770 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. md1_resync D ffff81011edba820 0 9770 55 9768 (L-TLB) ffff810083259d70 0000000000000046 0000000000000000 ffff81011ea66e0c ffff81011ea66c0c 000000000000000a ffff81011ccc27a0 ffff81011edba820 000011d56f2a0b28 000000000000100f ffff81011ccc2988 000000028008b4d7 Call Trace: [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021af66>] md_do_sync+0x1d8/0x833 [<ffffffff8008ca47>] enqueue_task+0x41/0x56 [<ffffffff8008cab2>] __activate_task+0x56/0x6d [<ffffffff8008c897>] dequeue_task+0x18/0x37 [<ffffffff80062ff8>] thread_return+0x62/0xfe [<ffffffff800a0b5f>] autoremove_wake_function+0x0/0x2e [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021b93a>] md_thread+0xf8/0x10e [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021b842>] md_thread+0x0/0x10e [<ffffffff8003296e>] kthread+0xfe/0x132 [<ffffffff8005dfb1>] child_rip+0xa/0x11 [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff80032870>] kthread+0x0/0x132 [<ffffffff8005dfa7>] child_rip+0x0/0x11
INFO: task md1_resync:9770 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. md1_resync D ffff81011edba820 0 9770 55 9768 (L-TLB) ffff810083259d70 0000000000000046 0000000000000000 ffff81011ea66e0c ffff81011ea66c0c 000000000000000a ffff81011ccc27a0 ffff81011edba820 000011d56f2a0b28 000000000000100f ffff81011ccc2988 000000028008b4d7 Call Trace: [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021af66>] md_do_sync+0x1d8/0x833 [<ffffffff8008ca47>] enqueue_task+0x41/0x56 [<ffffffff8008cab2>] __activate_task+0x56/0x6d [<ffffffff8008c897>] dequeue_task+0x18/0x37 [<ffffffff80062ff8>] thread_return+0x62/0xfe [<ffffffff800a0b5f>] autoremove_wake_function+0x0/0x2e [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021b93a>] md_thread+0xf8/0x10e [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021b842>] md_thread+0x0/0x10e [<ffffffff8003296e>] kthread+0xfe/0x132 [<ffffffff8005dfb1>] child_rip+0xa/0x11 [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff80032870>] kthread+0x0/0x132 [<ffffffff8005dfa7>] child_rip+0x0/0x11
INFO: task md1_resync:9770 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. md1_resync D ffff81011edba820 0 9770 55 9768 (L-TLB) ffff810083259d70 0000000000000046 0000000000000000 ffff81011ea66e0c ffff81011ea66c0c 000000000000000a ffff81011ccc27a0 ffff81011edba820 000011d56f2a0b28 000000000000100f ffff81011ccc2988 000000028008b4d7 Call Trace: [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021af66>] md_do_sync+0x1d8/0x833 [<ffffffff8008ca47>] enqueue_task+0x41/0x56 [<ffffffff8008cab2>] __activate_task+0x56/0x6d [<ffffffff8008c897>] dequeue_task+0x18/0x37 [<ffffffff80062ff8>] thread_return+0x62/0xfe [<ffffffff800a0b5f>] autoremove_wake_function+0x0/0x2e [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021b93a>] md_thread+0xf8/0x10e [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021b842>] md_thread+0x0/0x10e [<ffffffff8003296e>] kthread+0xfe/0x132 [<ffffffff8005dfb1>] child_rip+0xa/0x11 [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff80032870>] kthread+0x0/0x132 [<ffffffff8005dfa7>] child_rip+0x0/0x11
INFO: task md1_resync:9770 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. md1_resync D ffff81011edba820 0 9770 55 9768 (L-TLB) ffff810083259d70 0000000000000046 0000000000000000 ffff81011ea66e0c ffff81011ea66c0c 000000000000000a ffff81011ccc27a0 ffff81011edba820 000011d56f2a0b28 000000000000100f ffff81011ccc2988 000000028008b4d7 Call Trace: [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021af66>] md_do_sync+0x1d8/0x833 [<ffffffff8008ca47>] enqueue_task+0x41/0x56 [<ffffffff8008cab2>] __activate_task+0x56/0x6d [<ffffffff8008c897>] dequeue_task+0x18/0x37 [<ffffffff80062ff8>] thread_return+0x62/0xfe [<ffffffff800a0b5f>] autoremove_wake_function+0x0/0x2e [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021b93a>] md_thread+0xf8/0x10e [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021b842>] md_thread+0x0/0x10e [<ffffffff8003296e>] kthread+0xfe/0x132 [<ffffffff8005dfb1>] child_rip+0xa/0x11 [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff80032870>] kthread+0x0/0x132 [<ffffffff8005dfa7>] child_rip+0x0/0x11
INFO: task md1_resync:9770 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. md1_resync D ffff81011edba820 0 9770 55 9768 (L-TLB) ffff810083259d70 0000000000000046 0000000000000000 ffff81011ea66e0c ffff81011ea66c0c 000000000000000a ffff81011ccc27a0 ffff81011edba820 000011d56f2a0b28 000000000000100f ffff81011ccc2988 000000028008b4d7 Call Trace: [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021af66>] md_do_sync+0x1d8/0x833 [<ffffffff8008ca47>] enqueue_task+0x41/0x56 [<ffffffff8008cab2>] __activate_task+0x56/0x6d [<ffffffff8008c897>] dequeue_task+0x18/0x37 [<ffffffff80062ff8>] thread_return+0x62/0xfe [<ffffffff800a0b5f>] autoremove_wake_function+0x0/0x2e [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021b93a>] md_thread+0xf8/0x10e [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021b842>] md_thread+0x0/0x10e [<ffffffff8003296e>] kthread+0xfe/0x132 [<ffffffff8005dfb1>] child_rip+0xa/0x11 [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff80032870>] kthread+0x0/0x132 [<ffffffff8005dfa7>] child_rip+0x0/0x11
INFO: task md1_resync:9770 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. md1_resync D ffff81011edba820 0 9770 55 9768 (L-TLB) ffff810083259d70 0000000000000046 0000000000000000 ffff81011ea66e0c ffff81011ea66c0c 000000000000000a ffff81011ccc27a0 ffff81011edba820 000011d56f2a0b28 000000000000100f ffff81011ccc2988 000000028008b4d7 Call Trace: [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021af66>] md_do_sync+0x1d8/0x833 [<ffffffff8008ca47>] enqueue_task+0x41/0x56 [<ffffffff8008cab2>] __activate_task+0x56/0x6d [<ffffffff8008c897>] dequeue_task+0x18/0x37 [<ffffffff80062ff8>] thread_return+0x62/0xfe [<ffffffff800a0b5f>] autoremove_wake_function+0x0/0x2e [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021b93a>] md_thread+0xf8/0x10e [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021b842>] md_thread+0x0/0x10e [<ffffffff8003296e>] kthread+0xfe/0x132 [<ffffffff8005dfb1>] child_rip+0xa/0x11 [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff80032870>] kthread+0x0/0x132 [<ffffffff8005dfa7>] child_rip+0x0/0x11
INFO: task md1_resync:9770 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. md1_resync D ffff81011edba820 0 9770 55 9768 (L-TLB) ffff810083259d70 0000000000000046 0000000000000000 ffff81011ea66e0c ffff81011ea66c0c 000000000000000a ffff81011ccc27a0 ffff81011edba820 000011d56f2a0b28 000000000000100f ffff81011ccc2988 000000028008b4d7 Call Trace: [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021af66>] md_do_sync+0x1d8/0x833 [<ffffffff8008ca47>] enqueue_task+0x41/0x56 [<ffffffff8008cab2>] __activate_task+0x56/0x6d [<ffffffff8008c897>] dequeue_task+0x18/0x37 [<ffffffff80062ff8>] thread_return+0x62/0xfe [<ffffffff800a0b5f>] autoremove_wake_function+0x0/0x2e [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021b93a>] md_thread+0xf8/0x10e [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021b842>] md_thread+0x0/0x10e [<ffffffff8003296e>] kthread+0xfe/0x132 [<ffffffff8005dfb1>] child_rip+0xa/0x11 [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff80032870>] kthread+0x0/0x132 [<ffffffff8005dfa7>] child_rip+0x0/0x11
INFO: task md1_resync:9770 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. md1_resync D ffff81011edba820 0 9770 55 9768 (L-TLB) ffff810083259d70 0000000000000046 0000000000000000 ffff81011ea66e0c ffff81011ea66c0c 000000000000000a ffff81011ccc27a0 ffff81011edba820 000011d56f2a0b28 000000000000100f ffff81011ccc2988 000000028008b4d7 Call Trace: [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021af66>] md_do_sync+0x1d8/0x833 [<ffffffff8008ca47>] enqueue_task+0x41/0x56 [<ffffffff8008cab2>] __activate_task+0x56/0x6d [<ffffffff8008c897>] dequeue_task+0x18/0x37 [<ffffffff80062ff8>] thread_return+0x62/0xfe [<ffffffff800a0b5f>] autoremove_wake_function+0x0/0x2e [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021b93a>] md_thread+0xf8/0x10e [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff8021b842>] md_thread+0x0/0x10e [<ffffffff8003296e>] kthread+0xfe/0x132 [<ffffffff8005dfb1>] child_rip+0xa/0x11 [<ffffffff800a0947>] keventd_create_kthread+0x0/0xc4 [<ffffffff80032870>] kthread+0x0/0x132 [<ffffffff8005dfa7>] child_rip+0x0/0x11
md: md2: sync done. md: syncing RAID array md1 md: minimum _guaranteed_ reconstruction speed: 1000 KB/sec/disc. md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for reconstruction. md: using 128k window, over a total of 20479936 blocks. RAID1 conf printout: --- wd:2 rd:2 disk 0, wo:0, o:1, dev:sda5 disk 1, wo:0, o:1, dev:sdb5 md: md1: sync done. RAID1 conf printout: --- wd:2 rd:2 disk 0, wo:0, o:1, dev:sda3 disk 1, wo:0, o:1, dev:sdb3
2011/3/20 Alexander Farber alexander.farber@gmail.com
Hello,
yesterday night I had a problem with my server located at a hoster (strato.de). I couldn't ssh to it and over the remote serial console I saw "out of memory" errors (sorry, don't have the text).
Then I had reinstall CentOS 5.5/64 bit + all my setup (2h work), because I have a contract with a social network and they will shut down my little card game if it is not reponding.
Now the server seems to work ok, but I'm worried about those /var/log/message:
kernel: INFO: task md1_resync:9770 blocked for more than 120 seconds. kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables
My guess is that you only saw these messages while the RAID sync was still going on? You got those messages due to the system I/O being stressed, which hung the system in periods.
I wouldn't worry about it if your RAID is now in sync and you don't see the error messages anymore. You can lower the I/O stress of the system under a RAID-resync by setting a lower maximum kb/sek in /proc/sys/dev/raid/speed_limit_max (default is 200.000kb/sec ~ 200mb/sec). This will of course also extend the time used to complete the sync (which also can be bad, as you want it back in sync as fast as possible).
Best regards Kenni
Thanks Kenni, could you advise any commands for checking RAID status or health
On 20.3.2011 09:25, Alexander Farber wrote:
but I'm worried about those /var/log/message:
kernel: INFO: task md1_resync:9770 blocked for more than 120 seconds.
https://bugzilla.redhat.com/show_bug.cgi?id=573106#c31
Thank you, I've decreased /proc/sys/dev/raid/speed_limit_max from 200000 to 100000.
I think I don't care about the sync speed, but I'd like to avoid the OOM errors and server lockup like I had yesterday
(still not sure if this will help here or if it is just to get rid of the warning)
Regards Alex
On 20.3.2011 13:48, Alexander Farber wrote:
Thank you, I've decreased /proc/sys/dev/raid/speed_limit_max from 200000 to 100000.
I think I don't care about the sync speed, but I'd like to avoid the OOM errors and server lockup like I had yesterday
So you think the OOM was related to raid resync ?
(still not sure if this will help here or if it is just to get rid of the warning)
I do not see how decreasing the speed_limit_max should avoid the mdX_resync warnings. I would expect more of these warnings now, because sync takes longer? And: These warnings are harmless! Just ignore it, but you know that probably after reading bugzilla 573106
2011/3/20 Alexander Farber alexander.farber@gmail.com
Thank you, I've decreased /proc/sys/dev/raid/speed_limit_max from 200000 to 100000.
200000 is just the theoretical maximum. If your discs max out at 80000, you'll need to set it lower than that. While syncing, you can check the current sync speed with: cat /proc/mdstat
I think I don't care about the sync speed, but I'd like to avoid the OOM errors and server lockup like I had yesterday
AFAIK, the errors are harmless, it's some locking bug in the kernel which just hasn't been fixed in CentOS 5 yet. This is not related to any out-of-memory errors, and hence most likely not related to the lockup you experienced.
2011/3/20 Markus Falb markus.falb@fasel.at:
Ahh, yes, I forgot about that bugreport. According to that report, the issue has been fixed in the kernel in upstream 5.6...so it will get fixed in CentOS 5.6.
I do not see how decreasing the speed_limit_max should avoid the mdX_resync warnings. I would expect more of these warnings now, because sync takes longer?
Hmm, I received the same error messages on a Core i7 system I installed recently. While syncing, the system was close to being completely unresponsive (took ages to just get a SSH-connection). After limiting the I/O by setting a lower maximum sync speed, the system got responsive and the messages disappeared. Comment #36 in the bug report actually suggests the same workaround.
Best regards Kenni
Thanks for your replies. I've also seen the following in my log now:
Mar 20 05:08:55 mysite kernel: md: md1: sync done. Mar 20 05:08:55 mysite kernel: RAID1 conf printout: Mar 20 05:08:55 mysite kernel: --- wd:2 rd:2 Mar 20 05:08:55 mysite kernel: disk 0, wo:0, o:1, dev:sda3 Mar 20 05:08:55 mysite kernel: disk 1, wo:0, o:1, dev:sdb3