[CentOS] Trying to get a kernel dump

Thu May 30 21:02:32 UTC 2013
Lists <lists at benjamindsmith.com>

Trying to debug a database server getting cpu softlocks causing SSHD to 
hang and not let anybody log in. Figured that a good first step would be 
to get some kernel dumps when the problem occurs. According to what I 
read at the following web site, I can get dumps for softlock problems 
too. Here's the site:
http://blog.kreyolys.com/2011/03/17/no-panic-its-just-a-kernel-panic/

Using a throwaway 32 bit dev server, I followed the directions exactly 
with the below modifications.

     A) I got the kernel-debuginfo and kernel-debuginfo-common by enabling
/etc/yum.repos.d/CentOS-Debuginfo.repo and then installing with yum.

     B) One other change is that I had to use an alternate partition 
instead of the one resolving "/var" because the server was set up with 
LVM. The partition that I used ("/dev/sdb3") is actually mounted at 
/alt/home.


So when I simulate a crash by piping a "c" to /proc/sysrq-trigger, I see 
the following in /var/log/messages:
May 30 13:37:24 norman kernel: SysRq : HELP : loglevel(0-9) reBoot Crash 
terminate-all-tasks(E) memory-full-oom-kill(F) kill-all-tasks(I) 
thaw-filesystems(J) saK show-backtrace-all-active-cpus(L) 
show-memory-usage(M) nice-all-RT-tasks(N) powerOff show-registers(P) 
show-all-timers(Q) unRaw Sync show-task-states(T) Unmount 
show-blocked-tasks(W) dump-ftrace-buffer(Z)

But nothing else. NO files in /var/crash, or /alt/home/var/crash 
(directory exists) Nothing happens at all when I run
# echo 1 > /proc/sys/kernel/softlockup_panic

I've tried tweaking a few settings in /etc/kdump.conf, restarting after 
each time (since it creates a new boot img) to no avail. What am I messing?

Here's a bunch of hopefully relevant info:

kernel line in grub.conf:
kernel /vmlinuz-2.6.32-358.6.2.el6.i686 ro 
root=/dev/mapper/VolGroup-lv_root nomodeset rd_NO_LUKS LANG=en_US.UTF-8 
rd_NO_MD rd_LVM_LV=VolGroup/lv_swap SYSFONT=latarcyrheb-sun16 rhgb 
crashkernel=auto quiet rd_LVM_LV=VolGroup/lv_root  KEYBOARDTYPE=pc 
KEYTABLE=us rd_NO_DM crashkernel=128M nmi_watchdog=1

# grep -v "#" /etc/kdump.conf
ext4 /dev/sdb3
path /var/crash
core_collector makedumpfile -c -d 31


# cat /proc/meminfo
MemTotal:         379352 kB
MemFree:          232748 kB
Buffers:           22496 kB
Cached:            79040 kB
SwapCached:            0 kB
Active:            60512 kB
Inactive:          60152 kB
Active(anon):      20028 kB
Inactive(anon):     4640 kB
Active(file):      40484 kB
Inactive(file):    55512 kB
Unevictable:           0 kB
Mlocked:               0 kB
HighTotal:             0 kB
HighFree:              0 kB
LowTotal:         379352 kB
LowFree:          232748 kB
SwapTotal:       1048568 kB
SwapFree:        1048568 kB
Dirty:                48 kB
Writeback:             0 kB
AnonPages:         19124 kB
Mapped:            14320 kB
Shmem:              5556 kB
Slab:              16216 kB
SReclaimable:       8444 kB
SUnreclaim:         7772 kB
KernelStack:         816 kB
PageTables:         2188 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     1238244 kB
Committed_AS:     164964 kB
VmallocTotal:     506396 kB
VmallocUsed:        5344 kB
VmallocChunk:     491856 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        9692 kB
DirectMap2M:      514048 kB




// IN sysctl.conf
kernel.unknown_nmi_panic=1
kernel.panic_on_unrecovered_nmi=1
vm.panic_on_oom=1
kernel.sysrq = 1

# /cat/proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 15
model           : 1
model name      : Intel(R) Pentium(R) 4 CPU 1.50GHz
stepping        : 2
cpu MHz         : 1495.192
cache size      : 256 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 2
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca 
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm up pebs bts
bogomips        : 2990.38
clflush size    : 64
cache_alignment : 128
address sizes   : 36 bits physical, 32 bits virtual
power management:

# dmesg:
Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Linux version 2.6.32-358.6.2.el6.i686 
(mockbuild at c6b8.bsys.dev.centos.org) (gcc version 4.4.7 20120313 (Red 
Hat 4.4.7-3) (GCC) ) #1 SMP Thu May 16 18:12:13 UTC 2013
KERNEL supported cpus:
   Intel GenuineIntel
   AMD AuthenticAMD
   NSC Geode by NSC
   Cyrix CyrixInstead
   Centaur CentaurHauls
   Transmeta GenuineTMx86
   Transmeta TransmetaCPU
   UMC UMC UMC UMC
BIOS-provided physical RAM map:
  BIOS-e820: 0000000000000000 - 00000000000a0000 (usable)
  BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
  BIOS-e820: 0000000000100000 - 000000001ff77000 (usable)
  BIOS-e820: 000000001ff77000 - 000000001ff79000 (ACPI NVS)
  BIOS-e820: 000000001ff79000 - 0000000020000000 (reserved)
  BIOS-e820: 00000000fec00000 - 00000000fec10000 (reserved)
  BIOS-e820: 00000000fee00000 - 00000000fee10000 (reserved)
  BIOS-e820: 00000000ffb00000 - 0000000100000000 (reserved)
DMI 2.3 present.
SMBIOS version 2.3 @ 0xF0430
DMI: Dell Computer Corporation OptiPlex GX240 /OptiPlex 
GX240               , BIOS A05 07/19/2002
e820 update range: 0000000000000000 - 0000000000001000 (usable) ==> 
(reserved)
e820 remove range: 00000000000a0000 - 0000000000100000 (usable)
last_pfn = 0x1ff77 max_arch_pfn = 0x400000
MTRR default type: uncachable
MTRR fixed ranges enabled:
   00000-9FFFF write-back
   A0000-BFFFF uncachable
   C0000-DFFFF write-protect
   E0000-EFFFF uncachable
   F0000-FFFFF write-protect
MTRR variable ranges enabled:
   0 base 000000000 mask FE0000000 write-back
   1 disabled
   2 disabled
   3 disabled
   4 disabled
   5 disabled
   6 disabled
   7 disabled
x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
initial memory mapped : 0 - 01000000
init_memory_mapping: 0000000000000000-000000001ff77000
Using x86 segment limits to approximate NX protection
  0000000000 - 0000200000 page 4k
  0000200000 - 001fe00000 page 2M
  001fe00000 - 001ff77000 page 4k
kernel direct mapping tables up to 1ff77000 @ 7000-f000
RAMDISK: 1f211000 - 1ff66d6a
ACPI: RSDP 000fd560 00014 (v00 DELL  )
ACPI: RSDT 000fd574 00034 (v01 DELL    GX240   00000008 ASL 00000061)
ACPI: FACP 000fd5a8 00074 (v01 DELL    GX240   00000008 ASL 00000061)
ACPI: DSDT fffe3c22 02393 (v01   DELL    dt_ex 00001000 MSFT 0100000D)
ACPI: FACS 1ff77000 00040
ACPI: SSDT fffe60f2 000BA (v01   DELL    st_ex 00001000 MSFT 0100000D)
ACPI: APIC 000fd61c 0005C (v01 DELL    GX240   00000008 ASL 00000061)
ACPI: BOOT 000fd678 00028 (v01 DELL    GX240   00000008 ASL 00000061)
ACPI: Local APIC address 0xfee00000
0MB HIGHMEM available.
511MB LOWMEM available.
   mapped low ram: 0 - 1ff77000
   low ram: 0 - 1ff77000
   node 0 low ram: 00000000 - 1ff77000
   node 0 bootmap 00002000 - 00005ff0
(9 early reservations) ==> bootmem [0000000000 - 001ff77000]
   #0 [0000000000 - 0000001000]   BIOS data page ==> [0000000000 - 
0000001000]
   #1 [0000001000 - 0000002000]    EX TRAMPOLINE ==> [0000001000 - 
0000002000]
   #2 [0000006000 - 0000007000]       TRAMPOLINE ==> [0000006000 - 
0000007000]
   #3 [0000400000 - 0000c6f610]    TEXT DATA BSS ==> [0000400000 - 
0000c6f610]
   #4 [001f211000 - 001ff66d6a]          RAMDISK ==> [001f211000 - 
001ff66d6a]
   #5 [000009fc00 - 0000100000]    BIOS reserved ==> [000009fc00 - 
0000100000]
   #6 [0000c70000 - 0000c78174]              BRK ==> [0000c70000 - 
0000c78174]
   #7 [0000007000 - 000000b000]          PGTABLE ==> [0000007000 - 
000000b000]
   #8 [0000002000 - 0000006000]          BOOTMAP ==> [0000002000 - 
0000006000]
found SMP MP-table at [c00fe710] fe710
Reserving 128MB of memory at 16MB for crashkernel (System RAM: 511MB)
Zone PFN ranges:
   DMA      0x00000001 -> 0x00001000
   Normal   0x00001000 -> 0x0001ff77
   HighMem  0x0001ff77 -> 0x0001ff77
Movable zone start PFN for each node
early_node_map[2] active PFN ranges
     0: 0x00000001 -> 0x000000a0
     0: 0x00000100 -> 0x0001ff77
On node 0 totalpages: 130838
   DMA zone: 32 pages used for memmap
   DMA zone: 0 pages reserved
   DMA zone: 3967 pages, LIFO batch:0
   Normal zone: 991 pages used for memmap
   Normal zone: 125848 pages, LIFO batch:31
Using APIC driver default
ACPI: PM-Timer IO Port: 0x808
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] disabled)
ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 1, version 32, address 0xfec00000, GSI 0-23
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ9 used by override.
Using ACPI (MADT) for SMP configuration information
SMP: Allowing 2 CPUs, 1 hotplug CPUs
nr_irqs_gsi: 24
PM: Registered nosave memory: 00000000000a0000 - 00000000000f0000
PM: Registered nosave memory: 00000000000f0000 - 0000000000100000
Allocating PCI resources starting at 20000000 (gap: 20000000:dec00000)
Booting paravirtualized kernel on bare hardware
NR_CPUS:32 nr_cpumask_bits:32 nr_cpu_ids:2 nr_node_ids:1
PERCPU: Embedded 14 pages/cpu @c9600000 s36088 r0 d21256 u1048576
pcpu-alloc: s36088 r0 d21256 u1048576 alloc=1*2097152
pcpu-alloc: [0] 0 1
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 129815
Kernel command line: ro root=/dev/mapper/VolGroup-lv_root nomodeset 
rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD rd_LVM_LV=VolGroup/lv_swap 
SYSFONT=latarcyrheb-sun16 rhgb crashkernel=auto quiet 
rd_LVM_LV=VolGroup/lv_root  KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM 
crashkernel=128M nmi_watchdog=1
PID hash table entries: 2048 (order: 1, 8192 bytes)
Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Initializing CPU#0
Initializing HighMem for node 0 (00000000:00000000)
Memory: 364908k/523740k available (4419k kernel code, 158168k reserved, 
2500k data, 512k init, 0k highmem)
virtual kernel memory layout:
     fixmap  : 0xffad5000 - 0xfffff000   (5288 kB)
     pkmap   : 0xff600000 - 0xff800000   (2048 kB)
     vmalloc : 0xe0777000 - 0xff5fe000   ( 494 MB)
     lowmem  : 0xc0000000 - 0xdff77000   ( 511 MB)
       .init : 0xc0ac3000 - 0xc0b43000   ( 512 kB)
       .data : 0xc0850ea8 - 0xc0ac2008   (2500 kB)
       .text : 0xc0400000 - 0xc0850ea8   (4419 kB)
Checking if this processor honours the WP bit even in supervisor mode...Ok.
Hierarchical RCU implementation.
NR_IRQS:2304 nr_irqs:424
Console: colour VGA+ 80x25
console [tty0] enabled
allocated 1048576 bytes of page_cgroup
please try 'cgroup_disable=memory' option if you don't want memory cgroups
Fast TSC calibration using PIT
Detected 1495.192 MHz processor.
Calibrating delay loop (skipped), value calculated using timer 
frequency.. 2990.38 BogoMIPS (lpj=1495192)
pid_max: default: 32768 minimum: 301
Security Framework initialized
SELinux:  Initializing.
SELinux:  Starting in permissive mode
Mount-cache hash table entries: 512
Initializing cgroup subsys ns
Initializing cgroup subsys cpuacct
Initializing cgroup subsys memory
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
Initializing cgroup subsys net_cls
Initializing cgroup subsys blkio
Initializing cgroup subsys perf_event
Initializing cgroup subsys net_prio
CPU0: Hyper-Threading is disabled
mce: CPU supports 4 MCE banks
CPU0: Thermal monitoring enabled (TM1)
SMP alternatives: switching to UP code
ACPI: Core revision 20090903
Enabling APIC mode:  Flat.  Using 1 I/O APICs
..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
CPU0: Intel(R) Pentium(R) 4 CPU 1.50GHz stepping 02
Performance Events: Netburst events, Netburst P4/Xeon PMU driver.
... version:                0
... bit width:              40
... generic registers:      18
... value mask:             000000ffffffffff
... max period:             0000007fffffffff
... fixed-purpose events:   0
... event mask:             000000000003ffff
NMI watchdog enabled, takes one hw-pmu counter.
Brought up 1 CPUs
Total of 1 processors activated (2990.38 BogoMIPS).
sizeof(vma)=100 bytes
sizeof(page)=32 bytes
sizeof(inode)=352 bytes
sizeof(dentry)=132 bytes
sizeof(ext3inode)=508 bytes
sizeof(buffer_head)=56 bytes
sizeof(skbuff)=184 bytes
sizeof(task_struct)=1348 bytes
devtmpfs: initialized
PM: Registering ACPI NVS region at 1ff77000 (8192 bytes)
regulator: core version 0.5
NET: Registered protocol family 16
ACPI: bus type pci registered
PCI: PCI BIOS revision 2.10 entry at 0xfbe2e, last bus=2
PCI: Using configuration type 1 for base access
bio: create slab <bio-0> at 0
ACPI: EC: Look up EC in DSDT
ACPI: Interpreter enabled
ACPI: (supports S0 S1 S3 S4 S5)
ACPI: Using IOAPIC for interrupt routing
ACPI: No dock devices found.
HEST: Table not found.
PCI: Ignoring host bridge windows from ACPI; if necessary, use 
"pci=use_crs" and report a bug
ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
pci_root PNP0A03:00: host bridge window [io  0x0000-0x0cf7] (ignored)
pci_root PNP0A03:00: host bridge window [io  0x0d00-0xffff] (ignored)
pci_root PNP0A03:00: host bridge window [mem 0x000a0000-0x000bffff] 
(ignored)
pci_root PNP0A03:00: host bridge window [mem 0x000c8000-0x000dffff] 
(ignored)
pci_root PNP0A03:00: host bridge window [mem 0xc0000000-0xffdfffff] 
(ignored)
pci 0000:00:00.0: reg 10: [mem 0xf0000000-0xf3ffffff pref]
pci 0000:00:1f.0: quirk: [io  0x0800-0x087f] claimed by ICH4 ACPI/GPIO/TCO
pci 0000:00:1f.0: quirk: [io  0x0880-0x08bf] claimed by ICH4 GPIO
pci 0000:00:1f.1: reg 20: [io  0xffa0-0xffaf]
pci 0000:00:1f.2: reg 20: [io  0xff80-0xff9f]
pci 0000:00:1f.3: reg 20: [io  0xdcd0-0xdcdf]
pci 0000:00:1f.4: reg 20: [io  0xff60-0xff7f]
pci 0000:00:1f.5: reg 10: [io  0xd800-0xd8ff]
pci 0000:00:1f.5: reg 14: [io  0xdc40-0xdc7f]
pci 0000:01:00.0: reg 10: [mem 0xfc000000-0xfcffffff]
pci 0000:01:00.0: reg 14: [mem 0xe0000000-0xefffffff pref]
pci 0000:01:00.0: reg 18: [mem 0xfb000000-0xfbffffff]
pci 0000:01:00.0: reg 30: [mem 0x80000000-0x8001ffff pref]
pci 0000:00:01.0: PCI bridge to [bus 01-01]
pci 0000:00:01.0:   bridge window [io  0xf000-0x0000] (disabled)
pci 0000:00:01.0:   bridge window [mem 0xfb000000-0xfdffffff]
pci 0000:00:01.0:   bridge window [mem 0xe0000000-0xefffffff pref]
pci 0000:02:0c.0: reg 10: [io  0xec80-0xecff]
pci 0000:02:0c.0: reg 14: [mem 0xfe1ffc00-0xfe1ffc7f]
pci 0000:02:0c.0: reg 30: [mem 0xfe200000-0xfe21ffff pref]
pci 0000:02:0c.0: supports D1 D2
pci 0000:02:0c.0: PME# supported from D0 D1 D2 D3hot D3cold
pci 0000:02:0c.0: PME# disabled
pci 0000:00:1e.0: PCI bridge to [bus 02-02] (subtractive decode)
pci 0000:00:1e.0:   bridge window [io  0xe000-0xefff]
pci 0000:00:1e.0:   bridge window [mem 0xfe100000-0xfe2fffff]
pci 0000:00:1e.0:   bridge window [mem 0xfff00000-0x000fffff pref] 
(disabled)
pci 0000:00:1e.0:   bridge window [io  0x0000-0xffff] (subtractive decode)
pci 0000:00:1e.0:   bridge window [mem 0x00000000-0xfffffffff] 
(subtractive decode)
pci_bus 0000:00: on NUMA node 0
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PCI1._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 9 10 *11 12 15)
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 9 *10 11 12 15)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 7 9 10 *11 12 15)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 7 9 10 *11 12 15)
ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 9 10 11 12 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 9 10 11 12 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 9 10 11 12 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 6 7 *9 10 11 12 15)
vgaarb: device added: PCI:0000:01:00.0,decodes=io+mem,owns=io+mem,locks=none
vgaarb: loaded
vgaarb: bridge control possible 0000:01:00.0
SCSI subsystem initialized
libata version 3.00 loaded.
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
PCI: Using ACPI for IRQ routing
PCI: old code would have set cacheline size to 128 bytes, but 
clflush_size = 64
PCI: pci_cache_line_size set to 64 bytes
NetLabel: Initializing
NetLabel:  domain hash size = 128
NetLabel:  protocols = UNLABELED CIPSOv4
NetLabel:  unlabeled traffic allowed by default
Switching to clocksource jiffies
pnp: PnP ACPI init
ACPI: bus type pnp registered
pnp 00:00: [mem 0x00000000-0x0009ffff]
pnp 00:00: [mem 0x00100000-0x00ffffff]
pnp 00:00: [mem 0x01000000-0x1ff76fff]
pnp 00:00: [mem 0x000f0000-0x000fffff]
pnp 00:00: [mem 0xfec00000-0xfec0ffff]
pnp 00:00: [mem 0xfee00000-0xfee0ffff]
pnp 00:00: [mem 0xffb00000-0xffbfffff]
pnp 00:00: [mem 0xffc00000-0xffffffff]
pnp 00:00: Plug and Play ACPI device, IDs PNP0c01 (active)
pnp 00:01: [io  0x0cf8-0x0cff]
pnp 00:01: Plug and Play ACPI device, IDs PNP0a03 (active)
pnp 00:02: [io  0x0080-0x009f]
pnp 00:02: [io  0x0000-0x001f]
pnp 00:02: [io  0x00c0-0x00df]
pnp 00:02: [dma 4]
pnp 00:02: Plug and Play ACPI device, IDs PNP0200 (active)
pnp 00:03: [io  0x00f0-0x00ff]
pnp 00:03: [irq 13]
pnp 00:03: Plug and Play ACPI device, IDs PNP0c04 (active)
pnp 00:04: [io  0x0061]
pnp 00:04: Plug and Play ACPI device, IDs PNP0800 (active)
pnp 00:05: [io  0x0070-0x007f]
pnp 00:05: [irq 8]
pnp 00:05: Plug and Play ACPI device, IDs PNP0b00 (active)
pnp 00:06: [io  0x03f0-0x03f5]
pnp 00:06: [io  0x03f7]
pnp 00:06: [irq 6]
pnp 00:06: [dma 2]
pnp 00:06: Plug and Play ACPI device, IDs PNP0700 (active)
pnp 00:07: [io  0x03f8-0x03ff]
pnp 00:07: [irq 4]
pnp 00:07: Plug and Play ACPI device, IDs PNP0501 (active)
pnp 00:08: [io  0x02f8-0x02ff]
pnp 00:08: [irq 3]
pnp 00:08: Plug and Play ACPI device, IDs PNP0501 (active)
pnp 00:09: [io  0x0378-0x037f]
pnp 00:09: [io  0x0778-0x077f]
pnp 00:09: [irq 7]
pnp 00:09: [dma 0 disabled]
pnp 00:09: Plug and Play ACPI device, IDs PNP0401 (active)
pnp 00:0a: [io  0x0060]
pnp 00:0a: [io  0x0064]
pnp 00:0a: [io  0x0062-0x0063]
pnp 00:0a: [io  0x0065-0x006f]
pnp 00:0a: [io  0x00e0-0x00ef]
pnp 00:0a: [io  0x0800-0x085f]
pnp 00:0a: [io  0x0c00-0x0c7f]
pnp 00:0a: [io  0x0860-0x08ff]
pnp 00:0a: disabling [io  0x0800-0x085f] because it overlaps 
0000:00:1f.0 BAR 13 [io  0x0800-0x087f]
pnp 00:0a: disabling [io  0x0860-0x08ff] because it overlaps 
0000:00:1f.0 BAR 13 [io  0x0800-0x087f]
pnp 00:0a: Plug and Play ACPI device, IDs PNP0c01 (active)
pnp: PnP ACPI: found 11 devices
ACPI: ACPI bus type pnp unregistered
system 00:00: [mem 0x00000000-0x0009ffff] could not be reserved
system 00:00: [mem 0x00100000-0x00ffffff] could not be reserved
system 00:00: [mem 0x01000000-0x1ff76fff] could not be reserved
system 00:00: [mem 0x000f0000-0x000fffff] could not be reserved
system 00:00: [mem 0xfec00000-0xfec0ffff] could not be reserved
system 00:00: [mem 0xfee00000-0xfee0ffff] has been reserved
system 00:00: [mem 0xffb00000-0xffbfffff] has been reserved
system 00:00: [mem 0xffc00000-0xffffffff] has been reserved
system 00:0a: [io  0x0c00-0x0c7f] has been reserved
Switching to clocksource acpi_pm
pci 0000:01:00.0: no compatible bridge window for [mem 
0x80000000-0x8001ffff pref]
pci 0000:02:0c.0: address space collision: [mem 0xfe200000-0xfe21ffff 
pref] conflicts with PCI Bus 0000:02 [mem 0xfe100000-0xfe2fffff]
PCI: max bus depth: 1 pci_try_num: 2
pci 0000:00:1e.0: BAR 15: assigned [mem 0x20000000-0x200fffff pref]
pci 0000:01:00.0: BAR 6: assigned [mem 0xfd000000-0xfd01ffff pref]
pci 0000:00:01.0: PCI bridge to [bus 01-01]
pci 0000:00:01.0: PCI bridge to [bus 01-01]
pci 0000:00:01.0:   bridge window [io  disabled]
pci 0000:00:01.0:   bridge window [mem 0xfb000000-0xfdffffff]
pci 0000:00:01.0:   bridge window [mem 0xe0000000-0xefffffff pref]
pci 0000:02:0c.0: BAR 6: assigned [mem 0x20000000-0x2001ffff pref]
pci 0000:00:1e.0: PCI bridge to [bus 02-02]
pci 0000:00:1e.0: PCI bridge to [bus 02-02]
pci 0000:00:1e.0:   bridge window [io  0xe000-0xefff]
pci 0000:00:1e.0:   bridge window [mem 0xfe100000-0xfe2fffff]
pci 0000:00:1e.0:   bridge window [mem 0x20000000-0x200fffff pref]
pci 0000:00:1e.0: setting latency timer to 64
pci_bus 0000:00: resource 0 [io  0x0000-0xffff]
pci_bus 0000:00: resource 1 [mem 0x00000000-0xfffffffff]
pci_bus 0000:01: resource 1 [mem 0xfb000000-0xfdffffff]
pci_bus 0000:01: resource 2 [mem 0xe0000000-0xefffffff pref]
pci_bus 0000:02: resource 0 [io  0xe000-0xefff]
pci_bus 0000:02: resource 1 [mem 0xfe100000-0xfe2fffff]
pci_bus 0000:02: resource 2 [mem 0x20000000-0x200fffff pref]
pci_bus 0000:02: resource 4 [io  0x0000-0xffff]
pci_bus 0000:02: resource 5 [mem 0x00000000-0xfffffffff]
NET: Registered protocol family 2
IP route cache hash table entries: 4096 (order: 2, 16384 bytes)
TCP established hash table entries: 16384 (order: 5, 131072 bytes)
TCP bind hash table entries: 16384 (order: 5, 131072 bytes)
TCP: Hash tables configured (established 16384 bind 16384)
TCP reno registered
NET: Registered protocol family 1
   alloc irq_desc for 19 on node -1
   alloc kstat_irqs on node -1
pci 0000:00:1f.2: PCI INT D -> GSI 19 (level, low) -> IRQ 19
pci 0000:00:1f.2: PCI INT D disabled
   alloc irq_desc for 23 on node -1
   alloc kstat_irqs on node -1
pci 0000:00:1f.4: PCI INT C -> GSI 23 (level, low) -> IRQ 23
pci 0000:00:1f.4: PCI INT C disabled
pci 0000:01:00.0: Boot video device
Trying to unpack rootfs image as initramfs...
Freeing initrd memory: 13655k freed
Simple Boot Flag at 0x7a set to 0x1
apm: BIOS version 1.2 Flags 0x03 (Driver version 1.16ac)
apm: overridden by ACPI.
audit: initializing netlink socket (disabled)
type=2000 audit(1369944362.104:1): initialized
HugeTLB registered 2 MB page size, pre-allocated 0 pages
VFS: Disk quotas dquot_6.5.2
Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
msgmni has been set to 739
SELinux:  Registering netfilter hooks
alg: No test for stdrng (krng)
ksign: Installing public key data
Loading keyring
- Added public key 834D293187AE1760
- User ID: CentOS (Kernel Module GPG key)
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered (default)
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
pciehp: PCI Express Hot Plug Controller Driver version: 0.4
acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
ipmi message handler version 39.2
IPMI System Interface driver.
ipmi_si: Adding default-specified kcs state machine
ipmi_si: Trying default-specified kcs state machine at i/o address 
0xca2, slave address 0x0, irq 0
ipmi_si: Interface detection failed
ipmi_si: Adding default-specified smic state machine
ipmi_si: Trying default-specified smic state machine at i/o address 
0xca9, slave address 0x0, irq 0
ipmi_si: Interface detection failed
ipmi_si: Adding default-specified bt state machine
ipmi_si: Trying default-specified bt state machine at i/o address 0xe4, 
slave address 0x0, irq 0
ipmi_si: Interface detection failed
ipmi_si: Unable to find any System Interface(s)
input: Power Button as 
/devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input0
ACPI: Power Button [VBTN]
input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input1
ACPI: Power Button [PWRF]
ACPI: acpi_idle registered with cpuidle
ERST: Table is not found!
GHES: HEST is not enabled!
isapnp: Scanning for PnP cards...
isapnp: No Plug & Play device found
Non-volatile memory driver v1.3
Linux agpgart interface v0.103
agpgart-intel 0000:00:00.0: Intel i845 Chipset
agpgart-intel 0000:00:00.0: AGP aperture is 64M @ 0xf0000000
crash memory driver: version 1.1
Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
00:07: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
00:08: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
brd: module loaded
loop: module loaded
input: Macintosh mouse button emulation as /devices/virtual/input/input2
Fixed MDIO Bus: probed
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
uhci_hcd: USB Universal Host Controller Interface driver
uhci_hcd 0000:00:1f.2: PCI INT D -> GSI 19 (level, low) -> IRQ 19
uhci_hcd 0000:00:1f.2: setting latency timer to 64
uhci_hcd 0000:00:1f.2: UHCI Host Controller
uhci_hcd 0000:00:1f.2: new USB bus registered, assigned bus number 1
uhci_hcd 0000:00:1f.2: irq 19, io base 0x0000ff80
usb usb1: New USB device found, idVendor=1d6b, idProduct=0001
usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb1: Product: UHCI Host Controller
usb usb1: Manufacturer: Linux 2.6.32-358.6.2.el6.i686 uhci_hcd
usb usb1: SerialNumber: 0000:00:1f.2
usb usb1: configuration #1 chosen from 1 choice
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 2 ports detected
uhci_hcd 0000:00:1f.4: PCI INT C -> GSI 23 (level, low) -> IRQ 23
uhci_hcd 0000:00:1f.4: setting latency timer to 64
uhci_hcd 0000:00:1f.4: UHCI Host Controller
uhci_hcd 0000:00:1f.4: new USB bus registered, assigned bus number 2
uhci_hcd 0000:00:1f.4: irq 23, io base 0x0000ff60
usb usb2: New USB device found, idVendor=1d6b, idProduct=0001
usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb2: Product: UHCI Host Controller
usb usb2: Manufacturer: Linux 2.6.32-358.6.2.el6.i686 uhci_hcd
usb usb2: SerialNumber: 0000:00:1f.4
usb usb2: configuration #1 chosen from 1 choice
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 2 ports detected
PNP: No PS/2 controller found. Probing ports directly.
serio: i8042 KBD port at 0x60,0x64 irq 1
serio: i8042 AUX port at 0x60,0x64 irq 12
mice: PS/2 mouse device common for all mice
rtc_cmos 00:05: RTC can wake from S4
rtc_cmos 00:05: rtc core: registered rtc_cmos as rtc0
rtc0: alarms up to one day, 242 bytes nvram
cpuidle: using governor ladder
cpuidle: using governor menu
EFI Variables Facility v0.08 2004-May-17
usbcore: registered new interface driver hiddev
usbcore: registered new interface driver usbhid
usbhid: v2.6:USB HID core driver
TCP cubic registered
Initializing XFRM netlink socket
NET: Registered protocol family 17
Using IPI No-Shortcut mode
registered taskstats version 1
rtc_cmos 00:05: setting system clock to 2013-05-30 20:06:03 UTC (1369944363)
Initalizing network drop monitor service
Freeing unused kernel memory: 512k freed
Write protecting the kernel text: 4420k
Write protecting the kernel read-only data: 1924k
dracut: dracut-004-303.el6
Refined TSC clocksource calibration: 1495.148 MHz.
Switching to clocksource tsc
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.23.6-ioctl (2012-07-25) initialised: 
dm-devel at redhat.com
udev: starting version 147
ACPI: WMI: Mapper loaded
[drm] Initialized drm 1.1.0 20060810
dracut: Starting plymouth daemon
ata_piix 0000:00:1f.1: version 2.13
ata_piix 0000:00:1f.1: setting latency timer to 64
scsi0 : ata_piix
scsi1 : ata_piix
ata1: PATA max UDMA/100 cmd 0x1f0 ctl 0x3f6 bmdma 0xffa0 irq 14
ata2: PATA max UDMA/100 cmd 0x170 ctl 0x376 bmdma 0xffa8 irq 15
ata1.00: ATA-7: ST3500630A, 3.AAE, max UDMA/100
ata1.00: 976773168 sectors, multi 8: LBA48
ata2.00: ATA-7: ST3500630A, 3.AAF, max UDMA/100
ata2.00: 976773168 sectors, multi 8: LBA48
ata2.00: limited to UDMA/33 due to 40-wire cable
ata1.00: configured for UDMA/100
scsi 0:0:0:0: Direct-Access     ATA      ST3500630A       3.AA PQ: 0 ANSI: 5
ata2.00: configured for UDMA/33
scsi 1:0:0:0: Direct-Access     ATA      ST3500630A       3.AA PQ: 0 ANSI: 5
sd 0:0:0:0: [sda] 976773168 512-byte logical blocks: (500 GB/465 GiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't 
support DPO or FUA
  sda:
sd 1:0:0:0: [sdb] 976773168 512-byte logical blocks: (500 GB/465 GiB)
sd 1:0:0:0: [sdb] Write Protect is off
sd 1:0:0:0: [sdb] Mode Sense: 00 3a 00 00
sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't 
support DPO or FUA
  sdb: sda1 sda2
sd 0:0:0:0: [sda] Attached SCSI disk
  sdb1 sdb2 sdb3
sd 1:0:0:0: [sdb] Attached SCSI disk
dracut: Scanning devices sda2  for LVM logical volumes VolGroup/lv_swap 
VolGroup/lv_root
dracut: inactive '/dev/VolGroup/lv_root' [50.00 GiB] inherit
dracut: inactive '/dev/VolGroup/lv_home' [414.27 GiB] inherit
dracut: inactive '/dev/VolGroup/lv_swap' [1.00 GiB] inherit
EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts:
dracut: Mounted root filesystem /dev/mapper/VolGroup-lv_root
SELinux:  Disabled at runtime.
SELinux:  Unregistering netfilter hooks
type=1404 audit(1369944367.063:2): selinux=0 auid=4294967295 ses=4294967295
dracut:
dracut: Switching root
udev: starting version 147
intel_rng: FWH not detected
iTCO_vendor_support: vendor-support=0
iTCO_wdt: Intel TCO WatchDog Timer Driver v1.07rh
iTCO_wdt: Found a ICH2 TCO device (Version=1, TCOBASE=0x0860)
iTCO_wdt: initialized. heartbeat=30 sec (nowayout=0)
sd 0:0:0:0: Attached scsi generic sg0 type 0
sd 1:0:0:0: Attached scsi generic sg1 type 0
   alloc irq_desc for 17 on node -1
   alloc kstat_irqs on node -1
i801_smbus 0000:00:1f.3: PCI INT B -> GSI 17 (level, low) -> IRQ 17
Intel ICH 0000:00:1f.5: PCI INT B -> GSI 17 (level, low) -> IRQ 17
Intel ICH 0000:00:1f.5: setting latency timer to 64
intel8x0_measure_ac97_clock: measured 50922 usecs (4 samples)
intel8x0: measured clock 78 rejected
intel8x0_measure_ac97_clock: measured 50946 usecs (2852 samples)
intel8x0: clocking to 41157
   alloc irq_desc for 18 on node -1
   alloc kstat_irqs on node -1
3c59x 0000:02:0c.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
3c59x: Donald Becker and others.
0000:02:0c.0: 3Com PCI 3c905C Tornado at e0ec0c00.
dcdbas dcdbas: Dell Systems Management Base Driver (version 5.6.0-3.2)
parport_pc 00:09: reported by Plug and Play ACPI
parport0: PC-style at 0x378 (0x778), irq 7 [PCSPP,TRISTATE]
ppdev: user-space parallel port driver
EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts:
EXT4-fs (dm-2): mounted filesystem with ordered data mode. Opts:
EXT4-fs (sdb2): warning: checktime reached, running e2fsck is recommended
EXT4-fs (sdb2): mounted filesystem with ordered data mode. Opts:
EXT4-fs (sdb1): warning: checktime reached, running e2fsck is recommended
EXT4-fs (sdb1): mounted filesystem with ordered data mode. Opts:
EXT4-fs (sdb3): warning: checktime reached, running e2fsck is recommended
EXT4-fs (sdb3): mounted filesystem with ordered data mode. Opts:
Adding 1048568k swap on /dev/mapper/VolGroup-lv_swap.  Priority:-1 
extents:1 across:1048568k
NET: Registered protocol family 10
lo: Disabled Privacy Extensions
ip6_tables: (C) 2000-2006 Netfilter Core Team
nf_conntrack version 0.5.0 (5927 buckets, 23708 max)
ip_tables: (C) 2000-2006 Netfilter Core Team
eth0:  setting full-duplex.
eth0: no IPv6 routers present
SysRq : HELP : loglevel(0-9) reBoot Crash terminate-all-tasks(E) 
memory-full-oom-kill(F) kill-all-tasks(I) thaw-filesystems(J) saK 
show-backtrace-all-active-cpus(L) show-memory-usage(M) 
nice-all-RT-tasks(N) powerOff show-registers(P) show-all-timers(Q) unRaw 
Sync show-task-states(T) Unmount show-blocked-tasks(W) 
dump-ftrace-buffer(Z)
SysRq : HELP : loglevel(0-9) reBoot Crash terminate-all-tasks(E) 
memory-full-oom-kill(F) kill-all-tasks(I) thaw-filesystems(J) saK 
show-backtrace-all-active-cpus(L) show-memory-usage(M) 
nice-all-RT-tasks(N) powerOff show-registers(P) show-all-timers(Q) unRaw 
Sync show-task-states(T) Unmount show-blocked-tasks(W) 
dump-ftrace-buffer(Z)