[CentOS] kernel BUG at fs/bio.c (kernel 2.6.9-34.0.2.EL)

Thu Jul 27 09:53:43 UTC 2006
J.J.Garcia <stigmatedbrain at gmail.com>

Hi folks,

Backing up a host as noted below im getting over and over again the same issue
on actual kernel, details:

- Backup host has a pci/usb2.0 card, with a 250gb usb 2.0 disk attached,
- Celeron 900 family
- Remote host, local console from boot linux rescue:
  # dd if=/dev/hda conv=sync,noerror | ssh -c blowfish root at 192.168.0.6 "dd
of=/media/backup/vol-hda.raw.img"
- Backup host (192.168.0.6), kernel bug from netdump - here spoolbox host- as
netdump-server:

[root at spoolbox /]# cat /var/crash/192.168.0.6-2006-07-27-10:46/log
------------[ cut here ]------------
kernel BUG at fs/bio.c:99!
invalid operand: 0000 [#1]
Modules linked in: netconsole netdump vfat fat imm eeprom i2c_viapro nfsd
exportfs lockd nfs_acl lp autofs4 via686a i2c_sensor i2c_isa i2c_dev i2c_core
sunrpc ip_nat_ftp iptable_nat ip_conntrack_ftp ipt_REJECT ipt_state ip_conntrack
iptable_filter ip_tables dm_multipath sr_mod sd_mod usb_storage scsi_mod
ohci_hcd ehci_hcd parport_pc parport snd_via82xx snd_ac97_codec snd_pcm_oss
snd_mixer_oss snd_pcm snd_timer snd_page_alloc snd_mpu401_uart snd_rawmidi
snd_seq_device snd soundcore 8139too mii floppy dm_snapshot dm_zero dm_mirror
ext3 jbd dm_mod
CPU:    0
EIP:    0060:[<c016dca5>]    Not tainted VLI
EFLAGS: 00010206   (2.6.9-34.0.2.EL)
EIP is at bio_destructor+0x19/0x3a
eax: 00000008   ebx: c2356b00   ecx: caeffd7c   edx: c0361bc0
esi: 00001000   edi: c016d665   ebp: 00000000   esp: c03e6f24
ds: 007b   es: 007b   ss: 0068
Process sshd (pid: 3916, threadinfo=c03e6000 task=c79bc050)
Stack: c2356b00 c016deb4 c016d698 c2356b00 c016ea55 00001000 c2356b00 00000000
       c03e6f6c c0251ce5 c9ab918c 00000000 00000000 0001e000 00002000 c9ab918c
       d71ae088 d50fb7c0 00000001 d841cdd1 00000001 d110fc00 c9ab918c d50fb7c0
Call Trace:
 [<c016deb4>] bio_put+0x27/0x28
 [<c016d698>] end_bio_bh_io_sync+0x33/0x37
 [<c016ea55>] bio_endio+0x4f/0x54
 [<c0251ce5>] __end_that_request_first+0xea/0x1ab
 [<d841cdd1>] scsi_end_request+0x1b/0x174 [scsi_mod]
 [<d841d268>] scsi_io_completion+0x20b/0x417 [scsi_mod]
 [<d841800c>] scsi_finish_command+0xad/0xb1 [scsi_mod]
 [<d8417f31>] scsi_softirq+0xba/0xc2 [scsi_mod]
 [<c0126a9d>] __do_softirq+0x35/0x79
 [<c010934c>] do_softirq+0x46/0x4d
 =======================
 [<c0108910>] do_IRQ+0x2b3/0x2bf
 [<c03115c0>] common_interrupt+0x18/0x20
 [<c031007b>] rwsem_down_read_failed+0x167/0x204
Code: 0b 1a 00 e9 cc c9 ff ff e8 51 0b 1a 00 e9 26 ca ff ff 53 89 c3 8b 40 10 c1
e8 1c 89 c2 c1 e2 04 81 c2 40 1b 36 c0 83 f8 05 7e 08 <0f> 0b 63 00 0a 35 32 c0
8b 43 30 8b 52 0c e8 d8 d0 fd ff 89 d8


- grub.conf (backup host)

title CentOS (2.6.9-34.0.2.EL)
        root (hd0,0)
        kernel /vmlinuz-2.6.9-34.0.2.EL ro root=/dev/VolGroup00/LogVol00
ACPI=off vga=0x307 selinux=0
        initrd /initrd-2.6.9-34.0.2.EL.img

- Backup host dmesg (192.168.0.6):

[root at fattybox ~]# cat dmesg.txt
Linux version 2.6.9-34.0.2.EL (buildcentos at v20z-i386) (gcc version 3.4.5
20051201 (Red Hat 3.4.5-2)) #1 Fri Jul 7 19:24:57 CDT 2006
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
 BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 0000000017800000 (usable)
 BIOS-e820: 00000000ffff0000 - 0000000100000000 (reserved)
0MB HIGHMEM available.
376MB LOWMEM available.
Using x86 segment limits to approximate NX protection
zapping low mappings.
On node 0 totalpages: 96256
  DMA zone: 4096 pages, LIFO batch:1
  Normal zone: 92160 pages, LIFO batch:16
  HighMem zone: 0 pages, LIFO batch:1
DMI 2.2 present.
ACPI: Unable to locate RSDP
Built 1 zonelists
Kernel command line: ro root=/dev/VolGroup00/LogVol00 ACPI=off vga=0x307 selinux=0
Initializing CPU#0
CPU 0 irqstacks, hard=c03e7000 soft=c03e6000
PID hash table entries: 2048 (order: 11, 32768 bytes)
Detected 898.979 MHz processor.
Using tsc for high-res timesource
Console: colour dummy device 80x25
Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
Memory: 376504k/385024k available (2118k kernel code, 7824k reserved, 668k data,
144k init, 0k highmem)
Calibrating delay using timer specific routine.. 1800.03 BogoMIPS (lpj=900018)
Security Scaffold v1.0.0 initialized
SELinux:  Disabled at boot.
Capability LSM initialized
Mount-cache hash table entries: 512 (order: 0, 4096 bytes)
CPU: After generic identify, caps: 0383f9ff 00000000 00000000 00000000
CPU: After vendor identify, caps:  0383f9ff 00000000 00000000 00000000
CPU: L1 I cache: 16K, L1 D cache: 16K
CPU: L2 cache: 128K
CPU: After all inits, caps:        0383f1ff 00000000 00000000 00000040
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#0.
CPU: Intel Celeron (Coppermine) stepping 0a
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Checking 'hlt' instruction... OK.
checking if image is initramfs... it is
Freeing initrd memory: 984k freed
NET: Registered protocol family 16
PCI: PCI BIOS revision 2.10 entry at 0xfb4b0, last bus=1
PCI: Using configuration type 1
mtrr: v2.0 (20020519)
ACPI: Subsystem revision 20040816
ACPI: Interpreter disabled.
Linux Plug and Play Support v0.97 (c) Adam Belay
usbcore: registered new driver usbfs
usbcore: registered new driver hub
PCI: Probing PCI hardware
PCI: Probing PCI hardware (bus 00)
PCI: Using IRQ router VIA [1106/0686] at 0000:00:07.0
apm: BIOS version 1.2 Flags 0x07 (Driver version 1.16ac)
audit: initializing netlink socket (disabled)
audit(1153991211.456:1): initialized
Total HugeTLB memory allocated, 0
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
Initializing Cryptographic API
ksign: Installing public key data
Loading keyring
- Added public key 1D846148DE22BD48
- User ID: CentOS (Kernel Module GPG key)
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
vesafb: framebuffer at 0xec000000, mapped to 0xd8080000, size 2560k
vesafb: mode is 1280x1024x8, linelength=1280, pages=5
vesafb: protected mode interface info at c000:7e40
vesafb: scrolling: redraw
vesafb: Pseudocolor: size=6:6:6:6, shift=0:0:0:0
Console: switching to colour frame buffer device 160x64
fb0: VESA VGA frame buffer device
Real Time Clock Driver v1.12
Linux agpgart interface v0.100 (c) Dave Jones
agpgart: Detected VIA Apollo ProMedia/PLE133Ta chipset
agpgart: Maximum main memory to use for agp memory: 314M
agpgart: AGP aperture is 64M @ 0xe8000000
serio: i8042 AUX port at 0x60,0x64 irq 12
serio: i8042 KBD port at 0x60,0x64 irq 1
Serial: 8250/16550 driver $Revision: 1.90 $ 8 ports, IRQ sharing enabled
ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
RAMDISK driver initialized: 16 RAM disks of 16384K size 1024 blocksize
divert: not allocating divert_blk for non-ethernet device lo
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
VP_IDE: IDE controller at PCI slot 0000:00:07.1
VP_IDE: chipset revision 6
VP_IDE: not 100% native mode: will probe irqs later
VP_IDE: VIA vt82c686b (rev 40) IDE UDMA100 controller on pci0000:00:07.1
    ide0: BM-DMA at 0xe000-0xe007, BIOS settings: hda:DMA, hdb:pio
    ide1: BM-DMA at 0xe008-0xe00f, BIOS settings: hdc:DMA, hdd:DMA
Probing IDE interface ide0...
hda: ST36530A, ATA DISK drive
Using cfq io scheduler
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
Probing IDE interface ide1...
hdc: HL-DT-ST GCE-8525B, ATAPI CD/DVD-ROM drive
hdd: IBM-DTLA-307045, ATA DISK drive
ide1 at 0x170-0x177,0x376 on irq 15
Probing IDE interface ide2...
Probing IDE interface ide3...
Probing IDE interface ide4...
Probing IDE interface ide5...
hda: max request size: 128KiB
hda: 12594960 sectors (6448 MB) w/448KiB Cache, CHS=13328/15/63, UDMA(33)
hda: cache flushes not supported
 hda: hda1 hda2
hdd: max request size: 128KiB
hdd: 90069840 sectors (46115 MB) w/1916KiB Cache, CHS=65535/16/63, UDMA(100)
hdd: cache flushes not supported
 hdd: hdd1
hdc: ATAPI 52X CD-ROM CD-R/RW drive, 2048kB Cache, UDMA(33)
Uniform CD-ROM driver Revision: 3.20
ide-floppy driver 0.99.newide
usbcore: registered new driver hiddev
usbcore: registered new driver usbhid
drivers/usb/input/hid-core.c: v2.0:USB HID core driver
mice: PS/2 mouse device common for all mice
input: AT Translated Set 2 keyboard on isa0060/serio0
input: ImExPS/2 Logitech Explorer Mouse on isa0060/serio1
md: md driver 0.90.0 MAX_MD_DEVS=256, MD_SB_DISKS=27
NET: Registered protocol family 2
IP: routing cache hash table of 1024 buckets, 32Kbytes
TCP: Hash tables configured (established 32768 bind 9362)
Initializing IPsec netlink socket
NET: Registered protocol family 1
NET: Registered protocol family 17
Freeing unused kernel memory: 144k freed
device-mapper: 4.5.0-ioctl (2005-10-04) initialised: dm-devel at redhat.com
cdrom: open failed.
EXT3-fs: INFO: recovery required on readonly filesystem.
EXT3-fs: write access will be enabled during recovery.
kjournald starting.  Commit interval 5 seconds
EXT3-fs: recovery complete.
EXT3-fs: mounted filesystem with ordered data mode.
inserting floppy driver for 2.6.9-34.0.2.EL
FDC 0 is a post-1991 82077
8139too Fast Ethernet driver 0.9.27
PCI: Found IRQ 11 for device 0000:00:0d.0
PCI: Sharing IRQ 11 with 0000:00:09.0
divert: allocating divert_blk for eth0
eth0: RealTek RTL8139 at 0xe700, 00:01:29:fc:25:b8, IRQ 11
eth0:  Identified 8139 chip type 'RTL-8100B/8139D'
ip_tables: (C) 2000-2002 Netfilter core team
eth0: link up, 100Mbps, full-duplex, lpa 0x45E1
PCI: Found IRQ 10 for device 0000:00:07.5
PCI: Sharing IRQ 10 with 0000:00:09.2
PCI: Setting latency timer of device 0000:00:07.5 to 64
ip_tables: (C) 2000-2002 Netfilter core team
parport0: PC-style at 0x378 [PCSPP,EPP]
parport0: device reported incorrect length field (61, should be 62)
parport0 (addr 0): SCSI adapter, IMG VP1
parport_pc: Via 686A parallel port: io=0x378
pciehp: acpi_pciehprm:get_device PCI ROOT HID fail=0x1001
PCI: Found IRQ 10 for device 0000:00:09.2
PCI: Sharing IRQ 10 with 0000:00:07.5
ehci_hcd 0000:00:09.2: EHCI Host Controller
ehci_hcd 0000:00:09.2: irq 10, pci mem d807c000
ehci_hcd 0000:00:09.2: new USB bus registered, assigned bus number 1
ehci_hcd 0000:00:09.2: USB 2.0 enabled, EHCI 1.00, driver 2004-May-10
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 5 ports detected
ohci_hcd: 2004 Feb 02 USB 1.1 'Open' Host Controller (OHCI) Driver (PCI)
PCI: Found IRQ 11 for device 0000:00:09.0
PCI: Sharing IRQ 11 with 0000:00:0d.0
ohci_hcd 0000:00:09.0: OHCI Host Controller
ohci_hcd 0000:00:09.0: irq 11, pci mem d807e000
ohci_hcd 0000:00:09.0: new USB bus registered, assigned bus number 2
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 3 ports detected
PCI: Found IRQ 5 for device 0000:00:09.1
ohci_hcd 0000:00:09.1: OHCI Host Controller
ohci_hcd 0000:00:09.1: irq 5, pci mem d836e000
ohci_hcd 0000:00:09.1: new USB bus registered, assigned bus number 3
usb 1-2: new high speed USB device using address 2
hub 3-0:1.0: USB hub found
hub 3-0:1.0: 2 ports detected
md: Autodetecting RAID arrays.
md: autorun ...
md: ... autorun DONE.
usb 1-3: new high speed USB device using address 3
SCSI subsystem initialized
Initializing USB Mass Storage driver...
scsi0 : SCSI emulation for USB Mass Storage devices
  Vendor: Maxtor    Model: 3200              Rev: 0341
  Type:   Direct-Access                      ANSI SCSI revision: 02
USB Mass Storage device found at 2
scsi1 : SCSI emulation for USB Mass Storage devices
  Vendor: Genesys   Model: USB to IDE Disk   Rev: 0002
  Type:   Direct-Access                      ANSI SCSI revision: 02
usb 1-4: new high speed USB device using address 4
USB Mass Storage device found at 3
usbcore: registered new driver usb-storage
USB Mass Storage support registered.
SCSI device sda: 586114704 512-byte hdwr sectors (300091 MB)
sda: assuming drive cache: write through
SCSI device sda: 586114704 512-byte hdwr sectors (300091 MB)
sda: assuming drive cache: write through
 sda:<6>scsi2 : SCSI emulation for USB Mass Storage devices
  Vendor: Maxtor    Model: 6L250R0           Rev: BAH4
  Type:   Direct-Access                      ANSI SCSI revision: 02
 sda1
Attached scsi disk sda at scsi0, channel 0, id 0, lun 0
SCSI device sdb: 78140160 512-byte hdwr sectors (40008 MB)
sdb: test WP failed, assume Write Enabled
sdb: assuming drive cache: write through
SCSI device sdb: 78140160 512-byte hdwr sectors (40008 MB)
sdb: test WP failed, assume Write Enabled
sdb: assuming drive cache: write through
 sdb: sdb1
Attached scsi removable disk sdb at scsi1, channel 0, id 0, lun 0
SCSI device sdc: 490234752 512-byte hdwr sectors (251000 MB)
sdc: assuming drive cache: write through
SCSI device sdc: 490234752 512-byte hdwr sectors (251000 MB)
sdc: assuming drive cache: write through
 sdc: sdc1
Attached scsi disk sdc at scsi2, channel 0, id 0, lun 0
USB Mass Storage device found at 4
usb 1-5: new high speed USB device using address 5
scsi3 : SCSI emulation for USB Mass Storage devices
  Vendor: IOMEGA    Model: CDRW64892EXT3-B   Rev: 6OP1
  Type:   CD-ROM                             ANSI SCSI revision: 02
USB Mass Storage device found at 5
sr0: scsi3-mmc drive: 12x/12x writer cd/rw xa/form2 cdda tray
Attached scsi CD-ROM sr0 at scsi3, channel 0, id 0, lun 0
EXT3 FS on dm-0, internal journal
device-mapper: dm-multipath version 1.0.4 loaded
cdrom: open failed.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on hda1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on hdd1, internal journal
EXT3-fs: recovery complete.
EXT3-fs: mounted filesystem with ordered data mode.
Adding 786424k swap on /dev/VolGroup00/LogVol01.  Priority:-1 extents:1
IA-32 Microcode Update Driver: v1.14 <tigran at veritas.com>
microcode: CPU0 already at revision 0x1 (current=0x1)
microcode: No new microdata for cpu 0
IA-32 Microcode Update Driver v1.14 unregistered
ip_tables: (C) 2000-2002 Netfilter core team
ip_conntrack version 2.1 (3008 buckets, 24064 max) - 356 bytes per conntrack
i2c /dev entries driver
netdump: local port 6666
netdump: local IP 192.168.0.6
netdump: interface eth0
netdump: remote port 6666
netdump: remote IP 192.168.0.5
netdump: remote ethernet address 00:04:e2:54:be:69
netdump: network crash dump enabled
netlog: local port 6664
netlog: local IP 192.168.0.6
netlog: interface eth0
netlog: remote port 6666
netlog: remote IP 192.168.0.5
netlog: remote ethernet address 00:04:e2:54:be:69
netconsole: network logging started
lp0: using parport0 (polling).
lp0: console ready
Installing knfsd (copyright (C) 1996 okir at monad.swb.de).
imm: Version 2.05 (for Linux 2.4.0)
imm: Found device at ID 6, Attempting to use EPP 32 bit
imm: Found device at ID 6, Attempting to use SPP
imm: Communication established at 0x378 with ID 6 using SPP
scsi4 : Iomega VPI2 (imm) interface
  Vendor: IOMEGA    Model: ZIP 250           Rev: H.41
  Type:   Direct-Access                      ANSI SCSI revision: 02
Attached scsi removable disk sdd at scsi4, channel 0, id 6, lun 0
cdrom: This disc doesn't have any tracks I recognize!


Feel free to ask for more detailed information,

TIA

Jose