Hi folks, Backing up a host as noted below im getting over and over again the same issue on actual kernel, details: - Backup host has a pci/usb2.0 card, with a 250gb usb 2.0 disk attached, - Celeron 900 family - Remote host, local console from boot linux rescue: # dd if=/dev/hda conv=sync,noerror | ssh -c blowfish root at 192.168.0.6 "dd of=/media/backup/vol-hda.raw.img" - Backup host (192.168.0.6), kernel bug from netdump - here spoolbox host- as netdump-server: [root at spoolbox /]# cat /var/crash/192.168.0.6-2006-07-27-10:46/log ------------[ cut here ]------------ kernel BUG at fs/bio.c:99! invalid operand: 0000 [#1] Modules linked in: netconsole netdump vfat fat imm eeprom i2c_viapro nfsd exportfs lockd nfs_acl lp autofs4 via686a i2c_sensor i2c_isa i2c_dev i2c_core sunrpc ip_nat_ftp iptable_nat ip_conntrack_ftp ipt_REJECT ipt_state ip_conntrack iptable_filter ip_tables dm_multipath sr_mod sd_mod usb_storage scsi_mod ohci_hcd ehci_hcd parport_pc parport snd_via82xx snd_ac97_codec snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd_page_alloc snd_mpu401_uart snd_rawmidi snd_seq_device snd soundcore 8139too mii floppy dm_snapshot dm_zero dm_mirror ext3 jbd dm_mod CPU: 0 EIP: 0060:[<c016dca5>] Not tainted VLI EFLAGS: 00010206 (2.6.9-34.0.2.EL) EIP is at bio_destructor+0x19/0x3a eax: 00000008 ebx: c2356b00 ecx: caeffd7c edx: c0361bc0 esi: 00001000 edi: c016d665 ebp: 00000000 esp: c03e6f24 ds: 007b es: 007b ss: 0068 Process sshd (pid: 3916, threadinfo=c03e6000 task=c79bc050) Stack: c2356b00 c016deb4 c016d698 c2356b00 c016ea55 00001000 c2356b00 00000000 c03e6f6c c0251ce5 c9ab918c 00000000 00000000 0001e000 00002000 c9ab918c d71ae088 d50fb7c0 00000001 d841cdd1 00000001 d110fc00 c9ab918c d50fb7c0 Call Trace: [<c016deb4>] bio_put+0x27/0x28 [<c016d698>] end_bio_bh_io_sync+0x33/0x37 [<c016ea55>] bio_endio+0x4f/0x54 [<c0251ce5>] __end_that_request_first+0xea/0x1ab [<d841cdd1>] scsi_end_request+0x1b/0x174 [scsi_mod] [<d841d268>] scsi_io_completion+0x20b/0x417 [scsi_mod] [<d841800c>] scsi_finish_command+0xad/0xb1 [scsi_mod] [<d8417f31>] scsi_softirq+0xba/0xc2 [scsi_mod] [<c0126a9d>] __do_softirq+0x35/0x79 [<c010934c>] do_softirq+0x46/0x4d ======================= [<c0108910>] do_IRQ+0x2b3/0x2bf [<c03115c0>] common_interrupt+0x18/0x20 [<c031007b>] rwsem_down_read_failed+0x167/0x204 Code: 0b 1a 00 e9 cc c9 ff ff e8 51 0b 1a 00 e9 26 ca ff ff 53 89 c3 8b 40 10 c1 e8 1c 89 c2 c1 e2 04 81 c2 40 1b 36 c0 83 f8 05 7e 08 <0f> 0b 63 00 0a 35 32 c0 8b 43 30 8b 52 0c e8 d8 d0 fd ff 89 d8 - grub.conf (backup host) title CentOS (2.6.9-34.0.2.EL) root (hd0,0) kernel /vmlinuz-2.6.9-34.0.2.EL ro root=/dev/VolGroup00/LogVol00 ACPI=off vga=0x307 selinux=0 initrd /initrd-2.6.9-34.0.2.EL.img - Backup host dmesg (192.168.0.6): [root at fattybox ~]# cat dmesg.txt Linux version 2.6.9-34.0.2.EL (buildcentos at v20z-i386) (gcc version 3.4.5 20051201 (Red Hat 3.4.5-2)) #1 Fri Jul 7 19:24:57 CDT 2006 BIOS-provided physical RAM map: BIOS-e820: 0000000000000000 - 000000000009fc00 (usable) BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved) BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved) BIOS-e820: 0000000000100000 - 0000000017800000 (usable) BIOS-e820: 00000000ffff0000 - 0000000100000000 (reserved) 0MB HIGHMEM available. 376MB LOWMEM available. Using x86 segment limits to approximate NX protection zapping low mappings. On node 0 totalpages: 96256 DMA zone: 4096 pages, LIFO batch:1 Normal zone: 92160 pages, LIFO batch:16 HighMem zone: 0 pages, LIFO batch:1 DMI 2.2 present. ACPI: Unable to locate RSDP Built 1 zonelists Kernel command line: ro root=/dev/VolGroup00/LogVol00 ACPI=off vga=0x307 selinux=0 Initializing CPU#0 CPU 0 irqstacks, hard=c03e7000 soft=c03e6000 PID hash table entries: 2048 (order: 11, 32768 bytes) Detected 898.979 MHz processor. Using tsc for high-res timesource Console: colour dummy device 80x25 Dentry cache hash table entries: 65536 (order: 6, 262144 bytes) Inode-cache hash table entries: 32768 (order: 5, 131072 bytes) Memory: 376504k/385024k available (2118k kernel code, 7824k reserved, 668k data, 144k init, 0k highmem) Calibrating delay using timer specific routine.. 1800.03 BogoMIPS (lpj=900018) Security Scaffold v1.0.0 initialized SELinux: Disabled at boot. Capability LSM initialized Mount-cache hash table entries: 512 (order: 0, 4096 bytes) CPU: After generic identify, caps: 0383f9ff 00000000 00000000 00000000 CPU: After vendor identify, caps: 0383f9ff 00000000 00000000 00000000 CPU: L1 I cache: 16K, L1 D cache: 16K CPU: L2 cache: 128K CPU: After all inits, caps: 0383f1ff 00000000 00000000 00000040 Intel machine check architecture supported. Intel machine check reporting enabled on CPU#0. CPU: Intel Celeron (Coppermine) stepping 0a Enabling fast FPU save and restore... done. Enabling unmasked SIMD FPU exception support... done. Checking 'hlt' instruction... OK. checking if image is initramfs... it is Freeing initrd memory: 984k freed NET: Registered protocol family 16 PCI: PCI BIOS revision 2.10 entry at 0xfb4b0, last bus=1 PCI: Using configuration type 1 mtrr: v2.0 (20020519) ACPI: Subsystem revision 20040816 ACPI: Interpreter disabled. Linux Plug and Play Support v0.97 (c) Adam Belay usbcore: registered new driver usbfs usbcore: registered new driver hub PCI: Probing PCI hardware PCI: Probing PCI hardware (bus 00) PCI: Using IRQ router VIA [1106/0686] at 0000:00:07.0 apm: BIOS version 1.2 Flags 0x07 (Driver version 1.16ac) audit: initializing netlink socket (disabled) audit(1153991211.456:1): initialized Total HugeTLB memory allocated, 0 VFS: Disk quotas dquot_6.5.1 Dquot-cache hash table entries: 1024 (order 0, 4096 bytes) Initializing Cryptographic API ksign: Installing public key data Loading keyring - Added public key 1D846148DE22BD48 - User ID: CentOS (Kernel Module GPG key) pci_hotplug: PCI Hot Plug PCI Core version: 0.5 vesafb: framebuffer at 0xec000000, mapped to 0xd8080000, size 2560k vesafb: mode is 1280x1024x8, linelength=1280, pages=5 vesafb: protected mode interface info at c000:7e40 vesafb: scrolling: redraw vesafb: Pseudocolor: size=6:6:6:6, shift=0:0:0:0 Console: switching to colour frame buffer device 160x64 fb0: VESA VGA frame buffer device Real Time Clock Driver v1.12 Linux agpgart interface v0.100 (c) Dave Jones agpgart: Detected VIA Apollo ProMedia/PLE133Ta chipset agpgart: Maximum main memory to use for agp memory: 314M agpgart: AGP aperture is 64M @ 0xe8000000 serio: i8042 AUX port at 0x60,0x64 irq 12 serio: i8042 KBD port at 0x60,0x64 irq 1 Serial: 8250/16550 driver $Revision: 1.90 $ 8 ports, IRQ sharing enabled ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A RAMDISK driver initialized: 16 RAM disks of 16384K size 1024 blocksize divert: not allocating divert_blk for non-ethernet device lo Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2 ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx VP_IDE: IDE controller at PCI slot 0000:00:07.1 VP_IDE: chipset revision 6 VP_IDE: not 100% native mode: will probe irqs later VP_IDE: VIA vt82c686b (rev 40) IDE UDMA100 controller on pci0000:00:07.1 ide0: BM-DMA at 0xe000-0xe007, BIOS settings: hda:DMA, hdb:pio ide1: BM-DMA at 0xe008-0xe00f, BIOS settings: hdc:DMA, hdd:DMA Probing IDE interface ide0... hda: ST36530A, ATA DISK drive Using cfq io scheduler ide0 at 0x1f0-0x1f7,0x3f6 on irq 14 Probing IDE interface ide1... hdc: HL-DT-ST GCE-8525B, ATAPI CD/DVD-ROM drive hdd: IBM-DTLA-307045, ATA DISK drive ide1 at 0x170-0x177,0x376 on irq 15 Probing IDE interface ide2... Probing IDE interface ide3... Probing IDE interface ide4... Probing IDE interface ide5... hda: max request size: 128KiB hda: 12594960 sectors (6448 MB) w/448KiB Cache, CHS=13328/15/63, UDMA(33) hda: cache flushes not supported hda: hda1 hda2 hdd: max request size: 128KiB hdd: 90069840 sectors (46115 MB) w/1916KiB Cache, CHS=65535/16/63, UDMA(100) hdd: cache flushes not supported hdd: hdd1 hdc: ATAPI 52X CD-ROM CD-R/RW drive, 2048kB Cache, UDMA(33) Uniform CD-ROM driver Revision: 3.20 ide-floppy driver 0.99.newide usbcore: registered new driver hiddev usbcore: registered new driver usbhid drivers/usb/input/hid-core.c: v2.0:USB HID core driver mice: PS/2 mouse device common for all mice input: AT Translated Set 2 keyboard on isa0060/serio0 input: ImExPS/2 Logitech Explorer Mouse on isa0060/serio1 md: md driver 0.90.0 MAX_MD_DEVS=256, MD_SB_DISKS=27 NET: Registered protocol family 2 IP: routing cache hash table of 1024 buckets, 32Kbytes TCP: Hash tables configured (established 32768 bind 9362) Initializing IPsec netlink socket NET: Registered protocol family 1 NET: Registered protocol family 17 Freeing unused kernel memory: 144k freed device-mapper: 4.5.0-ioctl (2005-10-04) initialised: dm-devel at redhat.com cdrom: open failed. EXT3-fs: INFO: recovery required on readonly filesystem. EXT3-fs: write access will be enabled during recovery. kjournald starting. Commit interval 5 seconds EXT3-fs: recovery complete. EXT3-fs: mounted filesystem with ordered data mode. inserting floppy driver for 2.6.9-34.0.2.EL FDC 0 is a post-1991 82077 8139too Fast Ethernet driver 0.9.27 PCI: Found IRQ 11 for device 0000:00:0d.0 PCI: Sharing IRQ 11 with 0000:00:09.0 divert: allocating divert_blk for eth0 eth0: RealTek RTL8139 at 0xe700, 00:01:29:fc:25:b8, IRQ 11 eth0: Identified 8139 chip type 'RTL-8100B/8139D' ip_tables: (C) 2000-2002 Netfilter core team eth0: link up, 100Mbps, full-duplex, lpa 0x45E1 PCI: Found IRQ 10 for device 0000:00:07.5 PCI: Sharing IRQ 10 with 0000:00:09.2 PCI: Setting latency timer of device 0000:00:07.5 to 64 ip_tables: (C) 2000-2002 Netfilter core team parport0: PC-style at 0x378 [PCSPP,EPP] parport0: device reported incorrect length field (61, should be 62) parport0 (addr 0): SCSI adapter, IMG VP1 parport_pc: Via 686A parallel port: io=0x378 pciehp: acpi_pciehprm:get_device PCI ROOT HID fail=0x1001 PCI: Found IRQ 10 for device 0000:00:09.2 PCI: Sharing IRQ 10 with 0000:00:07.5 ehci_hcd 0000:00:09.2: EHCI Host Controller ehci_hcd 0000:00:09.2: irq 10, pci mem d807c000 ehci_hcd 0000:00:09.2: new USB bus registered, assigned bus number 1 ehci_hcd 0000:00:09.2: USB 2.0 enabled, EHCI 1.00, driver 2004-May-10 hub 1-0:1.0: USB hub found hub 1-0:1.0: 5 ports detected ohci_hcd: 2004 Feb 02 USB 1.1 'Open' Host Controller (OHCI) Driver (PCI) PCI: Found IRQ 11 for device 0000:00:09.0 PCI: Sharing IRQ 11 with 0000:00:0d.0 ohci_hcd 0000:00:09.0: OHCI Host Controller ohci_hcd 0000:00:09.0: irq 11, pci mem d807e000 ohci_hcd 0000:00:09.0: new USB bus registered, assigned bus number 2 hub 2-0:1.0: USB hub found hub 2-0:1.0: 3 ports detected PCI: Found IRQ 5 for device 0000:00:09.1 ohci_hcd 0000:00:09.1: OHCI Host Controller ohci_hcd 0000:00:09.1: irq 5, pci mem d836e000 ohci_hcd 0000:00:09.1: new USB bus registered, assigned bus number 3 usb 1-2: new high speed USB device using address 2 hub 3-0:1.0: USB hub found hub 3-0:1.0: 2 ports detected md: Autodetecting RAID arrays. md: autorun ... md: ... autorun DONE. usb 1-3: new high speed USB device using address 3 SCSI subsystem initialized Initializing USB Mass Storage driver... scsi0 : SCSI emulation for USB Mass Storage devices Vendor: Maxtor Model: 3200 Rev: 0341 Type: Direct-Access ANSI SCSI revision: 02 USB Mass Storage device found at 2 scsi1 : SCSI emulation for USB Mass Storage devices Vendor: Genesys Model: USB to IDE Disk Rev: 0002 Type: Direct-Access ANSI SCSI revision: 02 usb 1-4: new high speed USB device using address 4 USB Mass Storage device found at 3 usbcore: registered new driver usb-storage USB Mass Storage support registered. SCSI device sda: 586114704 512-byte hdwr sectors (300091 MB) sda: assuming drive cache: write through SCSI device sda: 586114704 512-byte hdwr sectors (300091 MB) sda: assuming drive cache: write through sda:<6>scsi2 : SCSI emulation for USB Mass Storage devices Vendor: Maxtor Model: 6L250R0 Rev: BAH4 Type: Direct-Access ANSI SCSI revision: 02 sda1 Attached scsi disk sda at scsi0, channel 0, id 0, lun 0 SCSI device sdb: 78140160 512-byte hdwr sectors (40008 MB) sdb: test WP failed, assume Write Enabled sdb: assuming drive cache: write through SCSI device sdb: 78140160 512-byte hdwr sectors (40008 MB) sdb: test WP failed, assume Write Enabled sdb: assuming drive cache: write through sdb: sdb1 Attached scsi removable disk sdb at scsi1, channel 0, id 0, lun 0 SCSI device sdc: 490234752 512-byte hdwr sectors (251000 MB) sdc: assuming drive cache: write through SCSI device sdc: 490234752 512-byte hdwr sectors (251000 MB) sdc: assuming drive cache: write through sdc: sdc1 Attached scsi disk sdc at scsi2, channel 0, id 0, lun 0 USB Mass Storage device found at 4 usb 1-5: new high speed USB device using address 5 scsi3 : SCSI emulation for USB Mass Storage devices Vendor: IOMEGA Model: CDRW64892EXT3-B Rev: 6OP1 Type: CD-ROM ANSI SCSI revision: 02 USB Mass Storage device found at 5 sr0: scsi3-mmc drive: 12x/12x writer cd/rw xa/form2 cdda tray Attached scsi CD-ROM sr0 at scsi3, channel 0, id 0, lun 0 EXT3 FS on dm-0, internal journal device-mapper: dm-multipath version 1.0.4 loaded cdrom: open failed. kjournald starting. Commit interval 5 seconds EXT3 FS on hda1, internal journal EXT3-fs: mounted filesystem with ordered data mode. kjournald starting. Commit interval 5 seconds EXT3 FS on hdd1, internal journal EXT3-fs: recovery complete. EXT3-fs: mounted filesystem with ordered data mode. Adding 786424k swap on /dev/VolGroup00/LogVol01. Priority:-1 extents:1 IA-32 Microcode Update Driver: v1.14 <tigran at veritas.com> microcode: CPU0 already at revision 0x1 (current=0x1) microcode: No new microdata for cpu 0 IA-32 Microcode Update Driver v1.14 unregistered ip_tables: (C) 2000-2002 Netfilter core team ip_conntrack version 2.1 (3008 buckets, 24064 max) - 356 bytes per conntrack i2c /dev entries driver netdump: local port 6666 netdump: local IP 192.168.0.6 netdump: interface eth0 netdump: remote port 6666 netdump: remote IP 192.168.0.5 netdump: remote ethernet address 00:04:e2:54:be:69 netdump: network crash dump enabled netlog: local port 6664 netlog: local IP 192.168.0.6 netlog: interface eth0 netlog: remote port 6666 netlog: remote IP 192.168.0.5 netlog: remote ethernet address 00:04:e2:54:be:69 netconsole: network logging started lp0: using parport0 (polling). lp0: console ready Installing knfsd (copyright (C) 1996 okir at monad.swb.de). imm: Version 2.05 (for Linux 2.4.0) imm: Found device at ID 6, Attempting to use EPP 32 bit imm: Found device at ID 6, Attempting to use SPP imm: Communication established at 0x378 with ID 6 using SPP scsi4 : Iomega VPI2 (imm) interface Vendor: IOMEGA Model: ZIP 250 Rev: H.41 Type: Direct-Access ANSI SCSI revision: 02 Attached scsi removable disk sdd at scsi4, channel 0, id 6, lun 0 cdrom: This disc doesn't have any tracks I recognize! Feel free to ask for more detailed information, TIA Jose