[CentOS] Problem with CentOS 4.3 on kernel and ipvsadm

Alessio Cecchi

acecchi at telemaco.it
Mon Jul 3 08:15:45 UTC 2006


I have installed two CentOS 4.3 boxes with LVS (from
http://mirror.centos.org/centos/4/csgfs/ ) but all boxes died withe this
error frequently:

kernel panic - not syncing: fs/block_dev.c:396: spin_lock
(fs/block_dev.c:c0361c0) already locked by fs/block_dev.c/287.

I have read from this thread http://threebit.net/mail-archive/centos/msg00243.html that this is an unsolved problem.

So i have installed the kernel-2.6.9-39.EL from testing repo, but the box diedi with another error:

Jul  2 16:40:40 lvs2 kernel: Bad page state at free_hot_cold_page (in process 'smtp', page c10a2140)
Jul  2 16:40:40 lvs2 kernel: flags:0x20000014 mapping:00000000 mapcount:256 count:0
Jul  2 16:40:40 lvs2 kernel: Backtrace:
Jul  2 16:40:40 lvs2 kernel:  [<c014eaad>] bad_page+0x58/0x89
Jul  2 16:40:40 lvs2 kernel:  [<c014f2e1>] free_hot_cold_page+0x5f/0xc8
Jul  2 16:40:40 lvs2 kernel:  [<c0159b6a>] zap_pte_range+0x1d9/0x226
Jul  2 16:40:40 lvs2 kernel:  [<c0159bf9>] zap_pmd_range+0x42/0x68
Jul  2 16:40:40 lvs2 kernel:  [<c0159c58>] unmap_page_range+0x39/0x5f
Jul  2 16:40:40 lvs2 kernel:  [<c0159d7f>] unmap_vmas+0x101/0x1f8
Jul  2 16:40:40 lvs2 kernel:  [<c015ed4a>] exit_mmap+0xb8/0x1d0
Jul  2 16:40:40 lvs2 kernel:  [<c0121bb4>] mmput+0xb3/0xd6
Jul  2 16:40:40 lvs2 kernel:  [<c0127248>] do_exit+0x255/0x59c
Jul  2 16:40:40 lvs2 kernel:  [<c0127727>] sys_exit_group+0x0/0xd
Jul  2 16:40:40 lvs2 kernel:  [<c0317637>] syscall_call+0x7/0xb
Jul  2 16:40:40 lvs2 kernel: Trying to fix it up, but a reboot is needed

Jul  3 04:02:02 lvs2 kernel: ------------[ cut here ]------------
Jul  3 04:02:02 lvs2 kernel: kernel BUG at mm/rmap.c:479!
Jul  3 04:02:02 lvs2 kernel: invalid operand: 0000 [#1]
Jul  3 04:02:02 lvs2 kernel: Modules linked in: ip_vs_wlc ip_vs md5 ipv6 iptable_filter ipt_MASQUERADE iptable_nat ip_conntrack ipt_MARK iptable_mangle ip_tables button battery ac uhci_hcd parport_pc parport snd_via82xx snd_ac97_codec snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd_page_alloc snd_mpu401_uart snd_rawmidi snd_seq_device snd soundcore 8139too mii floppy dm_snapshot dm_zero dm_mirror ext3 jbd dm_mod
Jul  3 04:02:02 lvs2 kernel: CPU:    0
Jul  3 04:02:02 lvs2 kernel: EIP:    0060:[<c0160ee9>]    Tainted: G    B VLI
Jul  3 04:02:02 lvs2 kernel: EFLAGS: 00010296   (2.6.9-39.EL)
Jul  3 04:02:02 lvs2 kernel: EIP is at page_remove_rmap+0x22/0x36
Jul  3 04:02:02 lvs2 kernel: eax: fffffe00   ebx: c1082140   ecx: c1082140   edx: c1082140
Jul  3 04:02:02 lvs2 kernel: esi: 00000000   edi: 00003000   ebp: c42fd384   esp: c4241dec
Jul  3 04:02:02 lvs2 kernel: ds: 007b   es: 007b   ss: 0068
Jul  3 04:02:02 lvs2 kernel: Process smtp (pid: 5596, threadinfo=c4241000 task=cc920840)
Jul  3 04:02:02 lvs2 kernel: Stack: c0159b58 0410a067 00005000 080de000 c0405094 080de000 080e3000 c474d080
Jul  3 04:02:02 lvs2 kernel:        c0405094 c0159bf9 00005000 00000000 080de000 c474d080 080e3000 c0405094
Jul  3 04:02:02 lvs2 kernel:        c0159c58 00005000 00000000 c4241e9c 080de000 cc5cc5b8 080e3000 c0159d7f
Jul  3 04:02:02 lvs2 kernel: Call Trace:
Jul  3 04:02:02 lvs2 kernel:  [<c0159b58>] zap_pte_range+0x1c7/0x226
Jul  3 04:02:02 lvs2 kernel:  [<c0159bf9>] zap_pmd_range+0x42/0x68
Jul  3 04:02:02 lvs2 kernel:  [<c0159c58>] unmap_page_range+0x39/0x5f
Jul  3 04:02:02 lvs2 kernel:  [<c0159d7f>] unmap_vmas+0x101/0x1f8
Jul  3 04:02:03 lvs2 kernel:  [<c015ed4a>] exit_mmap+0xb8/0x1d0
Jul  3 04:02:03 lvs2 kernel:  [<c0121bb4>] mmput+0xb3/0xd6
Jul  3 04:02:03 lvs2 kernel:  [<c0177e51>] exec_mmap+0x2cd/0x2e7
Jul  3 04:02:03 lvs2 kernel:  [<c01780d6>] flush_old_exec+0x43/0x367
Jul  3 04:02:03 lvs2 kernel:  [<c0177b7a>] kernel_read+0x31/0x3b
Jul  3 04:02:03 lvs2 kernel:  [<c019cbeb>] load_elf_binary+0x520/0xbe0
Jul  3 04:02:03 lvs2 kernel:  [<c01776c1>] copy_strings+0x22b/0x235
Jul  3 04:02:03 lvs2 kernel:  [<c01791b1>] search_binary_handler+0x81/0x1cc
Jul  3 04:02:03 lvs2 kernel:  [<c0179464>] do_execve+0x168/0x1f6
Jul  3 04:02:03 lvs2 kernel:  [<c010496c>] sys_execve+0x2e/0x185
Jul  3 04:02:03 lvs2 kernel:  [<c0317637>] syscall_call+0x7/0xb
Jul  3 04:02:03 lvs2 kernel: Code: ff 05 90 af 42 c0 50 9d c3 89 c2 8b 00 f6 c4 08 74 08 0f 0b dc 01 7c bd 32 c0 83 42 08 ff 0f 98 c0 84 c0 74 19 8b 42 08 40 79 08 <0f> 0b df 01 7c bd 32 c0 9c 58 fa ff 0d 90 af 42 c0 50 9d c3 55
Jul  3 04:02:03 lvs2 kernel:  <0>Fatal exception: panic in 5 seconds
Jul  3 04:02:03 lvs2 kernel: bad: scheduling while atomic!
Jul  3 04:02:03 lvs2 kernel:  [<c0314921>] schedule+0x2d/0x5f1
Jul  3 04:02:03 lvs2 kernel:  [<c023a692>] poke_blanked_console+0x8f/0x9a
Jul  3 04:02:03 lvs2 kernel:  [<c0315fde>] schedule_timeout+0x15a/0x179
Jul  3 04:02:03 lvs2 kernel:  [<c012e79e>] process_timeout+0x0/0x5
Jul  3 04:02:03 lvs2 kernel:  [<c0124919>] printk+0xe/0x11
Jul  3 04:02:03 lvs2 kernel:  [<c0106924>] die+0x210/0x221
Jul  3 04:02:03 lvs2 kernel:  [<c0139d5f>] search_exception_tables+0x1f/0x21
Jul  3 04:02:03 lvs2 kernel:  [<c0106c87>] do_invalid_op+0xcf/0xf2
Jul  3 04:02:03 lvs2 kernel:  [<c01299bb>] current_fs_time+0x44/0x4c
Jul  3 04:02:03 lvs2 kernel:  [<c0160ee9>] page_remove_rmap+0x22/0x36
Jul  3 04:02:03 lvs2 kernel:  [<c014bd63>] do_generic_mapping_read+0x3fb/0x403
Jul  3 04:02:03 lvs2 kernel:  [<c0106bb8>] do_invalid_op+0x0/0xf2
Jul  3 04:02:03 lvs2 kernel:  [<c03186df>] error_code+0x2f/0x40
Jul  3 04:02:03 lvs2 kernel:  [<c0160ee9>] page_remove_rmap+0x22/0x36
Jul  3 04:02:03 lvs2 kernel:  [<c0159b58>] zap_pte_range+0x1c7/0x226
Jul  3 04:02:03 lvs2 kernel:  [<c0159bf9>] zap_pmd_range+0x42/0x68
Jul  3 04:02:03 lvs2 kernel:  [<c0159c58>] unmap_page_range+0x39/0x5f
Jul  3 04:02:03 lvs2 kernel:  [<c0159d7f>] unmap_vmas+0x101/0x1f8
Jul  3 04:02:03 lvs2 kernel:  [<c015ed4a>] exit_mmap+0xb8/0x1d0
Jul  3 04:02:03 lvs2 kernel:  [<c0121bb4>] mmput+0xb3/0xd6
Jul  3 04:02:03 lvs2 kernel:  [<c0177e51>] exec_mmap+0x2cd/0x2e7
Jul  3 04:02:03 lvs2 kernel:  [<c01780d6>] flush_old_exec+0x43/0x367
Jul  3 04:02:03 lvs2 kernel:  [<c0177b7a>] kernel_read+0x31/0x3b
Jul  3 04:02:03 lvs2 kernel:  [<c019cbeb>] load_elf_binary+0x520/0xbe0
Jul  3 04:02:03 lvs2 kernel:  [<c01776c1>] copy_strings+0x22b/0x235
Jul  3 04:02:03 lvs2 kernel:  [<c01791b1>] search_binary_handler+0x81/0x1cc
Jul  3 04:02:03 lvs2 kernel:  [<c0179464>] do_execve+0x168/0x1f6
Jul  3 04:02:03 lvs2 kernel:  [<c010496c>] sys_execve+0x2e/0x185
Jul  3 04:02:03 lvs2 kernel:  [<c0317637>] syscall_call+0x7/0xb
Jul  3 04:02:03 lvs2 kernel: Bad page state at prep_new_page (in process 'smtp', page c1082400)
Jul  3 04:02:03 lvs2 kernel: flags:0x20000000 mapping:00000300 mapcount:0 count:0
Jul  3 04:02:03 lvs2 kernel: Backtrace:
Jul  3 04:02:03 lvs2 kernel:  [<c014eaad>] bad_page+0x58/0x89
Jul  3 04:02:03 lvs2 kernel:  [<c014eeba>] prep_new_page+0x24/0x3a
Jul  3 04:02:03 lvs2 kernel:  [<c014f51f>] buffered_rmqueue+0x1c4/0x1e7
Jul  3 04:02:03 lvs2 kernel:  [<c014f5f5>] __alloc_pages+0xb3/0x29a
Jul  3 04:02:03 lvs2 kernel:  [<c01604df>] anon_vma_prepare+0x1f/0x1bf
Jul  3 04:02:03 lvs2 kernel:  [<c015ad3a>] do_wp_page+0x1d4/0x422
Jul  3 04:02:03 lvs2 kernel:  [<c015bf5c>] handle_mm_fault+0x136/0x212
Jul  3 04:02:03 lvs2 kernel:  [<c011d967>] do_page_fault+0x1ac/0x50c
Jul  3 04:02:03 lvs2 kernel:  [<c01345fe>] sys_rt_sigaction+0xdc/0xf0
Jul  3 04:02:03 lvs2 kernel:  [<c016b77f>] filp_close+0x5a/0x60
Jul  3 04:02:03 lvs2 kernel:  [<c011d7bb>] do_page_fault+0x0/0x50c
Jul  3 04:02:03 lvs2 kernel:  [<c03186df>] error_code+0x2f/0x40
Jul  3 04:02:03 lvs2 kernel: Trying to fix it up, but a reboot is needed
Jul  3 04:02:03 lvs2 kernel: Bad page state at prep_new_page (in process 'syslogd', page c1082440)
Jul  3 04:02:03 lvs2 kernel: flags:0x20000000 mapping:00000100 mapcount:0 count:0
Jul  3 04:02:03 lvs2 kernel: Backtrace:
Jul  3 04:02:03 lvs2 kernel:  [<c014eaad>] bad_page+0x58/0x89
Jul  3 04:02:03 lvs2 kernel:  [<c014eeba>] prep_new_page+0x24/0x3a
Jul  3 04:02:03 lvs2 kernel:  [<c014f51f>] buffered_rmqueue+0x1c4/0x1e7
Jul  3 04:02:03 lvs2 kernel:  [<c014f5f5>] __alloc_pages+0xb3/0x29a
Jul  3 04:02:03 lvs2 kernel:  [<c014f7f4>] __get_free_pages+0x18/0x24
Jul  3 04:02:03 lvs2 kernel:  [<c018083e>] __pollwait+0x2d/0x94
Jul  3 04:02:03 lvs2 kernel:  [<c02b8ca0>] datagram_poll+0x25/0xc5
Jul  3 04:02:03 lvs2 kernel:  [<c02b2d28>] sock_poll+0x12/0x14
Jul  3 04:02:03 lvs2 kernel:  [<c0180ba5>] do_select+0x25b/0x377
Jul  3 04:02:03 lvs2 kernel:  [<c0180811>] __pollwait+0x0/0x94
Jul  3 04:02:03 lvs2 kernel:  [<c0180fb1>] sys_select+0x2dd/0x437
Jul  3 04:02:03 lvs2 kernel:  [<c016db16>] sys_fsync+0xcd/0xd5
Jul  3 04:02:03 lvs2 kernel:  [<c0317637>] syscall_call+0x7/0xb
Jul  3 04:02:03 lvs2 kernel: Trying to fix it up, but a reboot is needed
Jul  3 04:02:03 lvs2 kernel: Bad page state at prep_new_page (in process 'http', page c1082460)
Jul  3 04:02:03 lvs2 kernel: flags:0x20000000 mapping:00000300 mapcount:0 count:0
[...]
Jul  3 04:02:07 lvs2 kernel: Trying to fix it up, but a reboot is needed
Jul  3 04:02:07 lvs2 kernel: Bad page state at prep_new_page (in process 'pop3', page c10825a0)
Jul  3 04:02:07 lvs2 kernel: flags:0x20000000 mapping:00000100 mapcount:0 count:0
Jul  3 04:02:07 lvs2 kernel: Backtrace:
Jul  3 04:02:07 lvs2 kernel:  [<c014eaad>] bad_page+0x58/0x89
Jul  3 04:02:07 lvs2 kernel:  [<c014eeba>] prep_new_page+0x24/0x3a
Jul  3 04:02:07 lvs2 kernel:  [<c014f51f>] buffered_rmqueue+0x1c4/0x1e7
Jul  3 04:02:07 lvs2 kernel:  [<c014f5f5>] __alloc_pages+0xb3/0x29a
Jul  3 04:02:07 lvs2 kernel:  [<c011d21e>] pte_alloc_one+0x18/0x49
Jul  3 04:02:07 lvs2 kernel:  [<c0159465>] pte_alloc_map+0x66/0x12d
Jul  3 04:02:07 lvs2 kernel:  [<c0159737>] copy_page_range+0xfe/0x358
Jul  3 04:02:07 lvs2 kernel:  [<c0122380>] dup_mmap+0x3de/0x4a6
Jul  3 04:02:07 lvs2 kernel:  [<c0121f61>] copy_mm+0x10e/0x14f
Jul  3 04:02:07 lvs2 kernel:  [<c0123172>] copy_process+0x709/0xd52
Jul  3 04:02:07 lvs2 kernel:  [<c0186d5e>] d_alloc+0xc2/0x284
Jul  3 04:02:07 lvs2 kernel:  [<c01238b1>] do_fork+0x98/0x1a2
Jul  3 04:02:07 lvs2 kernel:  [<c0187057>] d_instantiate+0x137/0x13a

So LVS is unusable because after of no more hourse of running the kernel go to panic.

There is a solutions to this problem?
I can help to test some patch or kernel.

Have somebody solved this situations?

Thanks  




More information about the CentOS mailing list