I experienced a bug that is likely the same as https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1350373 . Commit b7dd0e350e0bd4c0fddcc9b8958342700b00b168 , which is supposed to fix it, doesn't appear in this kernel and doesn't apply cleanly either. Is there any point in trying to backport the patch?
The backtrace is as follows:
[ 32.304666] ------------[ cut here ]------------ [ 32.304679] kernel BUG at arch/x86/kernel/paravirt.c:252! [ 32.304683] invalid opcode: 0000 [#1] SMP [ 32.304687] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter intel_powerclamp coretemp pcspkr ip_tables ext4 mbcache jbd2 xen_netfront xen_blkfront crc32c_intel [ 32.304734] CPU: 0 PID: 3901 Comm: dracut Not tainted 3.10.0-514.16.1.el7.x86_64 #1 [ 32.304739] task: ffff880002598000 ti: ffff88001b728000 task.ti: ffff88001b728000 [ 32.304743] RIP: e030:[<ffffffff8167eb81>] [<ffffffff8167eb81>] enter_lazy.part.0+0x4/0x6 [ 32.304755] RSP: e02b:ffff88001f803aa8 EFLAGS: 00010002 [ 32.304758] RAX: 0000000000000001 RBX: ffff88001eacd640 RCX: 00003ffffffff000 [ 32.304761] RDX: ffff880000000640 RSI: ffffc900000c8000 RDI: 0000000000000001 [ 32.304765] RBP: ffff88001f803aa8 R08: ffff88001f803b78 R09: ffffffff813d50f9 [ 32.304771] R10: ffff88001e801e00 R11: ffffea0000093dc0 R12: ffffc900000c9000 [ 32.304777] R13: ffffc900000c8000 R14: 0000000000000000 R15: ffff88001d150340 [ 32.304787] FS: 00007f64425b0740(0000) GS:ffff88001f800000(0000) knlGS:0000000000000000 [ 32.304796] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b [ 32.304801] CR2: 00000000006de2c8 CR3: 000000001b405000 CR4: 0000000000002660 [ 32.304807] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 32.304813] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 32.304818] Stack: [ 32.304823] ffff88001f803ab8 ffffffff81061857 ffff88001f803b60 ffffffff811b1fe2 [ 32.304833] ffffc900000c8fff ffffc900000c9000 ffffffff819bac90 ffffc900000c8fff [ 32.304843] ffffc900000c9000 ffff88001eacb000 ffffffff810206b0 0000000000000000 [ 32.304854] Call Trace: [ 32.304858] <IRQ> [ 32.304861] [<ffffffff81061857>] paravirt_enter_lazy_mmu+0x27/0x30 [ 32.304879] [<ffffffff811b1fe2>] apply_to_page_range+0x282/0x460 [ 32.304888] [<ffffffff810206b0>] ? map_pte_fn+0x60/0x60 [ 32.304894] [<ffffffff810207fb>] arch_gnttab_map_status+0x3b/0x70 [ 32.304904] [<ffffffff813d5176>] gnttab_map_frames_v2+0xd6/0x150 [ 32.304910] [<ffffffff813d5291>] gnttab_map+0xa1/0x140 [ 32.304917] [<ffffffff813d5430>] get_free_entries+0x100/0x2e0 [ 32.304923] [<ffffffff813d56d5>] gnttab_alloc_grant_references+0x15/0x30 [ 32.304933] [<ffffffffa000bd4f>] do_blkif_request+0x6bf/0x8a0 [xen_blkfront] [ 32.304945] [<ffffffff812eb0e2>] ? __freed_request+0x92/0xa0 [ 32.304951] [<ffffffff812eb6e3>] __blk_run_queue+0x33/0x40 [ 32.304957] [<ffffffff812eb719>] blk_start_queue+0x29/0x40 [ 32.304964] [<ffffffffa000bf51>] kick_pending_request_queues+0x21/0x30 [xen_blkfront] [ 32.304975] [<ffffffffa000c6ce>] blkif_interrupt+0x76e/0x820 [xen_blkfront] [ 32.304986] [<ffffffff811dcc8b>] ? kmem_cache_free+0x1bb/0x1f0 [ 32.304995] [<ffffffff8113079e>] handle_irq_event_percpu+0x3e/0x1e0 [ 32.305003] [<ffffffff8113097d>] handle_irq_event+0x3d/0x60 [ 32.305004] [<ffffffff81133647>] handle_edge_irq+0x77/0x130 [ 32.305004] [<ffffffff813d6217>] __xen_evtchn_do_upcall+0x227/0x350 [ 32.305004] [<ffffffff813d83c3>] xen_evtchn_do_upcall+0x33/0x50 [ 32.305004] [<ffffffff81698c7e>] xen_do_hypervisor_callback+0x1e/0x30 [ 32.305004] <EOI> [ 32.305004] [<ffffffff811af916>] ? copy_pte_range+0x2b6/0x5a0 [ 32.305004] [<ffffffff811af8e6>] ? copy_pte_range+0x286/0x5a0 [ 32.305004] [<ffffffff811b24d2>] ? copy_page_range+0x312/0x490 [ 32.305004] [<ffffffff81083012>] ? dup_mm+0x362/0x680 [ 32.305004] [<ffffffff810847ae>] ? copy_process+0x144e/0x1960 [ 32.305004] [<ffffffff81084e71>] ? do_fork+0x91/0x2c0 [ 32.305004] [<ffffffff81085126>] ? SyS_clone+0x16/0x20 [ 32.305004] [<ffffffff816974d9>] ? stub_clone+0x69/0x90 [ 32.305004] [<ffffffff81697189>] ? system_call_fastpath+0x16/0x1b [ 32.305004] Code: 20 e9 2f ff ff ff 44 89 fa 44 89 ee 48 c7 c7 10 45 8c 81 31 c0 e8 9d 14 00 00 58 5a 5b 41 5c 41 5d 41 5e 41 5f 5d c3 55 48 89 e5 <0f> 0b 66 66 66 66 90 55 48 89 e5 0f 0b 55 48 89 e5 0f 0b 55 48 [ 32.305004] RIP [<ffffffff8167eb81>] enter_lazy.part.0+0x4/0x6 [ 32.305004] RSP <ffff88001f803aa8> [ 32.305004] ---[ end trace 49f67f0d85e1ac69 ]--- [ 32.305004] Kernel panic - not syncing: Fatal exception in interrupt
Thanks, Sarah
On Sat, May 20, 2017 at 8:30 PM, Sarah Newman srn@prgmr.com wrote:
I experienced a bug that is likely the same as https://bugs.launchpad.net/ ubuntu/+source/linux/+bug/1350373 . Commit b7dd0e350e0bd4c0fddcc9b8958342700b00b168 , which is supposed to fix it, doesn't appear in this kernel and doesn't apply cleanly either. Is there any point in trying to backport the patch?
The backtrace is as follows:
[ 32.304666] ------------[ cut here ]------------ [ 32.304679] kernel BUG at arch/x86/kernel/paravirt.c:252! [ 32.304683] invalid opcode: 0000 [#1] SMP [ 32.304687] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter intel_powerclamp coretemp pcspkr ip_tables ext4 mbcache jbd2 xen_netfront xen_blkfront crc32c_intel [ 32.304734] CPU: 0 PID: 3901 Comm: dracut Not tainted 3.10.0-514.16.1.el7.x86_64 #1 [ 32.304739] task: ffff880002598000 ti: ffff88001b728000 task.ti: ffff88001b728000 [ 32.304743] RIP: e030:[<ffffffff8167eb81>] [<ffffffff8167eb81>] enter_lazy.part.0+0x4/0x6 [ 32.304755] RSP: e02b:ffff88001f803aa8 EFLAGS: 00010002 [ 32.304758] RAX: 0000000000000001 RBX: ffff88001eacd640 RCX: 00003ffffffff000 [ 32.304761] RDX: ffff880000000640 RSI: ffffc900000c8000 RDI: 0000000000000001 [ 32.304765] RBP: ffff88001f803aa8 R08: ffff88001f803b78 R09: ffffffff813d50f9 [ 32.304771] R10: ffff88001e801e00 R11: ffffea0000093dc0 R12: ffffc900000c9000 [ 32.304777] R13: ffffc900000c8000 R14: 0000000000000000 R15: ffff88001d150340 [ 32.304787] FS: 00007f64425b0740(0000) GS:ffff88001f800000(0000) knlGS:0000000000000000 [ 32.304796] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b [ 32.304801] CR2: 00000000006de2c8 CR3: 000000001b405000 CR4: 0000000000002660 [ 32.304807] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 32.304813] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 32.304818] Stack: [ 32.304823] ffff88001f803ab8 ffffffff81061857 ffff88001f803b60 ffffffff811b1fe2 [ 32.304833] ffffc900000c8fff ffffc900000c9000 ffffffff819bac90 ffffc900000c8fff [ 32.304843] ffffc900000c9000 ffff88001eacb000 ffffffff810206b0 0000000000000000 [ 32.304854] Call Trace: [ 32.304858] <IRQ> [ 32.304861] [<ffffffff81061857>] paravirt_enter_lazy_mmu+0x27/0x30 [ 32.304879] [<ffffffff811b1fe2>] apply_to_page_range+0x282/0x460 [ 32.304888] [<ffffffff810206b0>] ? map_pte_fn+0x60/0x60 [ 32.304894] [<ffffffff810207fb>] arch_gnttab_map_status+0x3b/0x70 [ 32.304904] [<ffffffff813d5176>] gnttab_map_frames_v2+0xd6/0x150 [ 32.304910] [<ffffffff813d5291>] gnttab_map+0xa1/0x140 [ 32.304917] [<ffffffff813d5430>] get_free_entries+0x100/0x2e0 [ 32.304923] [<ffffffff813d56d5>] gnttab_alloc_grant_references+ 0x15/0x30 [ 32.304933] [<ffffffffa000bd4f>] do_blkif_request+0x6bf/0x8a0 [xen_blkfront] [ 32.304945] [<ffffffff812eb0e2>] ? __freed_request+0x92/0xa0 [ 32.304951] [<ffffffff812eb6e3>] __blk_run_queue+0x33/0x40 [ 32.304957] [<ffffffff812eb719>] blk_start_queue+0x29/0x40 [ 32.304964] [<ffffffffa000bf51>] kick_pending_request_queues+0x21/0x30 [xen_blkfront] [ 32.304975] [<ffffffffa000c6ce>] blkif_interrupt+0x76e/0x820 [xen_blkfront] [ 32.304986] [<ffffffff811dcc8b>] ? kmem_cache_free+0x1bb/0x1f0 [ 32.304995] [<ffffffff8113079e>] handle_irq_event_percpu+0x3e/0x1e0 [ 32.305003] [<ffffffff8113097d>] handle_irq_event+0x3d/0x60 [ 32.305004] [<ffffffff81133647>] handle_edge_irq+0x77/0x130 [ 32.305004] [<ffffffff813d6217>] __xen_evtchn_do_upcall+0x227/0x350 [ 32.305004] [<ffffffff813d83c3>] xen_evtchn_do_upcall+0x33/0x50 [ 32.305004] [<ffffffff81698c7e>] xen_do_hypervisor_callback+0x1e/0x30 [ 32.305004] <EOI> [ 32.305004] [<ffffffff811af916>] ? copy_pte_range+0x2b6/0x5a0 [ 32.305004] [<ffffffff811af8e6>] ? copy_pte_range+0x286/0x5a0 [ 32.305004] [<ffffffff811b24d2>] ? copy_page_range+0x312/0x490 [ 32.305004] [<ffffffff81083012>] ? dup_mm+0x362/0x680 [ 32.305004] [<ffffffff810847ae>] ? copy_process+0x144e/0x1960 [ 32.305004] [<ffffffff81084e71>] ? do_fork+0x91/0x2c0 [ 32.305004] [<ffffffff81085126>] ? SyS_clone+0x16/0x20 [ 32.305004] [<ffffffff816974d9>] ? stub_clone+0x69/0x90 [ 32.305004] [<ffffffff81697189>] ? system_call_fastpath+0x16/0x1b [ 32.305004] Code: 20 e9 2f ff ff ff 44 89 fa 44 89 ee 48 c7 c7 10 45 8c 81 31 c0 e8 9d 14 00 00 58 5a 5b 41 5c 41 5d 41 5e 41 5f 5d c3 55 48 89 e5 <0f> 0b 66 66 66 66 90 55 48 89 e5 0f 0b 55 48 89 e5 0f 0b 55 48 [ 32.305004] RIP [<ffffffff8167eb81>] enter_lazy.part.0+0x4/0x6 [ 32.305004] RSP <ffff88001f803aa8> [ 32.305004] ---[ end trace 49f67f0d85e1ac69 ]--- [ 32.305004] Kernel panic - not syncing: Fatal exception in interrupt
I had the same kernel panic while booting a PV domU on 3.10.0-693.2.2.el7.centos.plus.x86_64. I had to start the domU again to boot correctly. Can this patch be added to the CentOS 7 kernel-plus?
[ 13.372417] ------------[ cut here ]------------ [ 13.372434] kernel BUG at arch/x86/kernel/paravirt.c:252! [ 13.372441] invalid opcode: 0000 [#1] SMP [ 13.372450] Modules linked in: xt_owner nf_nat_ftp xt_REDIRECT nf_nat_redirect xt_conntrack iptable_mangle nf_conntrack_ftp xt_LOG xt_limit xt_multiport iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ipt_REJECT nf_reject_ipv4 iptable_filter vfat fat isofs xfs libcrc32c loop sb_edac edac_core coretemp intel_rapl iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr ip_tables ext4 mbcache jbd2 xen_blkfront xen_netfront crct10dif_pclmul crct10dif_common crc32c_intel [ 13.372545] CPU: 0 PID: 1138 Comm: mysqld Not tainted 3.10.0-693.2.2.el7.centos.plus.x86_64 #1 [ 13.372555] task: ffff8800fb1a8fd0 ti: ffff8801e9e20000 task.ti: ffff8801e9e20000 [ 13.372561] RIP: e030:[<ffffffff816ad7fe>] [<ffffffff816ad7fe>] enter_lazy.part.0+0x4/0x6 [ 13.372579] RSP: e02b:ffff8801fea03a80 EFLAGS: 00010002 [ 13.372584] RAX: 0000000000000001 RBX: ffff88017d05b280 RCX: ffffffff810215a0 [ 13.372593] RDX: ffff880000000280 RSI: 00003ffffffff000 RDI: 0000000000000001 [ 13.372599] RBP: ffff8801fea03a80 R08: ffff8801fea03b50 R09: ffffffff813f6559 [ 13.372605] R10: ffff88017fc01d00 R11: ffffea00000c0380 R12: ffffc90000c52000 [ 13.372616] R13: ffffc90000c50000 R14: 0000000000000000 R15: ffff8801e3770e00 [ 13.372632] FS: 00007febcc67a900(0000) GS:ffff8801fea00000(0000) knlGS:ffff8801fea00000 [ 13.372644] CS: e033 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 13.372654] CR2: 000056038881bd60 CR3: 00000000f996d000 CR4: 0000000000042660 [ 13.372663] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 13.372672] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 13.372680] Stack: [ 13.372685] ffff8801fea03a90 ffffffff81063b05 ffff8801fea03b38 ffffffff811b2a80 [ 13.372701] ffffc90000c51fff ffffc90000c52000 ffffffff81a0ac90 ffffc90000c51fff [ 13.372718] ffffc90000c52000 ffff88017fc50000 ffffffff810215a0 00000000fead6d00 [ 13.372733] Call Trace: [ 13.372740] <IRQ> [ 13.372744] [<ffffffff81063b05>] paravirt_enter_lazy_mmu+0x25/0x30 [ 13.372771] [<ffffffff811b2a80>] apply_to_page_range+0x260/0x430 [ 13.372784] [<ffffffff810215a0>] ? map_pte_fn+0x60/0x60 [ 13.372794] [<ffffffff810216eb>] arch_gnttab_map_status+0x3b/0x70 [ 13.372809] [<ffffffff813f65d6>] gnttab_map_frames_v2+0xd6/0x150 [ 13.372820] [<ffffffff813f66f1>] gnttab_map+0xa1/0x140 [ 13.372831] [<ffffffff813f6890>] get_free_entries+0x100/0x2e0
Karl
On Mon, Oct 23, 2017 at 12:57 PM, Karl Johnson karljohnson.it@gmail.com wrote:
On Sat, May 20, 2017 at 8:30 PM, Sarah Newman srn@prgmr.com wrote:
I experienced a bug that is likely the same as https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1350373 . Commit b7dd0e350e0bd4c0fddcc9b8958342700b00b168 , which is supposed to fix it, doesn't appear in this kernel and doesn't apply cleanly either. Is there any point in trying to backport the patch?
I had the same kernel panic while booting a PV domU on
3.10.0-693.2.2.el7.centos.plus.x86_64. I had to start the domU again to boot correctly. Can this patch be added to the CentOS 7 kernel-plus?
Karl
I can certainly add the patch (commit b7dd0e350e0bd4c0fddcc9b8958342700b00b168) to the Plus kernel. It would be best if you could file a request on http://bugs.centos.org so that we can track it better.
Akemi
On Mon, Oct 23, 2017 at 11:08 PM, Akemi Yagi amyagi@gmail.com wrote:
On Mon, Oct 23, 2017 at 12:57 PM, Karl Johnson karljohnson.it@gmail.com wrote:
On Sat, May 20, 2017 at 8:30 PM, Sarah Newman srn@prgmr.com wrote:
I experienced a bug that is likely the same as https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1350373 . Commit b7dd0e350e0bd4c0fddcc9b8958342700b00b168 , which is supposed to fix it, doesn't appear in this kernel and doesn't apply cleanly either. Is there any point in trying to backport the patch?
I had the same kernel panic while booting a PV domU on
3.10.0-693.2.2.el7.centos.plus.x86_64. I had to start the domU again to boot correctly. Can this patch be added to the CentOS 7 kernel-plus?
Karl
I can certainly add the patch (commit b7dd0e350e0bd4c0fddcc9b8958342700b00b168) to the Plus kernel. It would be best if you could file a request on http://bugs.centos.org so that we can track it better.
Akemi
A CentOSPlus kernel set with the referenced patch applied is available for testing at:
https://people.centos.org/toracat/kernel/7/plus/xen/
Feedback appreciated,
Akemi
On Tue, Oct 24, 2017 at 3:36 AM, Akemi Yagi amyagi@gmail.com wrote:
On Mon, Oct 23, 2017 at 11:08 PM, Akemi Yagi amyagi@gmail.com wrote:
On Mon, Oct 23, 2017 at 12:57 PM, Karl Johnson karljohnson.it@gmail.com wrote:
On Sat, May 20, 2017 at 8:30 PM, Sarah Newman srn@prgmr.com wrote:
I experienced a bug that is likely the same as https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1350373 . Commit b7dd0e350e0bd4c0fddcc9b8958342700b00b168 , which is supposed to fix it, doesn't appear in this kernel and doesn't apply cleanly either. Is there any point in trying to backport the patch?
I had the same kernel panic while booting a PV domU on
3.10.0-693.2.2.el7.centos.plus.x86_64. I had to start the domU again to boot correctly. Can this patch be added to the CentOS 7 kernel-plus?
Karl
I can certainly add the patch (commit b7dd0e350e0bd4c0fddcc9b8958342700b00b168) to the Plus kernel. It would be best if you could file a request on http://bugs.centos.org so that we can track it better.
Akemi
A CentOSPlus kernel set with the referenced patch applied is available for testing at:
https://people.centos.org/toracat/kernel/7/plus/xen/
Feedback appreciated,
Akemi
Thanks for the build Akemi. I will try to test this kernel in the next days however it will be hard to know if it fix the kernel panic because I can't reproduce it. It's seems to be random and pretty rare in my case.
On Tue, Oct 24, 2017 at 3:09 PM, Karl Johnson karljohnson.it@gmail.com wrote:
On Tue, Oct 24, 2017 at 3:36 AM, Akemi Yagi amyagi@gmail.com wrote:
On Mon, Oct 23, 2017 at 11:08 PM, Akemi Yagi amyagi@gmail.com wrote:
On Mon, Oct 23, 2017 at 12:57 PM, Karl Johnson <karljohnson.it@gmail.com
wrote:
On Sat, May 20, 2017 at 8:30 PM, Sarah Newman srn@prgmr.com wrote:
I experienced a bug that is likely the same as https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1350373 . Commit b7dd0e350e0bd4c0fddcc9b8958342700b00b168 , which is supposed to fix it, doesn't appear in this kernel and doesn't apply cleanly either. Is there any point in trying to backport the patch?
I had the same kernel panic while booting a PV domU on
3.10.0-693.2.2.el7.centos.plus.x86_64. I had to start the domU again to boot correctly. Can this patch be added to the CentOS 7 kernel-plus?
Karl
I can certainly add the patch (commit b7dd0e350e0bd4c0fddcc9b8958342700b00b168) to the Plus kernel. It would be best if you could file a request on http://bugs.centos.org so that we can track it better.
Akemi
A CentOSPlus kernel set with the referenced patch applied is available for testing at:
https://people.centos.org/toracat/kernel/7/plus/xen/
Feedback appreciated,
Akemi
Thanks for the build Akemi. I will try to test this kernel in the next days however it will be hard to know if it fix the kernel panic because I can't reproduce it. It's seems to be random and pretty rare in my case.
The test kernel doesn't boot on my side:
[ 0.000000] Initializing cgroup subsys cpuset [ 0.000000] Initializing cgroup subsys cpu [ 0.000000] Initializing cgroup subsys cpuacct [ 0.000000] Linux version 3.10.0-693.5.2.el7.centos.plus.1.x86_64 (yagi2@h64r7) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) ) #1 SMP Mon Oct 23 22:30:37 PDT 2017 [ 0.000000] Command line: console=hvc0 xencons=tty0 root=/dev/xvda1 ro LANG=en_CA.UTF-8 elevator=noop nohz=off [ 0.000000] ACPI in unprivileged domain disabled [ 0.000000] e820: BIOS-provided physical RAM map: [ 0.000000] Xen: [mem 0x0000000000000000-0x000000000009ffff] usable [ 0.000000] Xen: [mem 0x00000000000a0000-0x00000000000fffff] reserved [ 0.000000] Xen: [mem 0x0000000000100000-0x000000003fffffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] DMI not present or invalid. [ 0.000000] e820: last_pfn = 0x40000 max_arch_pfn = 0x400000000 [ 0.000000] RAMDISK: [mem 0x0242d000-0x038e0fff] [ 0.000000] NUMA turned off [ 0.000000] Faking a node at [mem 0x0000000000000000-0x000000003fffffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x3fe03000-0x3fe29fff] [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x00001000-0x00ffffff] [ 0.000000] DMA32 [mem 0x01000000-0xffffffff] [ 0.000000] Normal empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x00001000-0x0009ffff] [ 0.000000] node 0: [mem 0x00100000-0x3fffffff] [ 0.000000] Initmem setup node 0 [mem 0x00001000-0x3fffffff] [ 0.000000] SFI: Simple Firmware Interface v0.81 http://simplefirmware.org [ 0.000000] No local APIC present [ 0.000000] APIC: disable apic facility [ 0.000000] APIC: switched to apic NOOP [ 0.000000] smpboot: Allowing 2 CPUs, 0 hotplug CPUs [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000fffff] [ 0.000000] e820: [mem 0x40000000-0xffffffff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on Xen [ 0.000000] Xen version: 4.6.3-3.el6 (preserve-AD) [ 0.000000] setup_percpu: NR_CPUS:5120 nr_cpumask_bits:2 nr_cpu_ids:2 nr_node_ids:1 [ 0.000000] PERCPU: Embedded 33 pages/cpu @ffff88003f800000 s97112 r8192 d29864 u1048576 [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes) [ 0.000000] Built 1 zonelists in Node order, mobility grouping on. Total pages: 257930 [ 0.000000] Policy zone: DMA32 [ 0.000000] Kernel command line: console=hvc0 xencons=tty0 root=/dev/xvda1 ro LANG=en_CA.UTF-8 elevator=noop nohz=off [ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes) [ 0.000000] x86/fpu: xstate_offset[2]: 0240, xstate_sizes[2]: 0100 [ 0.000000] xsave: enabled xstate_bv 0x7, cntxt size 0x340 using standard form [ 0.000000] Memory: 989236k/1048576k available (6954k kernel code, 388k absent, 58952k reserved, 4575k data, 1768k init) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1 [ 0.000000] Hierarchical RCU implementation. [ 0.000000] RCU restricting CPUs from NR_CPUS=5120 to nr_cpu_ids=2. [ 0.000000] NR_IRQS:327936 nr_irqs:32 0 [ 0.000000] Console: colour dummy device 80x25 [ 0.000000] console [tty0] enabled [ 0.000000] console [hvc0] enabled [ 0.000000] allocated 4194304 bytes of page_cgroup [ 0.000000] please try 'cgroup_disable=memory' option if you don't want memory cgroups [ 0.000000] installing Xen timer for CPU 0 [ 0.000000] tsc: Fast TSC calibration using PIT [ 0.000000] tsc: Detected 2100.066 MHz processor [ 0.002000] Calibrating delay loop (skipped), value calculated using timer frequency.. 4200.06 BogoMIPS (lpj=2100030) [ 0.002000] pid_max: default: 32768 minimum: 301 [ 0.002000] Security Framework initialized [ 0.002000] SELinux: Initializing. [ 0.002000] Yama: becoming mindful. [ 0.002000] Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes) [ 0.002000] Inode-cache hash table entries: 65536 (order: 7, 524288 bytes) [ 0.002000] Mount-cache hash table entries: 2048 (order: 2, 16384 bytes) [ 0.002000] Mountpoint-cache hash table entries: 2048 (order: 2, 16384 bytes) [ 0.002086] Initializing cgroup subsys memory [ 0.002104] Initializing cgroup subsys devices [ 0.002111] Initializing cgroup subsys freezer [ 0.002116] Initializing cgroup subsys net_cls [ 0.002122] Initializing cgroup subsys blkio [ 0.002127] Initializing cgroup subsys perf_event [ 0.002133] Initializing cgroup subsys hugetlb [ 0.002138] Initializing cgroup subsys pids [ 0.002143] Initializing cgroup subsys net_prio [ 0.002207] ENERGY_PERF_BIAS: Set to 'normal', was 'performance' [ 0.002214] ENERGY_PERF_BIAS: View and update with x86_energy_perf_policy(8) [ 0.002221] CPU: Physical Processor ID: 0 [ 0.002225] CPU: Processor Core ID: 0 [ 0.003093] Last level iTLB entries: 4KB 512, 2MB 0, 4MB 0 [ 0.003098] Last level dTLB entries: 4KB 512, 2MB 0, 4MB 0 [ 0.003103] tlb_flushall_shift: 6 [ 0.036643] ftrace: allocating 26819 entries in 105 pages [ 0.043078] cpu 0 spinlock event irq 17 [ 0.043086] smpboot: Max logical packages: 1 [ 0.043118] Performance Events: unsupported p6 CPU model 62 no PMU driver, software events only. [ 0.044508] NMI watchdog: disabled (cpu0): hardware events not enabled [ 0.044515] NMI watchdog: Shutting down hard lockup detector on all cpus [ 0.044598] installing Xen timer for CPU 1 [ 0.044613] cpu 1 spinlock event irq 24 [ 0.044678] SMP alternatives: switching to SMP code [ 0.002000] [Firmware Bug]: CPU1: APIC id mismatch. Firmware: ffff APIC: 6 [ 0.072708] Brought up 2 CPUs [ 0.073046] devtmpfs: initialized [ 0.075736] EVM: security.selinux [ 0.075742] EVM: security.ima [ 0.075746] EVM: security.capability [ 0.076705] atomic64 test passed for x86-64 platform with CX8 and with SSE [ 0.076714] pinctrl core: initialized pinctrl subsystem [ 0.076763] xen:grant_table: Grant tables using version 2 layout [ 0.076775] BUG: unable to handle kernel NULL pointer dereference at 0000000000000010 [ 0.076786] IP: [<ffffffff813f6d0f>] gnttab_init+0xff/0x260 [ 0.076796] PGD 0 [ 0.076802] Oops: 0002 [#1] SMP [ 0.076808] Modules linked in: [ 0.076817] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.10.0-693.5.2.el7.centos.plus.1.x86_64 #1 [ 0.076825] task: ffff88003da38000 ti: ffff88003daa0000 task.ti: ffff88003daa0000 [ 0.076831] RIP: e030:[<ffffffff813f6d0f>] [<ffffffff813f6d0f>] gnttab_init+0xff/0x260 [ 0.076840] RSP: e02b:ffff88003daa3df8 EFLAGS: 00010286 [ 0.076844] RAX: ffff88003d405000 RBX: 0000000000000000 RCX: 000000000001a210 [ 0.076849] RDX: 0000000000000000 RSI: 000000000000001e RDI: 0000000000000000 [ 0.076854] RBP: ffff88003daa3e40 R08: 0000000000000000 R09: 000000000001a1b0 [ 0.076859] R10: ffff88003fe03800 R11: 0000000000000001 R12: 0000000000000000 [ 0.077000] R13: 0000000000000001 R14: 0000000000000010 R15: 0000000000000000 [ 0.077000] FS: 0000000000000000(0000) GS:ffff88003f800000(0000) knlGS:0000000000000000 [ 0.077000] CS: e033 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 0.077000] CR2: 0000000000000010 CR3: 0000000001a0a000 CR4: 0000000000042660 [ 0.077000] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 0.077000] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 0.077000] Stack: [ 0.077000] 0000000000000000 0000000400007ff0 ffff000000000020 000000001fffa6db [ 0.077000] ffffffff81a11020 ffff88003e002a70 ffffffff813f6e70 0000000000000000 [ 0.077000] 0000000000000000 ffff88003daa3e50 ffffffff813f6e93 ffff88003daa3e80 [ 0.077000] Call Trace: [ 0.077000] [<ffffffff813f6e70>] ? gnttab_init+0x260/0x260 [ 0.077000] [<ffffffff813f6e93>] __gnttab_init+0x23/0x40 [ 0.077000] [<ffffffff810020e8>] do_one_initcall+0xb8/0x230 [ 0.077000] [<ffffffff81b5d1fb>] kernel_init_freeable+0x17a/0x219 [ 0.077000] [<ffffffff81b5c9d4>] ? initcall_blacklist+0xb0/0xb0 [ 0.077000] [<ffffffff816a3d20>] ? rest_init+0x80/0x80 [ 0.077000] [<ffffffff816a3d2e>] kernel_init+0xe/0xf0 [ 0.077000] [<ffffffff816c5f98>] ret_from_fork+0x58/0x90 [ 0.077000] [<ffffffff816a3d20>] ? rest_init+0x80/0x80 [ 0.077000] Code: 00 00 66 2e 0f 1f 84 00 00 00 00 00 83 c3 01 41 39 dd 0f 86 84 00 00 00 4c 63 e3 31 f6 bf d0 00 00 00 4e 8d 34 e0 e8 01 09 d9 ff <49> 89 06 48 8b 05 37 0d bf 00 4a 83 3c e0 00 75 d0 48 89 c7 41 [ 0.077000] RIP [<ffffffff813f6d0f>] gnttab_init+0xff/0x260 [ 0.077000] RSP <ffff88003daa3df8> [ 0.077000] CR2: 0000000000000010 [ 0.077000] ---[ end trace ad7a936cdeb5166e ]--- [ 0.077000] Kernel panic - not syncing: Fatal exception
I switched back to 3.10.0-693.2.2.el7.centos.plus.x86_64.
On Tue, Oct 24, 2017 at 12:53 PM, Karl Johnson karljohnson.it@gmail.com wrote:
On Tue, Oct 24, 2017 at 3:09 PM, Karl Johnson karljohnson.it@gmail.com wrote:
On Tue, Oct 24, 2017 at 3:36 AM, Akemi Yagi amyagi@gmail.com wrote:
On Mon, Oct 23, 2017 at 11:08 PM, Akemi Yagi amyagi@gmail.com wrote:
On Mon, Oct 23, 2017 at 12:57 PM, Karl Johnson < karljohnson.it@gmail.com> wrote:
On Sat, May 20, 2017 at 8:30 PM, Sarah Newman srn@prgmr.com wrote:
I experienced a bug that is likely the same as https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1350373 . Commit b7dd0e350e0bd4c0fddcc9b8958342700b00b168 , which is supposed to fix it, doesn't appear in this kernel and doesn't apply cleanly either. Is there any point in trying to backport the patch?
I had the same kernel panic while booting a PV domU on
3.10.0-693.2.2.el7.centos.plus.x86_64. I had to start the domU again to boot correctly. Can this patch be added to the CentOS 7 kernel-plus?
Karl
I can certainly add the patch (commit b7dd0e350e0bd4c0fddcc9b8958342700b00b168) to the Plus kernel. It would be best if you could file a request on http://bugs.centos.org so that we can track it better.
Akemi
A CentOSPlus kernel set with the referenced patch applied is available for testing at:
https://people.centos.org/toracat/kernel/7/plus/xen/
Feedback appreciated,
Akemi
Thanks for the build Akemi. I will try to test this kernel in the next days however it will be hard to know if it fix the kernel panic because I can't reproduce it. It's seems to be random and pretty rare in my case.
The test kernel doesn't boot on my side:
[ 0.077000] Call Trace: [ 0.077000] [<ffffffff813f6e70>] ? gnttab_init+0x260/0x260 [ 0.077000] [<ffffffff813f6e93>] __gnttab_init+0x23/0x40 [ 0.077000] [<ffffffff810020e8>] do_one_initcall+0xb8/0x230 [ 0.077000] [<ffffffff81b5d1fb>] kernel_init_freeable+0x17a/0x219 [ 0.077000] [<ffffffff81b5c9d4>] ? initcall_blacklist+0xb0/0xb0 [ 0.077000] [<ffffffff816a3d20>] ? rest_init+0x80/0x80 [ 0.077000] [<ffffffff816a3d2e>] kernel_init+0xe/0xf0 [ 0.077000] [<ffffffff816c5f98>] ret_from_fork+0x58/0x90 [ 0.077000] [<ffffffff816a3d20>] ? rest_init+0x80/0x80 [ 0.077000] Code: 00 00 66 2e 0f 1f 84 00 00 00 00 00 83 c3 01 41 39 dd 0f 86 84 00 00 00 4c 63 e3 31 f6 bf d0 00 00 00 4e 8d 34 e0 e8 01 09 d9 ff <49> 89 06 48 8b 05 37 0d bf 00 4a 83 3c e0 00 75 d0 48 89 c7 41 [ 0.077000] RIP [<ffffffff813f6d0f>] gnttab_init+0xff/0x260 [ 0.077000] RSP <ffff88003daa3df8> [ 0.077000] CR2: 0000000000000010 [ 0.077000] ---[ end trace ad7a936cdeb5166e ]--- [ 0.077000] Kernel panic - not syncing: Fatal exception
I switched back to 3.10.0-693.2.2.el7.centos.plus.x86_64.
Looks as if the patch broke something before it could fix the problem...
Akemi
-
Akemi Yagi -
Karl Johnson -
Sarah Newman