virt June 2017

virt@lists.centos.org

9 participants
6 discussions

Crash in CentOS 7 kernel-3.10.0-514.16.1.el7.x86_64 in Xen PV mode
by Sarah Newman 24 Oct '17

24 Oct '17

I experienced a bug that is likely the same as https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1350373 . Commit b7dd0e350e0bd4c0fddcc9b8958342700b00b168 , which is supposed to fix it, doesn't appear in this kernel and doesn't apply cleanly either. Is there any point in trying to backport the patch? The backtrace is as follows: [ 32.304666] ------------[ cut here ]------------ [ 32.304679] kernel BUG at arch/x86/kernel/paravirt.c:252! [ 32.304683] invalid opcode: 0000 [#1] SMP [ 32.304687] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter intel_powerclamp coretemp pcspkr ip_tables ext4 mbcache jbd2 xen_netfront xen_blkfront crc32c_intel [ 32.304734] CPU: 0 PID: 3901 Comm: dracut Not tainted 3.10.0-514.16.1.el7.x86_64 #1 [ 32.304739] task: ffff880002598000 ti: ffff88001b728000 task.ti: ffff88001b728000 [ 32.304743] RIP: e030:[<ffffffff8167eb81>] [<ffffffff8167eb81>] enter_lazy.part.0+0x4/0x6 [ 32.304755] RSP: e02b:ffff88001f803aa8 EFLAGS: 00010002 [ 32.304758] RAX: 0000000000000001 RBX: ffff88001eacd640 RCX: 00003ffffffff000 [ 32.304761] RDX: ffff880000000640 RSI: ffffc900000c8000 RDI: 0000000000000001 [ 32.304765] RBP: ffff88001f803aa8 R08: ffff88001f803b78 R09: ffffffff813d50f9 [ 32.304771] R10: ffff88001e801e00 R11: ffffea0000093dc0 R12: ffffc900000c9000 [ 32.304777] R13: ffffc900000c8000 R14: 0000000000000000 R15: ffff88001d150340 [ 32.304787] FS: 00007f64425b0740(0000) GS:ffff88001f800000(0000) knlGS:0000000000000000 [ 32.304796] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b [ 32.304801] CR2: 00000000006de2c8 CR3: 000000001b405000 CR4: 0000000000002660 [ 32.304807] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 32.304813] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 32.304818] Stack: [ 32.304823] ffff88001f803ab8 ffffffff81061857 ffff88001f803b60 ffffffff811b1fe2 [ 32.304833] ffffc900000c8fff ffffc900000c9000 ffffffff819bac90 ffffc900000c8fff [ 32.304843] ffffc900000c9000 ffff88001eacb000 ffffffff810206b0 0000000000000000 [ 32.304854] Call Trace: [ 32.304858] <IRQ> [ 32.304861] [<ffffffff81061857>] paravirt_enter_lazy_mmu+0x27/0x30 [ 32.304879] [<ffffffff811b1fe2>] apply_to_page_range+0x282/0x460 [ 32.304888] [<ffffffff810206b0>] ? map_pte_fn+0x60/0x60 [ 32.304894] [<ffffffff810207fb>] arch_gnttab_map_status+0x3b/0x70 [ 32.304904] [<ffffffff813d5176>] gnttab_map_frames_v2+0xd6/0x150 [ 32.304910] [<ffffffff813d5291>] gnttab_map+0xa1/0x140 [ 32.304917] [<ffffffff813d5430>] get_free_entries+0x100/0x2e0 [ 32.304923] [<ffffffff813d56d5>] gnttab_alloc_grant_references+0x15/0x30 [ 32.304933] [<ffffffffa000bd4f>] do_blkif_request+0x6bf/0x8a0 [xen_blkfront] [ 32.304945] [<ffffffff812eb0e2>] ? __freed_request+0x92/0xa0 [ 32.304951] [<ffffffff812eb6e3>] __blk_run_queue+0x33/0x40 [ 32.304957] [<ffffffff812eb719>] blk_start_queue+0x29/0x40 [ 32.304964] [<ffffffffa000bf51>] kick_pending_request_queues+0x21/0x30 [xen_blkfront] [ 32.304975] [<ffffffffa000c6ce>] blkif_interrupt+0x76e/0x820 [xen_blkfront] [ 32.304986] [<ffffffff811dcc8b>] ? kmem_cache_free+0x1bb/0x1f0 [ 32.304995] [<ffffffff8113079e>] handle_irq_event_percpu+0x3e/0x1e0 [ 32.305003] [<ffffffff8113097d>] handle_irq_event+0x3d/0x60 [ 32.305004] [<ffffffff81133647>] handle_edge_irq+0x77/0x130 [ 32.305004] [<ffffffff813d6217>] __xen_evtchn_do_upcall+0x227/0x350 [ 32.305004] [<ffffffff813d83c3>] xen_evtchn_do_upcall+0x33/0x50 [ 32.305004] [<ffffffff81698c7e>] xen_do_hypervisor_callback+0x1e/0x30 [ 32.305004] <EOI> [ 32.305004] [<ffffffff811af916>] ? copy_pte_range+0x2b6/0x5a0 [ 32.305004] [<ffffffff811af8e6>] ? copy_pte_range+0x286/0x5a0 [ 32.305004] [<ffffffff811b24d2>] ? copy_page_range+0x312/0x490 [ 32.305004] [<ffffffff81083012>] ? dup_mm+0x362/0x680 [ 32.305004] [<ffffffff810847ae>] ? copy_process+0x144e/0x1960 [ 32.305004] [<ffffffff81084e71>] ? do_fork+0x91/0x2c0 [ 32.305004] [<ffffffff81085126>] ? SyS_clone+0x16/0x20 [ 32.305004] [<ffffffff816974d9>] ? stub_clone+0x69/0x90 [ 32.305004] [<ffffffff81697189>] ? system_call_fastpath+0x16/0x1b [ 32.305004] Code: 20 e9 2f ff ff ff 44 89 fa 44 89 ee 48 c7 c7 10 45 8c 81 31 c0 e8 9d 14 00 00 58 5a 5b 41 5c 41 5d 41 5e 41 5f 5d c3 55 48 89 e5 <0f> 0b 66 66 66 66 90 55 48 89 e5 0f 0b 55 48 89 e5 0f 0b 55 48 [ 32.305004] RIP [<ffffffff8167eb81>] enter_lazy.part.0+0x4/0x6 [ 32.305004] RSP <ffff88001f803aa8> [ 32.305004] ---[ end trace 49f67f0d85e1ac69 ]--- [ 32.305004] Kernel panic - not syncing: Fatal exception in interrupt Thanks, Sarah

3 6

4.9.34-29 EL6 and EL7 kernels
by Johnny Hughes 13 Jul '17

13 Jul '17

OK Guys and Gals, We have just kicked off a build of the 4.9.34-29 xen kernel on the Community Build Service for both EL6 and EL7. This kernel has some important things in it. 1. It has a patch for the stack guard issue (CVE-2017-1000364). Also the patch for Xen Security fix XSA-216. See these upstream changelogs for ALL kernel commits from 4.9.31 to 4.9.34: https://cdn.kernel.org/pub/linux/kernel/v4.x/ChangeLog-4.9.32 https://cdn.kernel.org/pub/linux/kernel/v4.x/ChangeLog-4.9.33 https://cdn.kernel.org/pub/linux/kernel/v4.x/ChangeLog-4.9.34 2. It has a fix for the xen-netback rate limit queues (http://www.spinics.net/lists/netdev/msg441578.html) (Thanks very much to Kevin Stange and Jean-Louis Dupond for finding a fix for this issue) 3. It now contains debuginfo files (thanks to Sarah Newman for this amazing addition!). ===================== The kernels are currently building, it will likely take 4 to 6 hours for them to make their way to the applicable testing repos: CentOS-7: https://buildlogs.centos.org/centos/7/virt/x86_64/xen-46/ or CentOS-6: https://buildlogs.centos.org/centos/6/virt/x86_64/xen-44/ https://buildlogs.centos.org/centos/6/virt/x86_64/xen-46/ ===================== If you actually need the debuginfo files, they will be here (also in 4 or so hours): CentOS-7: http://cbs.centos.org/repos/virt7-xen-common-testing/x86_64/debug/ CentOS-6: http://cbs.centos.org/repos/virt6-xen-common-testing/x86_64/debug/ ===================== Let's do some real testing of these and see if we can solve the issues people are having .. this should already solve server and the debuginfo files should help solve others. I think Kevin and Jean-Louis also know how to solve the Windows BSOD when run on a Xen 4.9.x dom0 server .. they can comment here. Hopefully whoever maintains the newer Windows drivers will fix the ACPI / APIC issues and no work around will be necessary soon(ish). I also want to personally (and publicly) thank Kevin, Jean-Louis, and Sarah for the substantial work and the community collaboration. This is EXACTLY what we need for the Virt SIG .. people solving problems and doing github pull requests so we can, together as a group, get these projects working much better. If someone else wants to help us with the Kernel .. or Xen .. here are our github branches: https://github.com/CentOS-virt7/xen-kernel https://github.com/CentOS-virt7/xen Fork them .. work on them .. talk to us on #centos-virt in freenode or on this mailing list .. then once we are all happy, you can submit a pull request and we'll get your issues fixed. Thanks, Johnny Hughes

2 2

Xen 4.6.3-15 packages, including XSAs 216-219, 221-225 on their way through the build system
by George Dunlap 27 Jun '17

27 Jun '17

Xen 4.6.3-15 packages for CentOS 6 and CentOS 7 are on their way through the build system. They should show up in centos-virt-testing in a few hours, and in the main mirrors tomorrow morning (God willing). These contain several critical updates; users are encouraged to update as soon as possible. XSA 220 unfortunately had several dependencies on changes included in 4.6.4 and 4.6.5. This particular vulnerability is slightly lower priorty; so I have queued up an update to 4.6.5 which includes this patch. Once 4.6.3-15 is tagged for release, I'll start building 4.6.5 put it in centos-virt-testing for people to test. -George

5 4

Loss of DHCP destroys bridge
by Ken Bass 27 Jun '17

27 Jun '17

I am having trouble with recovery. Today due to electrical work I powered down my networks Router / DHCP server. My Centos 7 host machines lost their DHCP lease (they are actually static leases). Once I power my Router / DHCP server back up none of my virtual machine were accessible. It appears that when the DHCP lease was lost on the virt server the bridge of the KVM guest were all losts. A 'br show' had none of the guests anymore. So each guest was sending out DHCP requests as expected but since the guest was no longer bridged the packets went no-where. How do I handle proper recovery from this? The sequence on the host seemed to be: 07:00:00 I powered off the DHCP server and router for electrical work Jun 26 10:08:02 vm-server dhclient[973]: DHCPREQUEST on br0 to 192.168.2.1 port 67 (xid=0x7627186d) requests continue for quite some hours and then NetworkManager chimes in. Jun 26 14:55:23 vm-server NetworkManager[840]: <info> [1498503323.1518] dhcp4 (br0): state changed bound -> expire Jun 26 14:55:23 vm-server NetworkManager[840]: <info> [1498503323.1681] dhcp4 (br0): canceled DHCP transaction, DHCP client pid 973 Jun 26 14:55:23 vm-server NetworkManager[840]: <info> [1498503323.1681] dhcp4 (br0): state changed expire -> done Jun 26 14:55:23 vm-server NetworkManager[840]: <info> [1498503323.1684] device (br0): scheduling DHCPv4 restart in 120 seconds, 3 tries left (reason: lease expired) (looks like 3 tries of something) Jun 26 15:00:52 vm-server NetworkManager[840]: <warn> [1498503652.9531] dhcp4 (br0): request timed out Jun 26 15:00:52 vm-server NetworkManager[840]: <info> [1498503652.9532] dhcp4 (br0): state changed unknown -> timeout Jun 26 15:00:52 vm-server NetworkManager[840]: <info> [1498503652.9694] dhcp4 (br0): canceled DHCP transaction, DHCP client pid 15937 Jun 26 15:00:52 vm-server NetworkManager[840]: <info> [1498503652.9694] dhcp4 (br0): state changed timeout -> done Jun 26 15:00:52 vm-server NetworkManager[840]: <info> [1498503652.9697] device (br0): scheduling DHCPv4 restart in 120 seconds, 1 tries left (reason: lease expired) Jun 26 15:02:52 vm-server NetworkManager[840]: <info> [1498503772.9548] dhcp4 (br0): activation: beginning transaction (timeout in 45 seconds) Jun 26 15:02:52 vm-server NetworkManager[840]: <info> [1498503772.9566] dhcp4 (br0): dhclient started with pid 15994 Jun 26 15:02:52 vm-server dhclient[15994]: DHCPDISCOVER on br0 to 255.255.255.255 port 67 interval 4 (xid=0x54083ff1) Jun 26 15:03:37 vm-server NetworkManager[840]: <warn> [1498503817.9818] dhcp4 (br0): request timed out Jun 26 15:03:37 vm-server NetworkManager[840]: <info> [1498503817.9819] dhcp4 (br0): state changed unknown -> timeout Jun 26 15:03:37 vm-server NetworkManager[840]: <info> [1498503817.9981] dhcp4 (br0): canceled DHCP transaction, DHCP client pid 15994 Jun 26 15:03:37 vm-server NetworkManager[840]: <info> [1498503817.9981] dhcp4 (br0): state changed timeout -> done Jun 26 15:03:37 vm-server NetworkManager[840]: <info> [1498503817.9984] device (br0): state change: activated -> failed (reason 'ip-config-unavailable') [100 120 5] Jun 26 15:03:37 vm-server NetworkManager[840]: <info> [1498503817.9987] manager: NetworkManager state is now CONNECTED_LOCAL Jun 26 15:03:38 vm-server NetworkManager[840]: <warn> [1498503818.0457] device (br0): Activation: failed for connection 'br0' Jun 26 15:03:38 vm-server kernel: device enp3s0 left promiscuous mode Jun 26 15:03:38 vm-server kernel: br0: port 1(enp3s0) entered disabled state Jun 26 15:03:38 vm-server NetworkManager[840]: <info> [1498503818.0473] device (br0): detached bridge port enp3s0 Jun 26 15:03:38 vm-server kernel: device vnet0 left promiscuous mode Jun 26 15:03:38 vm-server kernel: br0: port 2(vnet0) entered disabled state Jun 26 15:03:38 vm-server NetworkManager[840]: <info> [1498503818.0483] device (br0): detached bridge port vnet0 Jun 26 15:03:38 vm-server kernel: device vnet2 left promiscuous mode Jun 26 15:03:38 vm-server kernel: br0: port 3(vnet2) entered disabled state Jun 26 15:03:38 vm-server NetworkManager[840]: <info> [1498503818.0493] device (br0): detached bridge port vnet2 Jun 26 15:03:38 vm-server kernel: device vnet3 left promiscuous mode When I powered back on at: Jun 26 16:45:29 vm-server dhclient[26539]: DHCPOFFER from 192.168.2.1 Jun 26 16:45:29 vm-server dhclient[26539]: DHCPACK from 192.168.2.1 (xid=0x4c7b48be) None of the bridges that NetworkManager decided to detach were restored. Is this the case of being bitten by NetworkManager once again? Not sure why I am running it other than it is the default and I don't remember if it is responsible for setting up the bridges.

1 0

qemu-kvm-ev-2.6.0-28.el7_3.10.1 now available
by Sandro Bonazzola 23 Jun '17

23 Jun '17

Hi, qemu-kvm-ev-2.6.0-28.el7.10.1 <https://cbs.centos.org/koji/buildinfo?buildID=17495> has been tagged for release and will soon be available on CentOS mirrors. This release addresses a security issue (CVE-2017-7718) which has a security impact rated important. See https://www.redhat.com/archives/rhsa-announce/2017-June/msg00014.html for more details on this update. Here's the Changelog: * Fri Jun 23 2017 Sandro Bonazzola <sbonazzo(a)redhat.com> - ev-2.6.0-28.el7_3.10.1 - Removing RH branding from package name * Mon May 22 2017 Miroslav Rezanina <mrezanin(a)redhat.com> - rhev-2.6.0-28.el7_3.10 - kvm-virtio-rng-stop-virtqueue-while-the-CPU-is-stopped.patch [bz#1450375] - Resolves: bz#1450375 (Migration failed with postcopy enabled from rhel7.3.z host to rhel7.4 host "error while loading state for instance 0x0 of device 'pci) -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

2 3

xen4centos kernel version / debuginfo
by Sarah Newman 13 Jun '17

13 Jun '17

Is there any problem moving to 4.9.31? This contains upstream commits f2e767bb5d6ee0d9 for mpt3sas and 69861e0a52f87333 for dom0 memory mappings. I am also trying to build a debuginfo package but it's not correct. Specifically I'm able to get vmlinux but the .debug files for modules are stripped, which isn't right. Does anyone know how to fix this? I'm also not sure how you're supposed to build the kernel-firmware package with the current spec file definition. --Sarah

3 3

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

virt June 2017