virt November 2017

virt@lists.centos.org

10 participants
9 discussions

ovirt-engine package for oVirt 4.1.x
by C. L. Martinez 16 May '18

16 May '18

Hi all, I am trying to install oVirt 4.1.x from centos repos but it seems ovirt-engine doesn't exists. But instead, ovirt-hosted-engine-setup exists? Is ovirt-engine package removed? In official oVirt repos, exists. Thanks.

4 5

4.4.4-26 with XSA-226, 227, 230 in centos-virt-testing
by Kevin Stange 28 Nov '17

28 Nov '17

Xen 4.4.4 along with kernel 4.9.44 containing patches for XSAs (226 - 230) from August 15th are now available in centos-virt-testing. If possible, please test and provide feedback here so we can move these to release soon. XSA-228 did not affect Xen 4.4 XSA-229 only applies to the kernel XSA-235 disclosed today only affects ARM and isn't going to be added to these packages. Thanks. -- Kevin Stange Chief Technology Officer Steadfast | Managed Infrastructure, Datacenter and Cloud Services 800 S Wells, Suite 190 | Chicago, IL 60607 312.602.2689 X203 | Fax: 312.602.2688 kevin(a)steadfast.net | www.steadfast.net

3 4

Xen 4.6.6-7 packages in virt-testing
by George Dunlap 28 Nov '17

28 Nov '17

I've tagged the 4.6.6-7, which contain XSAs 246 and 247, in testing; they should show up in virt-testing soon. Please report any issues; I'll probably tag for release tomorrow (to show up Thursday). -George

1 0

Stability issues since moving to 4.6 - Kernel paging request bug + VM left in null state
by Nathan March 15 Nov '17

15 Nov '17

Since moving from 4.4 to 4.6, I've been seeing an increasing number of stability issues on our hypervisors. I'm not clear if there's a singular root cause here, or if I'm dealing with multiple bugs. One of the more common ones I've seen, is a VM on shutdown will remain in the null state and a kernel bug is thrown: xen001 log # xl list Name ID Mem VCPUs State Time(s) Domain-0 0 6144 24 r----- 6639.7 (null) 3 0 1 --pscd 36.3 [89920.839074] BUG: unable to handle kernel paging request at ffff88020ee9a000 [89920.839546] IP: [<ffffffff81430922>] __memcpy+0x12/0x20 [89920.839933] PGD 2008067 [89920.840022] PUD 17f43f067 [89920.840390] PMD 1e0976067 [89920.840469] PTE 0 [89920.840833] [89920.841123] Oops: 0000 [#1] SMP [89920.841417] Modules linked in: ebt_ip ebtable_filter ebtables arptable_filter arp_tables bridge xen_pciback xen_gntalloc nfsd auth_rpcgss nfsv3 nfs_acl nfs fscache lockd sunrpc grace 8021q mrp garp stp llc bonding xen_acpi_processor blktap xen_netback xen_blkback xen_gntdev xen_evtchn xenfs xen_privcmd dcdbas fjes pcspkr ipmi_devintf ipmi_si ipmi_msghandler joydev i2c_i801 i2c_smbus lpc_ich shpchp mei_me mei ioatdma ixgbe mdio igb dca ptp pps_core uas usb_storage wmi ttm [89920.847080] CPU: 4 PID: 1471 Comm: loop6 Not tainted 4.9.58-29.el6.x86_64 #1 [89920.847381] Hardware name: Dell Inc. PowerEdge C6220/03C9JJ, BIOS 2.7.1 03/04/2015 [89920.847893] task: ffff8801b75e0700 task.stack: ffffc900460e0000 [89920.848192] RIP: e030:[<ffffffff81430922>] [<ffffffff81430922>] __memcpy+0x12/0x20 [89920.848783] RSP: e02b:ffffc900460e3b20 EFLAGS: 00010246 [89920.849081] RAX: ffff88018916d000 RBX: ffff8801b75e0700 RCX: 0000000000000200 [89920.849384] RDX: 0000000000000000 RSI: ffff88020ee9a000 RDI: ffff88018916d000 [89920.849686] RBP: ffffc900460e3b38 R08: ffff88011da9fcf8 R09: 0000000000000002 [89920.849989] R10: ffff88019535bddc R11: ffffea0006245b5c R12: 0000000000001000 [89920.850294] R13: ffff88018916e000 R14: 0000000000001000 R15: ffffc900460e3b68 [89920.850605] FS: 00007fb865c30700(0000) GS:ffff880204b00000(0000) knlGS:0000000000000000 [89920.851118] CS: e033 DS: 0000 ES: 0000 CR0: 0000000080050033 [89920.851418] CR2: ffff88020ee9a000 CR3: 00000001ef03b000 CR4: 0000000000042660 [89920.851720] Stack: [89920.852009] ffffffff814375ca ffffc900460e3b38 ffffc900460e3d08 ffffc900460e3bb8 [89920.852821] ffffffff814381c5 ffffc900460e3b68 ffffc900460e3d08 0000000000001000 [89920.853633] ffffc900460e3d88 0000000000000000 0000000000001000 ffffea0000000000 [89920.854445] Call Trace: [89920.854741] [<ffffffff814375ca>] ? memcpy_from_page+0x3a/0x70 [89920.855043] [<ffffffff814381c5>] iov_iter_copy_from_user_atomic+0x265/0x290 [89920.855354] [<ffffffff811cf633>] generic_perform_write+0xf3/0x1d0 [89920.855673] [<ffffffff8101e39a>] ? xen_load_tls+0xaa/0x160 [89920.855992] [<ffffffffc025cf2b>] nfs_file_write+0xdb/0x200 [nfs] [89920.856297] [<ffffffff81269062>] vfs_iter_write+0xa2/0xf0 [89920.856599] [<ffffffff815fa365>] lo_write_bvec+0x65/0x100 [89920.856899] [<ffffffff815fc375>] do_req_filebacked+0x195/0x300 [89920.857202] [<ffffffff815fc53b>] loop_queue_work+0x5b/0x80 [89920.857505] [<ffffffff810c6898>] kthread_worker_fn+0x98/0x1b0 [89920.857808] [<ffffffff818d9dca>] ? schedule+0x3a/0xa0 [89920.858108] [<ffffffff818ddbb6>] ? _raw_spin_unlock_irqrestore+0x16/0x20 [89920.858411] [<ffffffff810c6800>] ? kthread_probe_data+0x40/0x40 [89920.858713] [<ffffffff810c63f5>] kthread+0xe5/0x100 [89920.859014] [<ffffffff810c6310>] ? __kthread_init_worker+0x40/0x40 [89920.859317] [<ffffffff818de2d5>] ret_from_fork+0x25/0x30 [89920.859615] Code: 81 f3 00 00 00 00 e9 1e ff ff ff 90 90 90 90 90 90 90 90 90 90 90 90 90 90 66 66 90 66 90 48 89 f8 48 89 d1 48 c1 e9 03 83 e2 07 <f3> 48 a5 89 d1 f3 a4 c3 66 0f 1f 44 00 00 48 89 f8 48 89 d1 f3 [89920.864410] RIP [<ffffffff81430922>] __memcpy+0x12/0x20 [89920.864749] RSP <ffffc900460e3b20> [89920.865021] CR2: ffff88020ee9a000 [89920.865294] ---[ end trace b77d2ce5646284d1 ]--- Wondering if anyone has advice on how to troubleshoot the above, or might have some insight into that the issue could be? This hypervisor was only up for a day, had almost no VMs running on it since boot, I booted a single windows test VM which BSOD'ed and then this happened. This is on xen 4.6.6-4.el6 with 4.9.58-29.el6.x86_64. I see these issues across a wide number of systems with from both Dell and Supermicro, although we run the same Intel x540 10gb nic's in each system with the same netapp nfs backend storage. Cheers, Nathan

3 3

Live migration haswell, broadwell
by T.Weyergraf 14 Nov '17

14 Nov '17

Hi I wonder, if live migration (back and forth) is possible on mixed Haswell (Xeon V3) and Broadwell (Xeon V4) installations. The only notable difference between the two is apparently a working TSX implementation on V4, which got disabled on V3 due to bugs. The rest (VMCS-shadowing, posted interrupts) should not apply to our environment, as we do not run nested-vmx nor device-passthrough on our Xen servers. Now, I found no sane way to disable TSX on a given system but I cannot rule out, that some (Linux-)software, such as Postgres will use them eventually. Also, I have a hard time trying to assess, if TSX can be disabled on V4 to enable seamless migration. Any hint would be greately appreciated.

2 1

Crash in network stack under Xen
by Sarah Newman 13 Nov '17

13 Nov '17

Hi, We had a potentially network related crash on a dom0 with Linux 4.9.39 / Xen 4.8 and as of today I can't find any fixes in stable/linux-4.9.y, xen/staging-4.8, or CPU microcode updates that look like a smoking gun. I can't rule out that it's Xen related. The backtraces are: ------------[ cut here ]------------ WARNING: CPU: 0 PID: 0 at net/ipv4/af_inet.c:1473 inet_gro_complete+0xbb/0xd0 Call Trace: <IRQ> dump_stack+0x63/0x8e __warn+0xd1/0xf0 warn_slowpath_null+0x1d/0x20 inet_gro_complete+0xbb/0xd0 napi_gro_complete+0x73/0xa0 napi_gro_flush+0x5f/0x80 napi_complete_done+0x6a/0xb0 igb_poll+0x38d/0x720 [igb] ? igb_msix_ring+0x2e/0x40 [igb] ? __handle_irq_event_percpu+0x4b/0x1a0 net_rx_action+0x158/0x360 __do_softirq+0xd1/0x283 irq_exit+0xe9/0x100 xen_evtchn_do_upcall+0x35/0x50 xen_do_hypervisor_callback+0x1e/0x40 <EOI> ? xen_hypercall_sched_op+0xa/0x20 ? xen_hypercall_sched_op+0xa/0x20 ? xen_safe_halt+0x10/0x20 ? default_idle+0x1e/0xd0 ? arch_cpu_idle+0xf/0x20 ? default_idle_call+0x2c/0x40 ? cpu_startup_entry+0x1ac/0x240 ? rest_init+0x77/0x80 ? start_kernel+0x4a7/0x4b4 ? set_init_arg+0x55/0x55 ? x86_64_start_reservations+0x24/0x26 ? xen_start_kernel+0x555/0x561 general protection fault: 0000 [#1] SMP Call Trace: <IRQ> ? napi_gro_complete+0x5e/0xa0 skb_release_all+0x24/0x30 kfree_skb+0x32/0x90 napi_gro_complete+0x5e/0xa0 napi_gro_flush+0x5f/0x80 napi_complete_done+0x6a/0xb0 igb_poll+0x38d/0x720 [igb] ? igb_msix_ring+0x2e/0x40 [igb] ? __handle_irq_event_percpu+0x4b/0x1a0 net_rx_action+0x158/0x360 __do_softirq+0xd1/0x283 irq_exit+0xe9/0x100 xen_evtchn_do_upcall+0x35/0x50 xen_do_hypervisor_callback+0x1e/0x40 <EOI> ? xen_hypercall_sched_op+0xa/0x20 ? xen_hypercall_sched_op+0xa/0x20 ? xen_safe_halt+0x10/0x20 ? default_idle+0x1e/0xd0 ? arch_cpu_idle+0xf/0x20 ? default_idle_call+0x2c/0x40 ? cpu_startup_entry+0x1ac/0x240 ? rest_init+0x77/0x80 ? start_kernel+0x4a7/0x4b4 ? set_init_arg+0x55/0x55 ? x86_64_start_reservations+0x24/0x26 ? xen_start_kernel+0x555/0x561 RIP skb_release_data+0x73/0xf0 Kernel panic - not syncing: Fatal exception in interrupt Kernel Offset: disabled (XEN) Hardware Dom0 crashed: rebooting machine in 5 seconds. If anyone has had a similar backtrace or knows of a potential fix please respond. This server has ECC and there were no ECC or other errors in the BIOS event log, nor were there any indications of any problems in the serial console log leading up to the warning. This particular server had an uptime of about a month and a half, and so far we've had this error exactly once across all our servers since switching to 4.9.39 in August, so I don't think it's going to be easy to reproduce. --- It looks to me like in the first backtrace, this check from inet_gro_complete failed: ops = rcu_dereference(inet_offloads[proto]); Which I'm guessing means the packet didn't have a valid layer 4 protocol definition, or we don't have that protocol enabled. Then when attempting to handle that failure there was a GPF, I believe by accessing invalid data in shinfo->frag_list . "skb_release_data+0x73" is in __read_once_size, which I think is generated by "kfree_skb: if (likely(atomic_read(&skb->users) == 1))" . --Sarah

2 1

Non Marketplace AMI for 1708
by Andrew Jeffree 13 Nov '17

13 Nov '17

Hi, Currently, there are some images that are published outside of the AWS Market Place [0] for 1703 are there plans to have updated ones available for 1708? Kind Regards, Andrew [0] - https://wiki.centos.org/Cloud/AWS#head-78d1e3a4e6ba5c5a3847750d88266916ffe6…

1 0

CentOS 7 x86_64 with Updates HVM not enabled for new c5.* instances, ENA support tagged incorrectly
by Chris Merrett 10 Nov '17

10 Nov '17

Hi, We’ve been looking to try out the c5.* instances with the latest AMI for the aforementioned image within us-east-1, but this will fail due to both the allowed instance list not containing c5.* and the ENA support flag being set to false. ENA support shipped with CentOS 7.4 and should now be fine to be enabled within the AMI metadata, which judging by a similar thread in July this year (https://lists.centos.org/pipermail/centos-virt/2017-July/005592.html <https://lists.centos.org/pipermail/centos-virt/2017-July/005592.html>) could also enable support for the c5.* instances where ENA is a requirement. Would it be possible to enable ENA support for this image, as well as enable support for c5.* instances if that doesn’t happen automatically? Thanks for everything, your work is very much appreciated. Regards, Chris Merrett

1 0

OT?: NetBSD domU on linux dom0 (XSA-240?)
by Tru Huynh 06 Nov '17

06 Nov '17

Hi No 1st hand experience, just passing the information... If some of you are hosting NetBSD on Xenserver, that might impact: http://mail-index.netbsd.org/port-xen/2017/10/23/msg009097.html and the work around listed in http://mail-index.netbsd.org/port-xen/2017/10/23/msg009098.html "I guess this is the patch from XSA-240, you need to boot with pv-linear-pt=true on the Xen command line" Cheers Tru -- Tru Huynh http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xBEFA581B

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

virt November 2017