Further to my messages back in May I have at last got round to trying to get my DomU to recognise USB devices.
I am using Xen 4.6 with CentOS kernel 3.18.34-20.el7.x86_64. I have to manually make the port available before creating the DomU by issuing the command: xl pci-assignable-add 00:1a.0 otherwise nothing shows in: xl pci-assignable-list
I have added this to my .cfg file as per the May Emails: pci=['00:1a.0,rdm_policy=relaxed']
and in the DomU, which starts fine, I get the following information: lspci 00:00.0 USB controller: Intel Corporation Wellsburg USB Enhanced Host Controller #2 (rev 05) lsusb Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
However when I plug in a device to this USB port (Am sure it is the correct port in the Dom0) I see nothing in the DomU at all.
What can I do now? Many thanks Francis
More information... I have pcifront showing as a module in the DomU and the usb shows in dmesg as: [ 3.167543] usbcore: registered new interface driver usbfs [ 3.167563] usbcore: registered new interface driver hub [ 3.167585] usbcore: registered new device driver usb [ 3.196056] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002 [ 3.196060] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 3.196064] usb usb1: Product: EHCI Host Controller [ 3.196068] usb usb1: Manufacturer: Linux 3.2.0-4-686-pae ehci_hcd [ 3.196071] usb usb1: SerialNumber: 0000:00:00.0 [ 3.508036] usb 1-1: new high-speed USB device number 2 using ehci_hcd [ 19.064072] usb 1-1: device not accepting address 2, error -110 [ 19.176070] usb 1-1: new high-speed USB device number 3 using ehci_hcd [ 34.732067] usb 1-1: device not accepting address 3, error -110 [ 34.844082] usb 1-1: new high-speed USB device number 4 using ehci_hcd [ 45.280073] usb 1-1: device not accepting address 4, error -110 [ 45.392067] usb 1-1: new high-speed USB device number 5 using ehci_hcd [ 55.824112] usb 1-1: device not accepting address 5, error -110
I am looking at xl dmesg in Dom0 where there are some messages relating to the PCI usb: [VT-D] It's disallowed to assign 0000:00:1a.0 with shared RMRR at 7b800000 for Dom6. (XEN) XEN_DOMCTL_assign_device: assign 0000:00:1a.0 to dom6 failed (-1) (XEN) [VT-D] It's risky to assign 0000:00:1a.0 with shared RMRR at 7b800000 for Dom7. (XEN) [VT-D] It's risky to assign 0000:00:1a.0 with shared RMRR at 7b800000 for Dom8. (XEN) [VT-D] It's risky to assign 0000:00:1a.0 with shared RMRR at 7b800000 for Dom9. (XEN) [VT-D] It's risky to assign 0000:00:1a.0 with shared RMRR at 7b800000 for Dom10.
In the as an aside... I just get blocks on the screen after the scrubbing message, and no text. I see there is a message: (XEN) Xen is relinquishing VGA console. How can I prevent this? Is there something wrong with my /etc/default/grub
... GRUB_CMDLINE_LINUX="crashkernel=auto intremap=no_x2apic_optout" GRUB_CMDLINE_XEN_DEFAULT="dom0_mem=13312M,max:14336M dom0_max_vcpus=6 dom0_vcpus_pin" GRUB_GFXMODE=1024x768 GRUB_GFXPAYLOAD_LINUX=keep GRUB_CMDLINE_LINUX_XEN_REPLACE_DEFAULT="console=hvc0 earlyprintk=xen"
Many thanks Francis
From: "Francis Greaves" francis@choughs.net To: "centos-virt" centos-virt@centos.org Sent: Wednesday, 22 June, 2016 09:56:44 Subject: [CentOS-virt] PCI Passthrough not working
Further to my messages back in May I have at last got round to trying to get my DomU to recognise USB devices.
I am using Xen 4.6 with CentOS kernel 3.18.34-20.el7.x86_64. I have to manually make the port available before creating the DomU by issuing the command: xl pci-assignable-add 00:1a.0 otherwise nothing shows in: xl pci-assignable-list
I have added this to my .cfg file as per the May Emails: pci=['00:1a.0,rdm_policy=relaxed']
and in the DomU, which starts fine, I get the following information: lspci 00:00.0 USB controller: Intel Corporation Wellsburg USB Enhanced Host Controller #2 (rev 05) lsusb Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
However when I plug in a device to this USB port (Am sure it is the correct port in the Dom0) I see nothing in the DomU at all.
What can I do now? Many thanks Francis
_______________________________________________ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
On Wed, Jun 22, 2016 at 11:49 AM, Francis Greaves francis@choughs.net wrote:
More information... I have pcifront showing as a module in the DomU and the usb shows in dmesg as: [ 3.167543] usbcore: registered new interface driver usbfs [ 3.167563] usbcore: registered new interface driver hub [ 3.167585] usbcore: registered new device driver usb [ 3.196056] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002 [ 3.196060] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 3.196064] usb usb1: Product: EHCI Host Controller [ 3.196068] usb usb1: Manufacturer: Linux 3.2.0-4-686-pae ehci_hcd [ 3.196071] usb usb1: SerialNumber: 0000:00:00.0 [ 3.508036] usb 1-1: new high-speed USB device number 2 using ehci_hcd [ 19.064072] usb 1-1: device not accepting address 2, error -110 [ 19.176070] usb 1-1: new high-speed USB device number 3 using ehci_hcd [ 34.732067] usb 1-1: device not accepting address 3, error -110 [ 34.844082] usb 1-1: new high-speed USB device number 4 using ehci_hcd [ 45.280073] usb 1-1: device not accepting address 4, error -110 [ 45.392067] usb 1-1: new high-speed USB device number 5 using ehci_hcd [ 55.824112] usb 1-1: device not accepting address 5, error -110
Can you post your question with your guest config file, lspci in dom0, and lspci in your domU to xen-users? There will be a lot more eyeballs there to help you get things sorted out.
I am looking at xl dmesg in Dom0 where there are some messages relating to the PCI usb: [VT-D] It's disallowed to assign 0000:00:1a.0 with shared RMRR at 7b800000 for Dom6. (XEN) XEN_DOMCTL_assign_device: assign 0000:00:1a.0 to dom6 failed (-1) (XEN) [VT-D] It's risky to assign 0000:00:1a.0 with shared RMRR at 7b800000 for Dom7. (XEN) [VT-D] It's risky to assign 0000:00:1a.0 with shared RMRR at 7b800000 for Dom8. (XEN) [VT-D] It's risky to assign 0000:00:1a.0 with shared RMRR at 7b800000 for Dom9. (XEN) [VT-D] It's risky to assign 0000:00:1a.0 with shared RMRR at 7b800000 for Dom10.
This looks like you tried once to start your guest without "rdm_policy=relaxed" (which failed), and then tried it four times with "rdm_policy=relaxed" (which succeeded). Other than warning that there's a shared RMRR (which could potentially be a security risk), everything here looks normal.
There's a possibility that the shared RMRR is what's tripping things up, but it's not very likely.
In the as an aside... I just get blocks on the screen after the scrubbing message, and no text. I see there is a message: (XEN) Xen is relinquishing VGA console. How can I prevent this? Is there something wrong with my /etc/default/grub
... GRUB_CMDLINE_LINUX="crashkernel=auto intremap=no_x2apic_optout" GRUB_CMDLINE_XEN_DEFAULT="dom0_mem=13312M,max:14336M dom0_max_vcpus=6 dom0_vcpus_pin" GRUB_GFXMODE=1024x768 GRUB_GFXPAYLOAD_LINUX=keep GRUB_CMDLINE_LINUX_XEN_REPLACE_DEFAULT="console=hvc0 earlyprintk=xen"
Could you post this as a separate message to xen-users?
Thanks. -George
Here is my post issued again from the beginning in some sort of logical order I hope, with additional information as suggested by George Dunlap.
I am having trouble getting PCI Passthrough to work from Dom0 to DomU I am using Xen 4.6 with CentOS kernel 3.18.34-20.el7.x86_64 on a Dell Poweredge T430. When I plug in a device to the USB port, nothing happens. I am Watching /var/log/messages in the DomU. Nothing
Here is my lspci on the Dom0 filtered to show USB and PCI devices
00:1a.0 USB controller: Intel Corporation C610/X99 series chipset USB Enhanced Host Controller #2 (rev 05) 00:1d.0 USB controller: Intel Corporation C610/X99 series chipset USB Enhanced Host Controller #1 (rev 05)
00:02.0 PCI bridge: Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 PCI Express Root Port 2 (rev 02) 00:03.0 PCI bridge: Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 PCI Express Root Port 3 (rev 02) 00:1c.0 PCI bridge: Intel Corporation C610/X99 series chipset PCI Express Root Port #1 (rev d5) 00:1c.1 PCI bridge: Intel Corporation C610/X99 series chipset PCI Express Root Port #2 (rev d5) 00:1c.2 PCI bridge: Intel Corporation C610/X99 series chipset PCI Express Root Port #3 (rev d5) 00:1c.4 PCI bridge: Intel Corporation C610/X99 series chipset PCI Express Root Port #5 (rev d5) 01:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet PCIe 01:00.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet PCIe 04:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet PCIe 04:00.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet PCIe 05:00.0 PCI bridge: Renesas Technology Corp. Device 001d 06:00.0 PCI bridge: Renesas Technology Corp. Device 001d 07:00.0 PCI bridge: Renesas Technology Corp. Device 001a 0a:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01) 0a:00.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01) 0a:00.2 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01) 0a:00.3 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01) 7f:10.0 System peripheral: Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 PCIe Ring Interface (rev 02) 7f:10.1 Performance counters: Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 PCIe Ring Interface (rev 02) 80:02.0 PCI bridge: Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 PCI Express Root Port 2 (rev 02) 80:02.2 PCI bridge: Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 PCI Express Root Port 2 (rev 02) ff:10.0 System peripheral: Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 PCIe Ring Interface (rev 02) ff:10.1 Performance counters: Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 PCIe Ring Interface (rev 02)
Here is my lspci on the DomU
00:00.0 USB controller: Intel Corporation Wellsburg USB Enhanced Host Controller #2 (rev 05)
Prior to starting the DomU I issue command:
xl pci-assignable-add 00:1a.0 xl pci-assignable-list 0000:00:1a.0
So this is OK
Now for the config file for the DomU
# Guest name ============================================================== name = "metsat.fsoft.nnet"
# Kernel command line options extra = "root=/dev/xvda1 swiotlb=force"
# Initial memory allocation (MB) memory = 2048
# Number of VCPUS vcpus = 2
# two ethernet devices, one for the network, one for the Eumetcast receiver vif = ['mac=00:16:3E:00:00:35, bridge=xenbr5', 'mac=00:16:3E:00:00:36, bridge=xenbr6']
# Disk Devices disk = ['phy:/dev/xen_vg/metsat_disk,xvda,w', 'phy:/dev/xen_vg/metsat_swap,xvdb,w', 'phy:/dev/xen_vg/metsat_receive,xvdc,w']
# for Eumetcast Dongle pci=['00:1a.0,rdm_policy=relaxed,permissive=1']
on_poweroff = 'destroy' on_reboot = 'restart' on_crash = 'restart'
# Run section ============================================================== bootloader = "/usr/lib/xen/bin/pygrub" ==============================================================
I have pcifront showing as a module in the DomU and the usb shows in dmesg as: [ 3.167543] usbcore: registered new interface driver usbfs [ 3.167563] usbcore: registered new interface driver hub [ 3.167585] usbcore: registered new device driver usb [ 3.196056] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002 [ 3.196060] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 3.196064] usb usb1: Product: EHCI Host Controller [ 3.196068] usb usb1: Manufacturer: Linux 3.2.0-4-686-pae ehci_hcd [ 3.196071] usb usb1: SerialNumber: 0000:00:00.0 [ 3.508036] usb 1-1: new high-speed USB device number 2 using ehci_hcd [ 19.064072] usb 1-1: device not accepting address 2, error -110 [ 19.176070] usb 1-1: new high-speed USB device number 3 using ehci_hcd [ 34.732067] usb 1-1: device not accepting address 3, error -110 [ 34.844082] usb 1-1: new high-speed USB device number 4 using ehci_hcd [ 45.280073] usb 1-1: device not accepting address 4, error -110 [ 45.392067] usb 1-1: new high-speed USB device number 5 using ehci_hcd [ 55.824112] usb 1-1: device not accepting address 5, error -110
Can anyone help sort this out, so I can get USB devices to be recognised. I want to plug in my Eumetsat Dongle for receiving Weather Satellite images.
Regards Francis
Further to my last post, I have removed the xen-pciback module from the Dom0 kernel, and reloaded it as modprobe xen-pciback passthrough=1
I now have the PCI device on the DomU matching the Dom0 Device usb usb1: SerialNumber: 0000:00:1a.0 instead of 0000:00:00.0
However I now have this error
ehci_hcd 0000:00:1a.0: Unlink after no-IRQ? Controller is probably using the wrong IRQ.
does this give a clue as to what is going on?
Many thanks Francis
From: "Francis Greaves" francis@choughs.net To: "Francis Greaves" francis@choughs.net, "centos-virt" centos-virt@centos.org Sent: Sunday, 3 July, 2016 11:19:49 Subject: Re: [CentOS-virt] PCI Passthrough not working
Further to my last post, I have removed the xen-pciback module from the Dom0 kernel, and reloaded it as modprobe xen-pciback passthrough=1
I now have the PCI device on the DomU matching the Dom0 Device usb usb1: SerialNumber: 0000:00:1a.0 instead of 0000:00:00.0
However I now have this error
ehci_hcd 0000:00:1a.0: Unlink after no-IRQ? Controller is probably using the wrong IRQ.
does this give a clue as to what is going on?
Many thanks Francis
More information: In the Dom0 I get this when booting the DomU, even with irwpoll in the DomU kernel line
xen_pciback: xen-pciback[0000:00:1a.0] IRQ line is not shared with other domains. Turning ISR off Jul 03 11:31:23 antares.fsoft.nnet kernel: irq 18: nobody cared (try booting with the "irqpoll" option) Jul 03 11:31:23 antares.fsoft.nnet kernel: CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.18.34-20.el7.x86_64 #1 Jul 03 11:31:23 antares.fsoft.nnet kernel: Hardware name: Dell Inc. PowerEdge T430/0975F3, BIOS 1.5.4 10/05/2015 Jul 03 11:31:23 antares.fsoft.nnet kernel: 0000000000000000 ffff8803bc603d88 ffffffff81653783 ffff8803b223e400 Jul 03 11:31:23 antares.fsoft.nnet kernel: ffff8803b223e48c ffff8803bc603db8 ffffffff810c6776 ffff8803bc613340 Jul 03 11:31:23 antares.fsoft.nnet kernel: ffff8803b223e400 0000000000000012 0000000000000000 ffff8803bc603e08 Jul 03 11:31:23 antares.fsoft.nnet kernel: Call Trace: Jul 03 11:31:23 antares.fsoft.nnet kernel: <IRQ> [<ffffffff81653783>] dump_stack+0x64/0x82 Jul 03 11:31:23 antares.fsoft.nnet kernel: [<ffffffff810c6776>] __report_bad_irq+0x36/0xd0 Jul 03 11:31:23 antares.fsoft.nnet kernel: [<ffffffff810c6c86>] note_interrupt+0x226/0x270 Jul 03 11:31:23 antares.fsoft.nnet kernel: [<ffffffff813e71cc>] ? add_interrupt_randomness+0x3c/0x1f0 Jul 03 11:31:23 antares.fsoft.nnet kernel: [<ffffffff810c43ec>] handle_irq_event_percpu+0xcc/0x1e0 Jul 03 11:31:23 antares.fsoft.nnet kernel: [<ffffffff810c453b>] handle_irq_event+0x3b/0x60
Message from syslogd@antares at Jul 3 11:31:23 ... kernel:Disabling IRQ #18 Jul 03 11:31:23 antares.fsoft.nnet kernel: [<ffffffff810c709a>] handle_fasteoi_irq+0x7a/0x130 Jul 03 11:31:23 antares.fsoft.nnet kernel: [<ffffffff810c394b>] generic_handle_irq+0x2b/0x40 Jul 03 11:31:23 antares.fsoft.nnet kernel: [<ffffffff813a2102>] evtchn_fifo_handle_events+0x162/0x170 Jul 03 11:31:23 antares.fsoft.nnet kernel: [<ffffffff8139efb0>] __xen_evtchn_do_upcall+0x50/0x90 Jul 03 11:31:23 antares.fsoft.nnet kernel: [<ffffffff813a0d67>] xen_evtchn_do_upcall+0x37/0x50 Jul 03 11:31:23 antares.fsoft.nnet kernel: [<ffffffff8165c05e>] xen_do_hypervisor_callback+0x1e/0x40 Jul 03 11:31:23 antares.fsoft.nnet kernel: <EOI> [<ffffffff810013aa>] ? xen_hypercall_sched_op+0xa/0x20 Jul 03 11:31:23 antares.fsoft.nnet kernel: [<ffffffff810013aa>] ? xen_hypercall_sched_op+0xa/0x20 Jul 03 11:31:23 antares.fsoft.nnet kernel: [<ffffffff8100b330>] ? xen_safe_halt+0x10/0x20 Jul 03 11:31:23 antares.fsoft.nnet kernel: [<ffffffff8101ea94>] ? default_idle+0x24/0xf0 Jul 03 11:31:23 antares.fsoft.nnet kernel: [<ffffffff8101f49f>] ? arch_cpu_idle+0xf/0x20 Jul 03 11:31:23 antares.fsoft.nnet kernel: [<ffffffff810adc92>] ? cpu_startup_entry+0x312/0x3e0 Jul 03 11:31:23 antares.fsoft.nnet kernel: [<ffffffff816445c7>] ? rest_init+0x77/0x80 Jul 03 11:31:23 antares.fsoft.nnet kernel: [<ffffffff81d77130>] ? start_kernel+0x4d0/0x4dd Jul 03 11:31:23 antares.fsoft.nnet kernel: [<ffffffff81d76a50>] ? set_init_arg+0x55/0x55 Jul 03 11:31:23 antares.fsoft.nnet kernel: [<ffffffff81d765ee>] ? x86_64_start_reservations+0x2a/0x2c Jul 03 11:31:23 antares.fsoft.nnet kernel: [<ffffffff81d7a7cc>] ? xen_start_kernel+0x5a9/0x5b5 Jul 03 11:31:23 antares.fsoft.nnet kernel: handlers: Jul 03 11:31:23 antares.fsoft.nnet kernel: [<ffffffff814b0f50>] usb_hcd_irq Jul 03 11:31:23 antares.fsoft.nnet kernel: [<ffffffffa050d3e0>] xen_pcibk_guest_interrupt [xen_pciback] Jul 03 11:31:23 antares.fsoft.nnet kernel: Disabling IRQ #18
=========================================================================
does that help? Francis
On Sun, Jul 3, 2016 at 12:34 PM, Francis Greaves francis@choughs.net wrote:
From: "Francis Greaves" francis@choughs.net To: "Francis Greaves" francis@choughs.net, "centos-virt" centos-virt@centos.org Sent: Sunday, 3 July, 2016 11:19:49 Subject: Re: [CentOS-virt] PCI Passthrough not working
Further to my last post, I have removed the xen-pciback module from the Dom0 kernel, and reloaded it as modprobe xen-pciback passthrough=1
I now have the PCI device on the DomU matching the Dom0 Device usb usb1: SerialNumber: 0000:00:1a.0 instead of 0000:00:00.0
However I now have this error
ehci_hcd 0000:00:1a.0: Unlink after no-IRQ? Controller is probably using the wrong IRQ.
does this give a clue as to what is going on?
Francis,
Thanks for posting with the extra info. Could you now also re-post it to the xen-users mailing list, as I asked, so that more people can get a look at what you're doing?
Thanks. :-)
-George
On Wed, Jun 22, 2016 at 10:56 AM, Francis Greaves francis@choughs.net wrote:
Further to my messages back in May I have at last got round to trying to get my DomU to recognise USB devices.
I am using Xen 4.6 with CentOS kernel 3.18.34-20.el7.x86_64. I have to manually make the port available before creating the DomU by issuing the command: xl pci-assignable-add 00:1a.0 otherwise nothing shows in: xl pci-assignable-list
I have added this to my .cfg file as per the May Emails: pci=['00:1a.0,rdm_policy=relaxed']
The two-stage process for assigning pci devices (first pci-assignible-add, then pci-add) is a "seatbelt" to make sure that an accidental mis-type doesn't cause you to grab (say) your hard disk controller instead of your USB controller.
You can add "seize=1" to your pci string to have xl automatically do both steps for you. Obviously, use this with care. :-)
More on your next post...
-George
-
Francis Greaves -
George Dunlap