Howdy,
I hit a snag trying to install Xen4CentOS on a Supermicro based system (X9DRD-7LN4F with the Broadcom/LSI 2308 chipset). I spent a few hours on this today, I'm posting this here in case it helps anyone else and saves them the frustration I dealt with.
On this system I did a fresh install of CentOS 7, updated it, rebooted it, then installed Xen.
The system was booting fine using the latest 3.10 kernel (3.10.0-514.16 at the time of writing this). But, when it tried to boot the kernel for Xen (4.9.25-27) it failed to do so.
On first boot all it showed was a blinking cursor after grub attempted to boot the 4.9 kernel, and it sat there indefinitely.
After some digging around I removed the "console=hvc0" setting /etc/default/grub, ran grub2-mkconfig, and tried booting 4.9 again. This time it actually showed messages, leading up to this:
[ 186.550326] dracut-initqueue[542]: Warning: dracut-iniqueue timeout - starting timeout scripts [ 186.570380] dracut-initqueue[542]: Warning: dracut-iniqueue timeout - starting timeout scripts [ 187.120156] dracut-initqueue[542]: Warning: dracut-iniqueue timeout - starting timeout scripts [ 188.633714] dracut-initqueue[542]: Warning: dracut-iniqueue timeout - starting timeout scripts [ 189.102305] dracut-initqueue[542]: Warning: dracut-iniqueue timeout - starting timeout scripts [...] [ OK ] Started dracut initqueue hook [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ ***] A start job is running for dev-disk...device (3min 35s / no limit)
I let this run for a while to see if anything changed (needed to turn my attention elsewhere). After a couple of hours it was still in the same place.
This last line was confusing:
[ ***] A start job is running for dev-disk...device (3min 35s / no limit)
Had the console not truncated that line and instead just wrapped the full message so I could see it I would have been a much happier person.
Turns out dracut was unable to mount the root file system. So I went back into the 3.10 kernel again to see if the mpt2sas or mpt3sas driver was in its initramfs file... and it wasn't:
$ sudo lsinitrd -k 4.9.25-27.el7.x86_64 | grep mpt -rw-r--r-- 1 root root 0 May 16 12:39 etc/fstab.empty -rw-r--r-- 1 root root 22 Nov 5 2016 usr/lib/kbd/unimaps/empty.uni
For comparison:
$ sudo lsinitrd -k 3.10.0-514.16.1.el7.x86_64 | grep mpt -rw-r--r-- 1 root root 0 May 16 04:37 etc/fstab.empty -rw-r--r-- 1 root root 22 Nov 5 2016 usr/lib/kbd/unimaps/empty.uni drwxr-xr-x 2 root root 0 May 16 04:37 usr/lib/modules/3.10.0-514.16.1.el7.x86_64/kernel/drivers/scsi/mpt3sas -rw-r--r-- 1 root root 379021 Apr 12 08:51 usr/lib/modules/3.10.0-514.16.1.el7.x86_64/kernel/drivers/scsi/mpt3sas/mpt2sas.ko
So I added it:
$ sudo dracut --force --add-drivers mpt3sas --kver=4.9.25-27.el7.x86_64 $ sudo lsinitrd -k 4.9.25-27.el7.x86_64 | grep mpt Arguments: --force --add-drivers 'mpt3sas' --kver '4.9.25-27.el7.x86_64' -rw-r--r-- 1 root root 0 May 16 12:57 etc/fstab.empty -rw-r--r-- 1 root root 22 Nov 5 2016 usr/lib/kbd/unimaps/empty.uni drwxr-xr-x 2 root root 0 May 16 12:57 usr/lib/modules/4.9.25-27.el7.x86_64/kernel/drivers/scsi/mpt3sas -rwxr--r-- 1 root root 374152 May 16 12:57 usr/lib/modules/4.9.25-27.el7.x86_64/kernel/drivers/scsi/mpt3sas/mpt3sas.ko
After this I was able to get the 4.9 kernel to boot and Xen is now working.
Jerry
On 05/16/2017 09:04 PM, Jerry wrote: <snip>
Turns out dracut was unable to mount the root file system. So I went back into the 3.10 kernel again to see if the mpt2sas or mpt3sas driver was in its initramfs file... and it wasn't:
$ sudo lsinitrd -k 4.9.25-27.el7.x86_64 | grep mpt -rw-r--r-- 1 root root 0 May 16 12:39 etc/fstab.empty -rw-r--r-- 1 root root 22 Nov 5 2016 usr/lib/kbd/unimaps/empty.uni
For comparison:
$ sudo lsinitrd -k 3.10.0-514.16.1.el7.x86_64 | grep mpt -rw-r--r-- 1 root root 0 May 16 04:37 etc/fstab.empty -rw-r--r-- 1 root root 22 Nov 5 2016 usr/lib/kbd/unimaps/empty.uni drwxr-xr-x 2 root root 0 May 16 04:37 usr/lib/modules/3.10.0-514.16.1.el7.x86_64/kernel/drivers/scsi/mpt3sas -rw-r--r-- 1 root root 379021 Apr 12 08:51 usr/lib/modules/3.10.0-514.16.1.el7.x86_64/kernel/drivers/scsi/mpt3sas/mpt2sas.ko
So I added it:
$ sudo dracut --force --add-drivers mpt3sas --kver=4.9.25-27.el7.x86_64 $ sudo lsinitrd -k 4.9.25-27.el7.x86_64 | grep mpt Arguments: --force --add-drivers 'mpt3sas' --kver '4.9.25-27.el7.x86_64' -rw-r--r-- 1 root root 0 May 16 12:57 etc/fstab.empty -rw-r--r-- 1 root root 22 Nov 5 2016 usr/lib/kbd/unimaps/empty.uni drwxr-xr-x 2 root root 0 May 16 12:57 usr/lib/modules/4.9.25-27.el7.x86_64/kernel/drivers/scsi/mpt3sas -rwxr--r-- 1 root root 374152 May 16 12:57 usr/lib/modules/4.9.25-27.el7.x86_64/kernel/drivers/scsi/mpt3sas/mpt3sas.ko
After this I was able to get the 4.9 kernel to boot and Xen is now working.
You should be able to make this change permanently from /etc/dracut.conf with the add_drivers+= line, or build a generic initrd with hostonly="no" .
I don't know why it wasn't originally added.
--Sarah
On Wed, May 17, 2017 at 10:07 AM, Sarah Newman srn@prgmr.com wrote:
On 05/16/2017 09:04 PM, Jerry wrote:
<snip> > > Turns out dracut was unable to mount the root file system. So I went back > into the 3.10 kernel again to see if the mpt2sas or mpt3sas driver was in > its initramfs file... and it wasn't: > > > $ sudo lsinitrd -k 4.9.25-27.el7.x86_64 | grep mpt > -rw-r--r-- 1 root root 0 May 16 12:39 etc/fstab.empty > -rw-r--r-- 1 root root 22 Nov 5 2016 > usr/lib/kbd/unimaps/empty.uni > > For comparison: > > $ sudo lsinitrd -k 3.10.0-514.16.1.el7.x86_64 | grep mpt > -rw-r--r-- 1 root root 0 May 16 04:37 etc/fstab.empty > -rw-r--r-- 1 root root 22 Nov 5 2016 > usr/lib/kbd/unimaps/empty.uni > drwxr-xr-x 2 root root 0 May 16 04:37 > usr/lib/modules/3.10.0-514.16.1.el7.x86_64/kernel/drivers/scsi/mpt3sas > -rw-r--r-- 1 root root 379021 Apr 12 08:51 > usr/lib/modules/3.10.0-514.16.1.el7.x86_64/kernel/drivers/ scsi/mpt3sas/mpt2sas.ko > > > So I added it: > > > $ sudo dracut --force --add-drivers mpt3sas --kver=4.9.25-27.el7.x86_64 > $ sudo lsinitrd -k 4.9.25-27.el7.x86_64 | grep mpt > Arguments: --force --add-drivers 'mpt3sas' --kver '4.9.25-27.el7.x86_64' > -rw-r--r-- 1 root root 0 May 16 12:57 etc/fstab.empty > -rw-r--r-- 1 root root 22 Nov 5 2016 > usr/lib/kbd/unimaps/empty.uni > drwxr-xr-x 2 root root 0 May 16 12:57 > usr/lib/modules/4.9.25-27.el7.x86_64/kernel/drivers/scsi/mpt3sas > -rwxr--r-- 1 root root 374152 May 16 12:57 > usr/lib/modules/4.9.25-27.el7.x86_64/kernel/drivers/scsi/ mpt3sas/mpt3sas.ko > > > After this I was able to get the 4.9 kernel to boot and Xen is now working.
You should be able to make this change permanently from /etc/dracut.conf with the add_drivers+= line, or build a generic initrd with hostonly="no" .
Thank you for this. Adding mpt2sas or mpt3sas to the drivers+ line worked. Running dracut generates a proper image now.
I don't know why it wasn't originally added.
--Sarah
For some reason each of the 5, 6 & 7 releases have issues with the mpt2sas driver (there isn't enough swiotlb allocated by default for the driver to work properly, and the units for that parameter changed between 5 and 6). It's like this driver has bad luck or something.
Jerry
On 05/17/2017 01:30 PM, Jerry wrote:
For some reason each of the 5, 6 & 7 releases have issues with the mpt2sas driver (there isn't enough swiotlb allocated by default for the driver to work properly, and the units for that parameter changed between 5 and 6). It's like this driver has bad luck or something.
To my best knowledge they shouldn't use swiotlb if you give the dom0 4GiB+ of RAM. I posted the exact same problem before and that was the recommendation I got. It works.
--Sarah
On Wed, May 17, 2017 at 1:45 PM, Sarah Newman srn@prgmr.com wrote:
On 05/17/2017 01:30 PM, Jerry wrote:
For some reason each of the 5, 6 & 7 releases have issues with the
mpt2sas
driver (there isn't enough swiotlb allocated by default for the driver to work properly, and the units for that parameter changed between 5 and 6). It's like this driver has bad luck or something.
To my best knowledge they shouldn't use swiotlb if you give the dom0 4GiB+ of RAM. I posted the exact same problem before and that was the recommendation I got. It works.
--Sarah
I'm not keen on letting dom0 take up so much memory :)
I'm giving swiotlb 256MB of memory (swiotlb=131072 on el6/7) and that resolved it for my configuration.
Jerry
-
Jerry -
Sarah Newman