On 6/15/20 2:46 PM, Stephen John Smoogen wrote: > > > On Sun, 14 Jun 2020 at 14:49, Manuel Wolfshant > <wolfy at nobugconsulting.ro <mailto:wolfy at nobugconsulting.ro>> wrote: > > Hello > > > For the past months I've been testing upgrading my Xen hosts > to CentOS 7 and I face an issue for which I need your help to solve. > > The testing machines are IBM blades, model H21 and H21XM. > Initial tests were performed on the H21 with 16 GB RAM; during the > last 6=7 weeks I've been using the H21XM with 64 GB. In all cases > the guests were fully updated CentOS 7 -- initially 7.6 ( most > recent at the time of the initial tests ), and respectively 7.8 > for the tests performed during the last 2 months. As host I used > initially CentOS 6 with latest kernel available in the centos virt > repo at the time of the tests and CentOS 7 with the latest kernel > as well. As xen versions I tested 4.8 and 4.12 ( xl info included > below ). The storage for the last tests is a Crucial MX500 but > results were similar when using traditional HDD. > > My problem, in short, is that the guests are extremely slow. > For instance , in the most recent tests, a yum install kernel > takes cca 1 min on the host and 12-15 (!!!) minutes in the guest, > all time being spent in dracut regenerating the initramfs images. > I've done rough tests with the storage ( via dd if=/dev/zero > of=a_test_file size bs=10M count=1000 ) and the speed was > comparable between the hosts and the guests. The version of the > kernel in use inside the guest also did not seem to make any > difference . OTOH, sysbench ( > https://github.com/akopytov/sysbench/ ) as well as p7zip benchmark > report for the guests a speed which is between 10% and 50% of the > host. Quite obviously, changing the elevator had no influence either. > > Here is the info which I think that should be relevant for the > software versions in use. Feel free to ask for any additional info. > > > Is there a way to boot up a PV guest versus an HVM? If I understood the docs correctly, newer xen does only PVHVM ( xen_platform_pci=1 activates that ) and HVM. But they say it's better than PV. And I did verify, PVHVM is indeed enabled and active > I could not find a H21XM but found an HS21XM on the iBM My bad. The blades are indeed HS21 (Type 8853) and HS21 XM (Type 7995). The XM blades have 2*Xeon E5450 at 3GHz / 12GB L1 cache processors. The options I can fiddle with are https://imgur.com/a/DonXe5P AFAICS the setttings are reasonable but please do let me know if there is anything there that should not be as it is > site and that seemed to be a 4 core 8 thread cpu which looks 'old' > enough that the Spectre/etc fixes to improve performance after the > initial hit were not done. (Basically I was told that if the CPU was > older than 2012, just turn off hyperthreading altogether to try and > get back some performance.. but don't expect much). I can live with that. My problem is that DomU are much much slower that Dom0 so it seems xen virtualization affects ( heavily ) the performance. > As such I would also try turning off HT on the CPU to see if that > improves anything. I got inspired by Adi's earlier suggestion and after reading https://access.redhat.com/articles/3311301 I've tried today all variants of disabling the spectre mitigations. Whatever I do, immediately after a reboot, yum reinstall kernel does not take less than 5 minutes :( It goes down to 2 min if I repeat the operation afterwards so I guess some caching kicks in. I will try later today the kernels from elrepo and maybe even xen.crc.id.au ( I kind of hate the "disable selinux" recommendation from the install page so I postponed it in the hope of other solution ). -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.centos.org/pipermail/centos-virt/attachments/20200615/37b3f4ca/attachment-0005.html>