<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<div class="moz-cite-prefix">On 6/15/20 2:46 PM, Stephen John
Smoogen wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CANnLRdgBYTdgurXivb=NfL=+rL3iy40KnbEt3po18SXuvZ9gaw@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">
<div dir="ltr"><br>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Sun, 14 Jun 2020 at
14:49, Manuel Wolfshant <<a
href="mailto:wolfy@nobugconsulting.ro"
moz-do-not-send="true">wolfy@nobugconsulting.ro</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div>
<p>Hello</p>
<p><br>
</p>
<p> For the past months I've been testing upgrading my
Xen hosts to CentOS 7 and I face an issue for which I
need your help to solve.</p>
<p> The testing machines are IBM blades, model H21 and
H21XM. Initial tests were performed on the H21 with 16
GB RAM; during the last 6=7 weeks I've been using the
H21XM with 64 GB. In all cases the guests were fully
updated CentOS 7 -- initially 7.6 ( most recent at the
time of the initial tests ), and respectively 7.8 for
the tests performed during the last 2 months. As host I
used initially CentOS 6 with latest kernel available in
the centos virt repo at the time of the tests and CentOS
7 with the latest kernel as well. As xen versions I
tested 4.8 and 4.12 ( xl info included below ). The
storage for the last tests is a Crucial MX500 but
results were similar when using traditional HDD.<br>
</p>
<p> My problem, in short, is that the guests are
extremely slow. For instance , in the most recent tests,
a yum install kernel takes cca 1 min on the host and
12-15 (!!!) minutes in the guest, all time being spent
in dracut regenerating the initramfs images. I've done
rough tests with the storage ( via dd if=/dev/zero
of=a_test_file size bs=10M count=1000 ) and the speed
was comparable between the hosts and the guests. The
version of the kernel in use inside the guest also did
not seem to make any difference . OTOH, sysbench ( <a
href="https://github.com/akopytov/sysbench/"
target="_blank" moz-do-not-send="true">https://github.com/akopytov/sysbench/</a>
) as well as p7zip benchmark report for the guests a
speed which is between 10% and 50% of the host. Quite
obviously, changing the elevator had no influence
either. <br>
</p>
<p> Here is the info which I think that should be
relevant for the software versions in use. Feel free to
ask for any additional info.</p>
</div>
</blockquote>
<div><br>
</div>
<div>Is there a way to boot up a PV guest versus an HVM?</div>
</div>
</div>
</blockquote>
<p>If I understood the docs correctly, newer xen does only PVHVM (
xen_platform_pci=1
activates that ) and HVM. But they say it's better than PV. And I
did verify, PVHVM is indeed enabled and active<br>
</p>
<p><br>
</p>
<p><br>
</p>
<blockquote type="cite"
cite="mid:CANnLRdgBYTdgurXivb=NfL=+rL3iy40KnbEt3po18SXuvZ9gaw@mail.gmail.com">
<div dir="ltr">
<div class="gmail_quote">
<div> I could not find a H21XM but found an HS21XM on the iBM
</div>
</div>
</div>
</blockquote>
<p>My bad. The blades are indeed HS21 (Type 8853) and HS21 XM (Type
7995). The XM blades have 2*Xeon E5450@3GHz / 12GB L1 cache
processors. The options I can fiddle with are
<a class="moz-txt-link-freetext" href="https://imgur.com/a/DonXe5P">https://imgur.com/a/DonXe5P</a> <br>
</p>
<p>AFAICS the setttings are reasonable but please do let me know if
there is anything there that should not be as it is<br>
</p>
<p><br>
</p>
<blockquote type="cite"
cite="mid:CANnLRdgBYTdgurXivb=NfL=+rL3iy40KnbEt3po18SXuvZ9gaw@mail.gmail.com">
<div dir="ltr">
<div class="gmail_quote">
<div>site and that seemed to be a 4 core 8 thread cpu which
looks 'old' enough that the Spectre/etc fixes to improve
performance after the initial hit were not done. (Basically
I was told that if the CPU was older than 2012, just turn
off hyperthreading altogether to try and get back some
performance.. but don't expect much).</div>
</div>
</div>
</blockquote>
<p>I can live with that. My problem is that DomU are much much
slower that Dom0 so it seems xen virtualization affects ( heavily
) the performance.<br>
</p>
<p><br>
</p>
<blockquote type="cite"
cite="mid:CANnLRdgBYTdgurXivb=NfL=+rL3iy40KnbEt3po18SXuvZ9gaw@mail.gmail.com">
<div dir="ltr">
<div class="gmail_quote">
<div> As such I would also try turning off HT on the CPU to
see if that improves anything.</div>
</div>
</div>
</blockquote>
<p>I got inspired by Adi's earlier suggestion and after reading
<a class="moz-txt-link-freetext" href="https://access.redhat.com/articles/3311301">https://access.redhat.com/articles/3311301</a> I've tried today all
variants of disabling the spectre mitigations. Whatever I do,
immediately after a reboot, yum reinstall kernel does not take
less than 5 minutes :( It goes down to 2 min if I repeat the
operation afterwards so I guess some caching kicks in. I will try
later today the kernels from elrepo and maybe even xen.crc.id.au (
I kind of hate the "disable selinux" recommendation from the
install page so I postponed it in the hope of other solution ).<br>
</p>
</body>
</html>