[CentOS-virt] Nested KVM issue

Mon Aug 22 23:22:39 UTC 2016
Adi Pircalabu <adi at ddns.com.au>

Laurentiu,

Just to chip in, without thoroughly checking the thread (which I'd like 
to apologize in advance for): have you tried to test other kernel 
instead of the stock one? You can have a look at 
http://elrepo.org/tiki/kernel-ml and http://elrepo.org/tiki/kernel-lt 
and use them for some comparison testing.
Cheers,

---
Adi Pircalabu

On 22-08-2016 18:31, Laurentiu Soica wrote:
> No luck with qemu-kvm-ev, the behavior is the same. Running perf
> record -a -g on the baremetal shows that most of the CPU time is in
> _raw_spin_lock
> 
>   Children      Self  Command          Shared Object
> Symbol
>                                                     -  93.62%
> 93.62%  qemu-kvm         [kernel.kallsyms]            [k]
> _raw_spin_lock
> 
> 
>    - _raw_spin_lock
> 
> 
>       + 45.30% kvm_mmu_sync_roots
> 
> 
>       + 28.49% kvm_mmu_load
> 
> 
>       + 25.00% mmu_free_roots
> 
> 
>       + 1.12% tdp_page_fault
> 
> În joi, 18 aug. 2016 la 11:59, Laurentiu Soica a
> scris:
> 
>> I've tried with KSM disabled and nothing changed.
>> 
>> I've upgraded KVM to qemu-kvm-ev. I'm waiting to see if there are
>> any improvements and report back.
>> 
>> În mie., 17 aug. 2016 la 15:10, Boris Derzhavets
>> <bderzhavets at hotmail.com> a scris:
>> 
>> For myself KSM is unpredictable feature. The problem is Compute,
>> just this node
>> 
>> does "copy on write" , so only Compute.
>> 
>> My concern exactly is where would it lead to worse or better Guest
>> behavior ?
>> 
>> I am not expecting complete fix.  I would track via top/htop  and
>> dmesg via Cron on 1-2 hr
>> 
>> period.
>> 
>> -------------------------
>> 
>> FROM: centos-virt-bounces at centos.org
>> <centos-virt-bounces at centos.org> on behalf of Laurentiu Soica
>> <laurentiu at soica.ro>
>> 
>> SENT: Wednesday, August 17, 2016 6:38 AM
>> 
>> TO: Discussion about the virtualization on CentOS
>> SUBJECT: Re: [CentOS-virt] Nested KVM issue
>> 
>> Both baremetal and compute ? Are there any other metrics do you
>> consider useful to collect for troubleshooting purposes ?
>> 
>> În mie., 17 aug. 2016 la 13:04, Boris Derzhavets
>> a scris:
>> 
>> It sounds weird, but attempt to disable KSM and see would it help or
>> no ?
>> 
>> -------------------------
>> 
>> FROM: centos-virt-bounces at centos.org
>> <centos-virt-bounces at centos.org> on behalf of Laurentiu Soica
>> 
>> SENT: Wednesday, August 17, 2016 4:56 AM
>> 
>> TO: Discussion about the virtualization on CentOS
>> SUBJECT: Re: [CentOS-virt] Nested KVM issue
>> 
>> Enabled the logging on both compute and baremetal. Nothing strange
>> in logs:
>> 
>> on baremetal :
>> 
>> Wed Aug 17 11:51:01 EEST 2016: committed 62310764 free 58501808
>> Wed Aug 17 11:51:01 EEST 2016: 87025667 < 123574516 and free >
>> 24714903, stop ksm
>> 
>> on compute:
>> 
>> Wed Aug 17 08:52:52 UTC 2016: committed 24547132 free 76730936
>> Wed Aug 17 08:52:52 UTC 2016: 45139624 < 102962460 and free >
>> 20592492, stop ksm
>> 
>> and the compute node is again at 100% CPU utilization.
>> 
>> În mar., 16 aug. 2016 la 15:26, Boris Derzhavets
>> a scris:
>> 
>> I would enable ksmtuned logging ,if it has been done verify logs
>> 
>> -------------------------
>> 
>> FROM: centos-virt-bounces at centos.org
>> <centos-virt-bounces at centos.org> on behalf of Laurentiu Soica
>> 
>> SENT: Tuesday, August 16, 2016 7:16 AM
>> 
>> TO: Discussion about the virtualization on CentOS
>> SUBJECT: Re: [CentOS-virt] Nested KVM issue
>> 
>> Yes. It is on both baremetal and compute node.
>> 
>> _______________________________________________
>> CentOS-virt mailing list
>> CentOS-virt at centos.org
>> https://lists.centos.org/mailman/listinfo/centos-virt
>  _______________________________________________
> CentOS-virt mailing list
> CentOS-virt at centos.org
> https://lists.centos.org/mailman/listinfo/centos-virt
> 
> --
> 
> Laurentiu Soica
> _______________________________________________
> CentOS-virt mailing list
> CentOS-virt at centos.org
> https://lists.centos.org/mailman/listinfo/centos-virt