Hi List,
I'm trying to work out the balloon driver on CentOS6 HVM but it seems to be not working with default 2.6.32 or newer 3.4 kernel.
Does balloon driver work on HVM linux guests with Linux Kernel 3.4 from centos repo ?
PV is fine and is able to balloon.
My host is Xen 4.1.2 with dom0 as 3.4.46-8.el6.centos.alt.x86_64 My guest is CentOS6.4 with kernel 3.4.46-8.el6.centos.alt.x86_64
Please have a look at below guest logs, [root@hvmc6 ~]# uname -a Linux hvmc6 3.4.50-8.el6.centos.alt.x86_64 #1 SMP Fri Jun 21 03:14:22 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux [root@hvmc6 ~]# xenstore-ls memory initial-reservation = "1193984" initial-target = "1048576" static-max = "16777216" target = "1048576" dynamic-min = "1048576" dynamic-max = "1048576" memory-offset = "-258100" [root@hvmc6 ~]# dmesg | grep -i balloon xen/balloon: Initialising balloon driver. xen-balloon: Initialising balloon driver. [root@hvmc6 ~]# free -m total used free shared buffers cached Mem: 463 221 241 0 24 13 -/+ buffers/cache: 184 278 Swap: 9987 7 9980
When I try to set it manually from host, I get # xl mem-set 229 2049 xc: error: do_memory_op: WARNING: op=16 not converted to safe ioctls: Internal error
Anyone has made CentOS6 PVHVM work with ballooning on Xen ?
Thanks. Rushikesh
Hi List,
It seems that the xen balloon functionality for guest is not enabled in CentOS 3.4.50-8.el6 xen kernel by default via centos-release-xen.x86_64 repo. I think this should have been present since this extra Xen repo needs to support as both Guest and Host (Dom0).
The kernel had to be recompiled with CONFIG_XEN_SELFBALLOONING. How can I request to make it default enable in centos repo ? or is there any reason to not have it default ?
Thanks.
On Fri, Jul 12, 2013 at 7:42 PM, Rushikesh Jadhav 2rushikeshj@gmail.comwrote:
Hi List,
I'm trying to work out the balloon driver on CentOS6 HVM but it seems to be not working with default 2.6.32 or newer 3.4 kernel.
Does balloon driver work on HVM linux guests with Linux Kernel 3.4 from centos repo ?
PV is fine and is able to balloon.
My host is Xen 4.1.2 with dom0 as 3.4.46-8.el6.centos.alt.x86_64 My guest is CentOS6.4 with kernel 3.4.46-8.el6.centos.alt.x86_64
Please have a look at below guest logs, [root@hvmc6 ~]# uname -a Linux hvmc6 3.4.50-8.el6.centos.alt.x86_64 #1 SMP Fri Jun 21 03:14:22 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux [root@hvmc6 ~]# xenstore-ls memory initial-reservation = "1193984" initial-target = "1048576" static-max = "16777216" target = "1048576" dynamic-min = "1048576" dynamic-max = "1048576" memory-offset = "-258100" [root@hvmc6 ~]# dmesg | grep -i balloon xen/balloon: Initialising balloon driver. xen-balloon: Initialising balloon driver. [root@hvmc6 ~]# free -m total used free shared buffers cached Mem: 463 221 241 0 24 13 -/+ buffers/cache: 184 278 Swap: 9987 7 9980
When I try to set it manually from host, I get # xl mem-set 229 2049 xc: error: do_memory_op: WARNING: op=16 not converted to safe ioctls: Internal error
Anyone has made CentOS6 PVHVM work with ballooning on Xen ?
Thanks. Rushikesh
On 07/15/2013 08:44 PM, Rushikesh Jadhav wrote:
Hi List,
It seems that the xen balloon functionality for guest is not enabled in CentOS 3.4.50-8.el6 xen kernel by default via centos-release-xen.x86_64 repo. I think this should have been present since this extra Xen repo needs to support as both Guest and Host (Dom0).
The kernel had to be recompiled with CONFIG_XEN_SELFBALLOONING. How can I request to make it default enable in centos repo ? or is there any reason to not have it default ?
can you file a request at bugs.centos.org for this please ?
Also, do one of the xen guys want to weigh in ?
On Tue, Jul 16, 2013 at 2:49 PM, Karanbir Singh mail-lists@karan.orgwrote:
On 07/15/2013 08:44 PM, Rushikesh Jadhav wrote:
Hi List,
It seems that the xen balloon functionality for guest is not enabled in CentOS 3.4.50-8.el6 xen kernel by default via centos-release-xen.x86_64 repo. I think this should have been present since this extra Xen repo needs to support as both Guest and Host (Dom0).
The kernel had to be recompiled with CONFIG_XEN_SELFBALLOONING. How can I request to make it default enable in centos repo ? or is there any reason to not have it default ?
can you file a request at bugs.centos.org for this please ?
I have filed two separate bugs. One which relates to the subject issue is in http://bugs.centos.org/view.php?id=6561 and one which relates to default kernel 2.6.32 (not related to subject) is http://bugs.centos.org/view.php?id=6484 & https://bugzilla.redhat.com/show_bug.cgi?id=969601
Also, do one of the xen guys want to weigh in ?
-- Karanbir Singh +44-207-0999389 | http://www.karan.org/ | twitter.com/kbsingh GnuPG Key : http://www.karan.org/publickey.asc _______________________________________________ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
On 15/07/13 20:44, Rushikesh Jadhav wrote:
Hi List,
It seems that the xen balloon functionality for guest is not enabled in CentOS 3.4.50-8.el6 xen kernel by default via centos-release-xen.x86_64 repo. I think this should have been present since this extra Xen repo needs to support as both Guest and Host (Dom0).
The kernel is only intended for use as a dom0 kernel. We recommend the use of the existing 2.6.32 kernel for guests.
The kernel had to be recompiled with CONFIG_XEN_SELFBALLOONING. How can I request to make it default enable in centos repo ? or is there any reason to not have it default ?
We do not want self-ballooning enabled in the dom0 kernel. This option is also not required for toolstack-directed ballooning.
On Fri, Jul 12, 2013 at 7:42 PM, Rushikesh Jadhav <2rushikeshj@gmail.com mailto:2rushikeshj@gmail.com> wrote:
Hi List, I'm trying to work out the balloon driver on CentOS6 HVM but it seems to be not working with default 2.6.32 or newer 3.4 kernel. Does balloon driver work on HVM linux guests with Linux Kernel 3.4 from centos repo ? PV is fine and is able to balloon. My host is Xen 4.1.2 with dom0 as 3.4.46-8.el6.centos.alt.x86_64 My guest is CentOS6.4 with kernel 3.4.46-8.el6.centos.alt.x86_64 Please have a look at below guest logs, [root@hvmc6 ~]# uname -a Linux hvmc6 3.4.50-8.el6.centos.alt.x86_64 #1 SMP Fri Jun 21 03:14:22 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux [root@hvmc6 ~]# xenstore-ls memory initial-reservation = "1193984" initial-target = "1048576" static-max = "16777216" target = "1048576" dynamic-min = "1048576" dynamic-max = "1048576" memory-offset = "-258100" [root@hvmc6 ~]# dmesg | grep -i balloon xen/balloon: Initialising balloon driver. xen-balloon: Initialising balloon driver. [root@hvmc6 ~]# free -m total used free shared buffers cached Mem: 463 221 241 0 24 13 -/+ buffers/cache: 184 278 Swap: 9987 7 9980
It's not clear what sequence of toolstack operations you've done prior to the above commands. Are you trying to increase or decrease the amount of memory in the guest, what were the toolstack commands you used and what is the initial guest configuration.
When I try to set it manually from host, I get # xl mem-set 229 2049 xc: error: do_memory_op: WARNING: op=16 not converted to safe ioctls: Internal error
This (harmless) warning only occurs on XenServer hosts. What host are you actually running?
David
On Tue, Jul 16, 2013 at 3:26 PM, David Vrabel david.vrabel@citrix.comwrote:
On 15/07/13 20:44, Rushikesh Jadhav wrote:
Hi List,
It seems that the xen balloon functionality for guest is not enabled in CentOS 3.4.50-8.el6 xen kernel by default via centos-release-xen.x86_64 repo. I think this should have been present since this extra Xen repo needs to support as both Guest and Host (Dom0).
The kernel is only intended for use as a dom0 kernel. We recommend the use of the existing 2.6.32 kernel for guests.
Any idea for not using 3.4.50 as guest kernel ? 2.6.32 is giving lot of trouble in ballooning, I have filed below bugs for it. https://bugzilla.redhat.com/show_bug.cgi?id=969601 http://bugs.centos.org/view.php?id=6484
Please have a look at comment https://bugzilla.redhat.com/show_bug.cgi?id=969601#c2
The kernel had to be recompiled with CONFIG_XEN_SELFBALLOONING. How can I request to make it default enable in centos repo ? or is there any reason to not have it default ?
We do not want self-ballooning enabled in the dom0 kernel. This option is also not required for toolstack-directed ballooning.
I'm currently using only xl toolstack and not using XAPI/Xenserver toolstack, I have both environments and intent to fix in both. Since 2.6.32 is not ballooning, I had to find another kernel which supports this. 3.4.50 seems to be working in Xenserver but does not work on native Xen,
I'm using centos-release-xen repo to setup host ( dom0 CentOS6 + 3.4.50 ) & guest ( domU CentOS6 + 3.4.50 ). In my testing it was observed that if CONFIG_XEN_SELFBALLOONING is not set then the guest does not balloon. ( Verified by xl mem-set )
Problem may be with CONFIG_XEN_BALLOON_MEMORY_HOTPLUG=y as well. What do you think ? I'll try to disable selfballooning and test with only CONFIG_XEN_BALLOON_MEMORY_HOTPLUG.
Thanks.
On Fri, Jul 12, 2013 at 7:42 PM, Rushikesh Jadhav <2rushikeshj@gmail.com mailto:2rushikeshj@gmail.com> wrote:
Hi List, I'm trying to work out the balloon driver on CentOS6 HVM but it seems to be not working with default 2.6.32 or newer 3.4 kernel. Does balloon driver work on HVM linux guests with Linux Kernel 3.4 from centos repo ? PV is fine and is able to balloon. My host is Xen 4.1.2 with dom0 as 3.4.46-8.el6.centos.alt.x86_64 My guest is CentOS6.4 with kernel 3.4.46-8.el6.centos.alt.x86_64 Please have a look at below guest logs, [root@hvmc6 ~]# uname -a Linux hvmc6 3.4.50-8.el6.centos.alt.x86_64 #1 SMP Fri Jun 21 03:14:22 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux [root@hvmc6 ~]# xenstore-ls memory initial-reservation = "1193984" initial-target = "1048576" static-max = "16777216" target = "1048576" dynamic-min = "1048576" dynamic-max = "1048576" memory-offset = "-258100" [root@hvmc6 ~]# dmesg | grep -i balloon xen/balloon: Initialising balloon driver. xen-balloon: Initialising balloon driver. [root@hvmc6 ~]# free -m total used free shared buffers cached Mem: 463 221 241 0 24 13 -/+ buffers/cache: 184 278 Swap: 9987 7 9980It's not clear what sequence of toolstack operations you've done prior to the above commands. Are you trying to increase or decrease the amount of memory in the guest, what were the toolstack commands you used and what is the initial guest configuration.
When I try to set it manually from host, I get # xl mem-set 229 2049 xc: error: do_memory_op: WARNING: op=16 not converted to safe ioctls: Internal errorThis (harmless) warning only occurs on XenServer hosts. What host are you actually running?
David
On 16/07/13 14:18, Rushikesh Jadhav wrote:
On Tue, Jul 16, 2013 at 3:26 PM, David Vrabel <david.vrabel@citrix.com mailto:david.vrabel@citrix.com> wrote:
On 15/07/13 20:44, Rushikesh Jadhav wrote: > Hi List, > > It seems that the xen balloon functionality for guest is not enabled in > CentOS 3.4.50-8.el6 xen kernel by default via centos-release-xen.x86_64 > repo. I think this should have been present since this extra Xen repo > needs to support as both Guest and Host (Dom0). The kernel is only intended for use as a dom0 kernel. We recommend the use of the existing 2.6.32 kernel for guests.Any idea for not using 3.4.50 as guest kernel ?
The specific 3.4 kernel provided as part of the Xen4CentOS project as a configuration selected for use as a dom0 kernel and thus it may not have guest-centric options (like ballooning) correctly configured.
Problem may be with CONFIG_XEN_BALLOON_MEMORY_HOTPLUG=y as well. What do you think ?
CONFIG_XEN_BALLOON_MEMORY_HOTPLUG=y is required to allow most Linux PV guests to balloon above their initial allocation. However, it is broken in some cases when enabled in a dom0 kernel and has been disabled.
As a workaround I would suggest starting all guests with an initial allocation that corresponds to the maximum they may ever use. You can then immediately balloon them down.
David
On Wed, Jul 17, 2013 at 5:42 PM, David Vrabel david.vrabel@citrix.comwrote:
On 16/07/13 14:18, Rushikesh Jadhav wrote:
On Tue, Jul 16, 2013 at 3:26 PM, David Vrabel <david.vrabel@citrix.com mailto:david.vrabel@citrix.com> wrote:
On 15/07/13 20:44, Rushikesh Jadhav wrote: > Hi List, > > It seems that the xen balloon functionality for guest is not enabled in > CentOS 3.4.50-8.el6 xen kernel by default via centos-release-xen.x86_64 > repo. I think this should have been present since this extra Xenrepo
> needs to support as both Guest and Host (Dom0). The kernel is only intended for use as a dom0 kernel. We recommendthe
use of the existing 2.6.32 kernel for guests.Any idea for not using 3.4.50 as guest kernel ?
The specific 3.4 kernel provided as part of the Xen4CentOS project as a configuration selected for use as a dom0 kernel and thus it may not have guest-centric options (like ballooning) correctly configured.
Problem may be with CONFIG_XEN_BALLOON_MEMORY_HOTPLUG=y as well. What do you think ?
CONFIG_XEN_BALLOON_MEMORY_HOTPLUG=y is required to allow most Linux PV guests to balloon above their initial allocation. However, it is broken in some cases when enabled in a dom0 kernel and has been disabled.
Can we enable it with a code check for xen_initial_domain() ? We really need an authenticated guest Centos kernel which has bug free ballooning as Xen guest.
KVM or HVM Xen is fine but PV will always suffer till CentOS/RHEL switches to 3.X I don't have any news on CentOS7 or RHEL7.
As a workaround I would suggest starting all guests with an initial allocation that corresponds to the maximum they may ever use. You can then immediately balloon them down.
This workaround works good only while first boot. If the guest gets rebooted after balloon down then the base memory is changed to last known allocated memory. After reboot the guest wont be able to balloon up.
David
-
David Vrabel -
Karanbir Singh -
Rushikesh Jadhav