dear All,
I had a Centos 5.5 OS running for about 6 months used as a Xen VM server prfectly running 3 Virtual machines
Its a Sun Blade server with 8 core Xeon Proceesor with 32 GB Ram
couple of days back I added another 32 gb ram .
The bios shows the added ram that is now it shows me 64 GB but the Centos OS just recognizes 32gb only
the OS details are:
Centos release 5.5 final Kernel 2.6.18-194.32.1.el5xen on an x86_64
now also running uname -a shows me
Linux hypervisor2 2.6.18-194.32.1.el5xen #1 SMP Wed Jan 5 18:44:24 EST 2011 x86_64 x86_64 x86_64 GNU/Linux
shows that PAE support is not there
I do believe that installing kernel PAE with yum should solve the problem but since the server is a online production server just wanted to verify if I would run into some problems
apprecite your kinf help and suggestions
regards
sylvan
Hello Sylvan,
On Fri, 2011-08-26 at 10:10 +0000, sylvan.dcunha@gmail.com wrote:
Kernel 2.6.18-194.32.1.el5xen on an x86_64
I do believe that installing kernel PAE with yum should solve the problem but since the server is a online production server just wanted to verify if I would run into some problems
I have no idea why your server doesn't see the extra 32GB, but you do *not* need a PAE kernel. PAE is a hack for 32bit system to use more than 4GB of memory. Your machine is a 64bit system, so you don't need a PAE kernel on it.
Regards, Leonard.
Thanks Leonard,
Thanks for the immedite reply . apprecite. actually many post s say the PAE kernel required for addressing more than 4 gb ram . but since my server already detects 32 gb ram , detecting 64 also should not be an issue..
but just wondering why??.
regards
simon
On Fri, Aug 26, 2011 at 1:23 PM, Leonard den Ottolander < leonard@den.ottolander.nl> wrote:
Hello Sylvan,
On Fri, 2011-08-26 at 10:10 +0000, sylvan.dcunha@gmail.com wrote:
Kernel 2.6.18-194.32.1.el5xen on an x86_64
I do believe that installing kernel PAE with yum should solve the problem but since the server is a online production server just wanted to verify if I would run into some problems
I have no idea why your server doesn't see the extra 32GB, but you do *not* need a PAE kernel. PAE is a hack for 32bit system to use more than 4GB of memory. Your machine is a 64bit system, so you don't need a PAE kernel on it.
Regards, Leonard.
-- mount -t life -o ro /dev/dna /genetic/research
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On 08/26/2011 12:51 PM, benedict dcunha wrote:
Thanks Leonard, Thanks for the immedite reply . apprecite. actually many post s say the PAE kernel required for addressing more than 4 gb ram . but since my server already detects 32 gb ram , detecting 64 also should not be an issue..
"many posts say" a lot of things. Either these posts are specifically talking about 32bit systems or they are wrong. With a 32bit integer you can only address 4gb of ram so a hack was devised to make it possible go beyond that limit called PAE. Since with 64bit you no longer have that problem PAE doesn't exist on a 64bit system.
http://en.wikipedia.org/wiki/Physical_Address_Extension
but just wondering why??.
No idea but it has nothing to do with PAE. Can you post the output of "xm info"?
Regards, Dennis
Thanks Guys
really apprecite your quick responses. ( Dennis was right in tellin me about PAE since my system is 64 bit and if I do run yum install kernel-PAE there is nothing found.)
actually i found something more as i was figuring my issue out.
when I do a top i see the following ---------------
Tasks: 285 total, 1 running, 284 sleeping, 0 stopped, 0 zombie Cpu(s): 0.0%us, 0.0%sy, 0.0%ni, 99.8%id, 0.2%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 33554432k total, 15430836k used, 18123596k free, 323176k buffers Swap: 2819396k total, 0k used, 2819396k free, 13860960k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND ---------------
when I run cat /proc/meminfo i see
----MemTotal: 33554432 kB MemFree: 18123588 kB Buffers: 323192 kB Cached: 13860992 kB SwapCached: 0 kB Active: 9601264 kB Inactive: 4643904 kB HighTotal: 0 kB HighFree: 0 kB LowTotal: 33554432 kB LowFree: 18123588 kB SwapTotal: 2819396 kB SwapFree: 2819396 kB Dirty: 8 kB Writeback: 0 kB AnonPages: 60972 kB Mapped: 12528 kB Slab: 360860 kB PageTables: 18444 kB NFS_Unstable: 0 kB Bounce: 0 kB CommitLimit: 19596612 kB Committed_AS: 394740 kB VmallocTotal: 34359738367 kB VmallocUsed: 3304 kB VmallocChunk: 34359733919 kB ------------
actually I had run the above 2 command and found the memory was 32gb but as dennis said when I run the command xm info ---------------------- [root@hypervisor2 ~]# xm info host : hypervisor2 release : 2.6.18-194.32.1.el5xen version : #1 SMP Wed Jan 5 18:44:24 EST 2011 machine : x86_64 nr_cpus : 16 nr_nodes : 1 sockets_per_node : 2 cores_per_socket : 4 threads_per_core : 2 cpu_mhz : 2527 hw_caps : bfebfbff:28100800:00000000:00000140:009ce3bd:00000000:00000001 total_memory : 65527 free_memory : 22989 node_to_cpu : node0:0-15 xen_major : 3 xen_minor : 1 xen_extra : .2-194.32.1.el5 xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 xen_pagesize : 4096 platform_params : virt_start=0xffff800000000000 xen_changeset : unavailable cc_compiler : gcc version 4.1.2 20080704 (Red Hat 4.1.2-48) cc_compile_by : mockbuild cc_compile_domain : centos.org cc_compile_date : Wed Jan 5 17:43:03 EST 2011 xend_config_format : 2 -------------------- and then I ran xm top i see
-------------
5 domains: 1 running, 4 blocked, 0 paused, 0 crashed, 0 dying, 0 shutdown Mem: 67099744k total, 43558844k used, 23540900k free CPUs: 16 @ 2527MHz NAME STATE CPU(sec) CPU(%) MEM(k) MEM(%) MAXMEM(k) MAXMEM(%) VCPUS NETS NETTX(k) NETRX(k) VBDS VBD_OO VBD_RD VBD _WR SSID Domain-0 -----r 8401 2.4 33554688 50.0 no limit n/a 16 4 1892 16848120 0 0 0 0 0 sepmback --b--- 9758 2.6 2105220 3.1 4210688 6.3 4 1 0 0 1 0 0 0 0 winserver2 --b--- 5758 0.9 1056644 1.6 4210688 6.3 1 1 0 0 1 0 0 0 0 wsusserver --b--- 25812 11.4 3256196 4.9 8404992 12.5 4 1 0 0 1 0 0 0 0 zimbra --b--- 26183 7.8 2105220 3.1 4210688 6.3 4 1 0 16 1 0 0 0 0 ----------------------
so the above 2 command show me 64gb
Now I m confused ..
Is my Centos XEN server actually using the 64 bit ... and which command actually show me the right memory status
apprecite once again and thnaks
regards
sylvan
On Fri, Aug 26, 2011 at 3:29 PM, Dennis Jacobfeuerborn < dennisml@conversis.de> wrote:
On 08/26/2011 12:51 PM, benedict dcunha wrote:
Thanks Leonard, Thanks for the immedite reply . apprecite. actually many post s say the PAE kernel required for addressing more than
4
gb ram . but since my server already detects 32 gb ram , detecting 64
also
should not be an issue..
"many posts say" a lot of things. Either these posts are specifically talking about 32bit systems or they are wrong. With a 32bit integer you can only address 4gb of ram so a hack was devised to make it possible go beyond that limit called PAE. Since with 64bit you no longer have that problem PAE doesn't exist on a 64bit system.
http://en.wikipedia.org/wiki/Physical_Address_Extension
but just wondering why??.
No idea but it has nothing to do with PAE. Can you post the output of "xm info"?
Regards, Dennis _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Thanks Guys
really apprecite your quick responses. ( Dennis was right in tellin me about PAE since my system is 64 bit and if I do run yum install kernel-PAE there is nothing found.)
actually i found something more as i was figuring my issue out.
when I do a top i see the following
Tasks: 285 total, 1 running, 284 sleeping, 0 stopped, 0 zombie Cpu(s): 0.0%us, 0.0%sy, 0.0%ni, 99.8%id, 0.2%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 33554432k total, 15430836k used, 18123596k free, 323176k buffers Swap: 2819396k total, 0k used, 2819396k free, 13860960k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
when I run cat /proc/meminfo i see
----MemTotal: 33554432 kB MemFree: 18123588 kB Buffers: 323192 kB Cached: 13860992 kB SwapCached: 0 kB Active: 9601264 kB Inactive: 4643904 kB HighTotal: 0 kB HighFree: 0 kB LowTotal: 33554432 kB LowFree: 18123588 kB SwapTotal: 2819396 kB SwapFree: 2819396 kB Dirty: 8 kB Writeback: 0 kB AnonPages: 60972 kB Mapped: 12528 kB Slab: 360860 kB PageTables: 18444 kB NFS_Unstable: 0 kB Bounce: 0 kB CommitLimit: 19596612 kB Committed_AS: 394740 kB VmallocTotal: 34359738367 kB VmallocUsed: 3304 kB VmallocChunk: 34359733919 kB
actually I had run the above 2 command and found the memory was 32gb but as dennis said when I run the command xm info
[root@hypervisor2 ~]# xm info host : hypervisor2 release : 2.6.18-194.32.1.el5xen version : #1 SMP Wed Jan 5 18:44:24 EST 2011 machine : x86_64 nr_cpus : 16 nr_nodes : 1 sockets_per_node : 2 cores_per_socket : 4 threads_per_core : 2 cpu_mhz : 2527 hw_caps : bfebfbff:28100800:00000000:00000140:009ce3bd:00000000:00000001 total_memory : 65527 free_memory : 22989 node_to_cpu : node0:0-15 xen_major : 3 xen_minor : 1 xen_extra : .2-194.32.1.el5 xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 xen_pagesize : 4096 platform_params : virt_start=0xffff800000000000 xen_changeset : unavailable cc_compiler : gcc version 4.1.2 20080704 (Red Hat 4.1.2-48) cc_compile_by : mockbuild cc_compile_domain : centos.org cc_compile_date : Wed Jan 5 17:43:03 EST 2011 xend_config_format : 2
and then I ran xm top i see
5 domains: 1 running, 4 blocked, 0 paused, 0 crashed, 0 dying, 0 shutdown Mem: 67099744k total, 43558844k used, 23540900k free CPUs: 16 @ 2527MHz NAME STATE CPU(sec) CPU(%) MEM(k) MEM(%) MAXMEM(k) MAXMEM(%) VCPUS NETS NETTX(k) NETRX(k) VBDS VBD_OO VBD_RD VBD _WR SSID Domain-0 -----r 8401 2.4 33554688 50.0 no limit n/a 16 4 1892 16848120 0 0 0 0 0 sepmback --b--- 9758 2.6 2105220 3.1 4210688 6.3 4 1 0 0 1 0 0 0 0 winserver2 --b--- 5758 0.9 1056644 1.6 4210688 6.3 1 1 0 0 1 0 0 0 0 wsusserver --b--- 25812 11.4 3256196 4.9 8404992 12.5 4 1 0 0 1 0 0 0 0 zimbra --b--- 26183 7.8 2105220 3.1 4210688 6.3 4 1 0 16 1 0 0 0 0
so the above 2 command show me 64gb
Now I m confused ..
Is my Centos XEN server actually using the 64 bit ... and which command actually show me the right memory status
I'm not sure but I think everything is okay with your box. "xm top" shows that your box run with 64Gb memory, and what "top" shows is that your Dom0 OS runs with 32Gb memory, which means your DomU's take the other 32Gb. Did you up the memory settings in your virtual hosts configuration? After doing so you should see the Dom0 memory to go down when looking at it with "top".
Regards, Simon
On 08/27/2011 02:35 PM, Simon Matter wrote:
Thanks Guys
really apprecite your quick responses. ( Dennis was right in tellin me about PAE since my system is 64 bit and if I do run yum install kernel-PAE there is nothing found.)
actually i found something more as i was figuring my issue out.
when I do a top i see the following
Tasks: 285 total, 1 running, 284 sleeping, 0 stopped, 0 zombie Cpu(s): 0.0%us, 0.0%sy, 0.0%ni, 99.8%id, 0.2%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 33554432k total, 15430836k used, 18123596k free, 323176k buffers Swap: 2819396k total, 0k used, 2819396k free, 13860960k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
when I run cat /proc/meminfo i see
----MemTotal: 33554432 kB MemFree: 18123588 kB Buffers: 323192 kB Cached: 13860992 kB SwapCached: 0 kB Active: 9601264 kB Inactive: 4643904 kB HighTotal: 0 kB HighFree: 0 kB LowTotal: 33554432 kB LowFree: 18123588 kB SwapTotal: 2819396 kB SwapFree: 2819396 kB Dirty: 8 kB Writeback: 0 kB AnonPages: 60972 kB Mapped: 12528 kB Slab: 360860 kB PageTables: 18444 kB NFS_Unstable: 0 kB Bounce: 0 kB CommitLimit: 19596612 kB Committed_AS: 394740 kB VmallocTotal: 34359738367 kB VmallocUsed: 3304 kB VmallocChunk: 34359733919 kB
actually I had run the above 2 command and found the memory was 32gb but as dennis said when I run the command xm info
[root@hypervisor2 ~]# xm info host : hypervisor2 release : 2.6.18-194.32.1.el5xen version : #1 SMP Wed Jan 5 18:44:24 EST 2011 machine : x86_64 nr_cpus : 16 nr_nodes : 1 sockets_per_node : 2 cores_per_socket : 4 threads_per_core : 2 cpu_mhz : 2527 hw_caps : bfebfbff:28100800:00000000:00000140:009ce3bd:00000000:00000001 total_memory : 65527 free_memory : 22989 node_to_cpu : node0:0-15 xen_major : 3 xen_minor : 1 xen_extra : .2-194.32.1.el5 xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 xen_pagesize : 4096 platform_params : virt_start=0xffff800000000000 xen_changeset : unavailable cc_compiler : gcc version 4.1.2 20080704 (Red Hat 4.1.2-48) cc_compile_by : mockbuild cc_compile_domain : centos.org cc_compile_date : Wed Jan 5 17:43:03 EST 2011 xend_config_format : 2
and then I ran xm top i see
5 domains: 1 running, 4 blocked, 0 paused, 0 crashed, 0 dying, 0 shutdown Mem: 67099744k total, 43558844k used, 23540900k free CPUs: 16 @ 2527MHz NAME STATE CPU(sec) CPU(%) MEM(k) MEM(%) MAXMEM(k) MAXMEM(%) VCPUS NETS NETTX(k) NETRX(k) VBDS VBD_OO VBD_RD VBD _WR SSID Domain-0 -----r 8401 2.4 33554688 50.0 no limit n/a 16 4 1892 16848120 0 0 0 0 0 sepmback --b--- 9758 2.6 2105220 3.1 4210688 6.3 4 1 0 0 1 0 0 0 0 winserver2 --b--- 5758 0.9 1056644 1.6 4210688 6.3 1 1 0 0 1 0 0 0 0 wsusserver --b--- 25812 11.4 3256196 4.9 8404992 12.5 4 1 0 0 1 0 0 0 0 zimbra --b--- 26183 7.8 2105220 3.1 4210688 6.3 4 1 0 16 1 0 0 0 0
so the above 2 command show me 64gb
Now I m confused ..
Is my Centos XEN server actually using the 64 bit ... and which command actually show me the right memory status
I'm not sure but I think everything is okay with your box. "xm top" shows that your box run with 64Gb memory, and what "top" shows is that your Dom0 OS runs with 32Gb memory, which means your DomU's take the other 32Gb. Did you up the memory settings in your virtual hosts configuration? After doing so you should see the Dom0 memory to go down when looking at it with "top".
Benedict is right. Remember that under Xen Dom0 is really just another VM with special privileges. In fact on all my host systems I restrict Dom0 to 1G of ram by adding a "dom0_mem=1024M" argument to the grub configuration. In practice Xen should reduce the amount of ram used by Dom0 dynamically as needed for VMs i.e. if your virtual machines use 48G of ram then dom0 should only show 16G left. A while ago this mechanism lead to instability though so a general recommendation was to set dom0 memory to a fixed 1G or 512M to not rely on this dynamic memory handling.
Check this out for example: http://lists.xensource.com/archives/html/xen-users/2008-06/msg00729.html
Regards, Dennis
Dear Dennis,
Thanks a lot for the wise reply.. really did boost my knowledge.. honestly was unware of the fact that dom0 is just like another VM ... Anyway I had never restricted dom0 mem and since my 4 vms were working fine with no issues i never bothered much.
It was only after I added more 32 gb to existing 32 gb i did realise the above issue..
anyway I will try to restrict my dom0 to 1 GB ... and check it out.
but just still a litle confused why xm top & xm info shows 65gb and top , free and cat /proc/meminfo shows 32 gb
apprecite your kind advice n help
Regards
simon
On Aug 27, 2011 3:44pm, Dennis Jacobfeuerborn dennisml@conversis.de wrote:
On 08/27/2011 02:35 PM, Simon Matter wrote:
Thanks Guys
really apprecite your quick responses.
( Dennis was right in tellin me about PAE since my system is 64 bit
and if
I
do run yum install kernel-PAE there is nothing found.)
actually i found something more as i was figuring my issue out.
when I do a top i see the following
Tasks: 285 total, 1 running, 284 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.0%us, 0.0%sy, 0.0%ni, 99.8%id, 0.2%wa, 0.0%hi, 0.0%si,
0.0%st
Mem: 33554432k total, 15430836k used, 18123596k free, 323176k buffers
Swap: 2819396k total, 0k used, 2819396k free, 13860960k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
when I run cat /proc/meminfo i see
----MemTotal: 33554432 kB
MemFree: 18123588 kB
Buffers: 323192 kB
Cached: 13860992 kB
SwapCached: 0 kB
Active: 9601264 kB
Inactive: 4643904 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 33554432 kB
LowFree: 18123588 kB
SwapTotal: 2819396 kB
SwapFree: 2819396 kB
Dirty: 8 kB
Writeback: 0 kB
AnonPages: 60972 kB
Mapped: 12528 kB
Slab: 360860 kB
PageTables: 18444 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
CommitLimit: 19596612 kB
Committed_AS: 394740 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 3304 kB
VmallocChunk: 34359733919 kB
actually I had run the above 2 command and found the memory was 32gb
but as dennis said when I run the command
xm info
[root@hypervisor2 ~]# xm info
host : hypervisor2
release : 2.6.18-194.32.1.el5xen
version : #1 SMP Wed Jan 5 18:44:24 EST 2011
machine : x86_64
nr_cpus : 16
nr_nodes : 1
sockets_per_node : 2
cores_per_socket : 4
threads_per_core : 2
cpu_mhz : 2527
hw_caps :
bfebfbff:28100800:00000000:00000140:009ce3bd:00000000:00000001
total_memory : 65527
free_memory : 22989
node_to_cpu : node0:0-15
xen_major : 3
xen_minor : 1
xen_extra : .2-194.32.1.el5
xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32
hvm-3.0-x86_32p hvm-3.0-x86_64
xen_pagesize : 4096
platform_params : virt_start=0xffff800000000000
xen_changeset : unavailable
cc_compiler : gcc version 4.1.2 20080704 (Red Hat 4.1.2-48)
cc_compile_by : mockbuild
cc_compile_domain : centos.org
cc_compile_date : Wed Jan 5 17:43:03 EST 2011
xend_config_format : 2
and then I ran xm top i see
5 domains: 1 running, 4 blocked, 0 paused, 0 crashed, 0 dying, 0
shutdown
Mem: 67099744k total, 43558844k used, 23540900k free CPUs: 16 @ 2527MHz
NAME STATE CPU(sec) CPU(%) MEM(k) MEM(%) MAXMEM(k) MAXMEM(%)
VCPUS NETS NETTX(k) NETRX(k) VBDS VBD_OO VBD_RD VBD
_WR SSID
Domain-0 -----r 8401 2.4 33554688 50.0 no limit
n/a 16 4 1892 16848120 0 0 0
0 0
sepmback --b--- 9758 2.6 2105220 3.1 4210688
6.3 4 1 0 0 1 0 0
0 0
winserver2 --b--- 5758 0.9 1056644 1.6 4210688
6.3 1 1 0 0 1 0 0
0 0
wsusserver --b--- 25812 11.4 3256196 4.9 8404992
12.5 4 1 0 0 1 0 0
0 0
zimbra --b--- 26183 7.8 2105220 3.1 4210688
6.3 4 1 0 16 1 0 0
0 0
so the above 2 command show me 64gb
Now I m confused ..
Is my Centos XEN server actually using the 64 bit ...
and which command actually show me the right memory status
I'm not sure but I think everything is okay with your box. "xm top"
shows
that your box run with 64Gb memory, and what "top" shows is that your
Dom0
OS runs with 32Gb memory, which means your DomU's take the other 32Gb.
Did
you up the memory settings in your virtual hosts configuration? After
doing so you should see the Dom0 memory to go down when looking at it
with
"top".
Benedict is right. Remember that under Xen Dom0 is really just another VM
with special privileges. In fact on all my host systems I restrict Dom0 to
1G of ram by adding a "dom0_mem=1024M" argument to the grub configuration.
In practice Xen should reduce the amount of ram used by Dom0 dynamically as
needed for VMs ie if your virtual machines use 48G of ram then dom0
should only show 16G left. A while ago this mechanism lead to instability
though so a general recommendation was to set dom0 memory to a fixed 1G or
512M to not rely on this dynamic memory handling.
Check this out for example:
http://lists.xensource.com/archives/html/xen-users/2008-06/msg00729.html
Regards,
Dennis
CentOS mailing list
CentOS@centos.org
On 08/27/2011 09:12 PM, sylvan.dcunha@gmail.com wrote:
Dear Dennis,
Thanks a lot for the wise reply.. really did boost my knowledge.. honestly was unware of the fact that dom0 is just like another VM ... Anyway I had never restricted dom0 mem and since my 4 vms were working fine with no issues i never bothered much.
Yes, this is different from KVM where the VMs really are just normal processes on the host system and the host system itself isn't a VM.
On a Xen system if you look at /etc/grub.conf you'll notice that it looks slightly different than on a non-virtualized system. Specifically you'll find the following line: kernel /xen.gz-2.6.18-164.el5
That's the actual hypervisor and really the host system and once started it will basically start dom0 and give it special privileges. So Dom0 and the DomU's all run on top of the actual hypervisor.
It was only after I added more 32 gb to existing 32 gb i did realise the above issue..
Apparently dom0 has a 32G limit but that shouldn't be an issue unless you actually really require more than 32G specifically for dom0 and not the VMs.
anyway I will try to restrict my dom0 to 1 GB ... and check it out.
Remember that the problems with the dynamic memory management are most likely fixed nowadays so the limitation is not strictly necessary. But then 1G will probably be more than enough for dom0 so it doesn't really hurt either.
but just still a litle confused why xm top & xm info shows 65gb and top , free and cat /proc/meminfo shows 32 gb
The xm tools show you the actual physical memory in the system while the /proc/meminfo shows you only the memory visible to dom0 which as I mentioned above is apparently limited to 32G.
Regards, Dennis
On Tue, Aug 30, 2011 at 03:30:46PM +0200, Dennis Jacobfeuerborn wrote:
On 08/27/2011 09:12 PM, sylvan.dcunha@gmail.com wrote:
Dear Dennis,
Thanks a lot for the wise reply.. really did boost my knowledge.. honestly was unware of the fact that dom0 is just like another VM ... Anyway I had never restricted dom0 mem and since my 4 vms were working fine with no issues i never bothered much.
Yes, this is different from KVM where the VMs really are just normal processes on the host system and the host system itself isn't a VM.
On a Xen system if you look at /etc/grub.conf you'll notice that it looks slightly different than on a non-virtualized system. Specifically you'll find the following line: kernel /xen.gz-2.6.18-164.el5
That's the actual hypervisor and really the host system and once started it will basically start dom0 and give it special privileges. So Dom0 and the DomU's all run on top of the actual hypervisor.
It was only after I added more 32 gb to existing 32 gb i did realise the above issue..
Apparently dom0 has a 32G limit but that shouldn't be an issue unless you actually really require more than 32G specifically for dom0 and not the VMs.
anyway I will try to restrict my dom0 to 1 GB ... and check it out.
Remember that the problems with the dynamic memory management are most likely fixed nowadays so the limitation is not strictly necessary. But then 1G will probably be more than enough for dom0 so it doesn't really hurt either.
Still today you should dedicate a fixed amount of memory for dom0! say, 1GB, or so.
It's because of how Linux kernel allocates (and wastes) page struct memory: http://wiki.xen.org/xenwiki/XenBestPractices
-- Pasi
On 08/31/2011 05:38 PM, Pasi Kärkkäinen wrote:
On Tue, Aug 30, 2011 at 03:30:46PM +0200, Dennis Jacobfeuerborn wrote:
On 08/27/2011 09:12 PM, sylvan.dcunha@gmail.com wrote:
Dear Dennis,
Thanks a lot for the wise reply.. really did boost my knowledge.. honestly was unware of the fact that dom0 is just like another VM ... Anyway I had never restricted dom0 mem and since my 4 vms were working fine with no issues i never bothered much.
Yes, this is different from KVM where the VMs really are just normal processes on the host system and the host system itself isn't a VM.
On a Xen system if you look at /etc/grub.conf you'll notice that it looks slightly different than on a non-virtualized system. Specifically you'll find the following line: kernel /xen.gz-2.6.18-164.el5
That's the actual hypervisor and really the host system and once started it will basically start dom0 and give it special privileges. So Dom0 and the DomU's all run on top of the actual hypervisor.
It was only after I added more 32 gb to existing 32 gb i did realise the above issue..
Apparently dom0 has a 32G limit but that shouldn't be an issue unless you actually really require more than 32G specifically for dom0 and not the VMs.
anyway I will try to restrict my dom0 to 1 GB ... and check it out.
Remember that the problems with the dynamic memory management are most likely fixed nowadays so the limitation is not strictly necessary. But then 1G will probably be more than enough for dom0 so it doesn't really hurt either.
Still today you should dedicate a fixed amount of memory for dom0! say, 1GB, or so.
It's because of how Linux kernel allocates (and wastes) page struct memory: http://wiki.xen.org/xenwiki/XenBestPractices
Very good to know. Thanks for the information!
Regards, Dennis
Dear Guys,
First of all thanks for all your wise replies.. a little late since was tied with some other stuff and wanted to test the change I made on my xen server actually as dennis says ... and hes perfectly right
actually what I did was restrict my DOM0 mem size to 1 gb with dom0_mem=1024M in grub.conf and now top and free was showing 1 gb as my memory. also i have disabled balloning in the xend-config file as recommended and xm top and xm info were showing me 64 gb as total memory and the free memory and the memory used by my V.Ms was equall to 64 gb which is perfect
So I am now clear that free and top actually shows the memory assigned to DOM 0 and not the totalsystem memory under xen
Once again thaks and apprecite your responses guys..
regards
sylvan
On Sat, Aug 27, 2011 at 3:44 PM, Dennis Jacobfeuerborn < dennisml@conversis.de> wrote:
On 08/27/2011 02:35 PM, Simon Matter wrote:
Thanks Guys
really apprecite your quick responses. ( Dennis was right in tellin me about PAE since my system is 64 bit and
if
I do run yum install kernel-PAE there is nothing found.)
actually i found something more as i was figuring my issue out.
when I do a top i see the following
Tasks: 285 total, 1 running, 284 sleeping, 0 stopped, 0 zombie Cpu(s): 0.0%us, 0.0%sy, 0.0%ni, 99.8%id, 0.2%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 33554432k total, 15430836k used, 18123596k free, 323176k buffers Swap: 2819396k total, 0k used, 2819396k free, 13860960k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
when I run cat /proc/meminfo i see
----MemTotal: 33554432 kB MemFree: 18123588 kB Buffers: 323192 kB Cached: 13860992 kB SwapCached: 0 kB Active: 9601264 kB Inactive: 4643904 kB HighTotal: 0 kB HighFree: 0 kB LowTotal: 33554432 kB LowFree: 18123588 kB SwapTotal: 2819396 kB SwapFree: 2819396 kB Dirty: 8 kB Writeback: 0 kB AnonPages: 60972 kB Mapped: 12528 kB Slab: 360860 kB PageTables: 18444 kB NFS_Unstable: 0 kB Bounce: 0 kB CommitLimit: 19596612 kB Committed_AS: 394740 kB VmallocTotal: 34359738367 kB VmallocUsed: 3304 kB VmallocChunk: 34359733919 kB
actually I had run the above 2 command and found the memory was 32gb but as dennis said when I run the command xm info
[root@hypervisor2 ~]# xm info host : hypervisor2 release : 2.6.18-194.32.1.el5xen version : #1 SMP Wed Jan 5 18:44:24 EST 2011 machine : x86_64 nr_cpus : 16 nr_nodes : 1 sockets_per_node : 2 cores_per_socket : 4 threads_per_core : 2 cpu_mhz : 2527 hw_caps : bfebfbff:28100800:00000000:00000140:009ce3bd:00000000:00000001 total_memory : 65527 free_memory : 22989 node_to_cpu : node0:0-15 xen_major : 3 xen_minor : 1 xen_extra : .2-194.32.1.el5 xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 xen_pagesize : 4096 platform_params : virt_start=0xffff800000000000 xen_changeset : unavailable cc_compiler : gcc version 4.1.2 20080704 (Red Hat 4.1.2-48) cc_compile_by : mockbuild cc_compile_domain : centos.org cc_compile_date : Wed Jan 5 17:43:03 EST 2011 xend_config_format : 2
and then I ran xm top i see
5 domains: 1 running, 4 blocked, 0 paused, 0 crashed, 0 dying, 0
shutdown
Mem: 67099744k total, 43558844k used, 23540900k free CPUs: 16 @
2527MHz
NAME STATE CPU(sec) CPU(%) MEM(k) MEM(%) MAXMEM(k)MAXMEM(%)
VCPUS NETS NETTX(k) NETRX(k) VBDS VBD_OO VBD_RD VBD _WR SSID Domain-0 -----r 8401 2.4 33554688 50.0 no limit n/a 16 4 1892 16848120 0 0 0 0 0 sepmback --b--- 9758 2.6 2105220 3.1 4210688 6.3 4 1 0 0 1 0 0 0 0 winserver2 --b--- 5758 0.9 1056644 1.6 4210688 6.3 1 1 0 0 1 0 0 0 0 wsusserver --b--- 25812 11.4 3256196 4.9 8404992 12.5 4 1 0 0 1 0 0 0 0 zimbra --b--- 26183 7.8 2105220 3.1 4210688 6.3 4 1 0 16 1 0 0 0 0
so the above 2 command show me 64gb
Now I m confused ..
Is my Centos XEN server actually using the 64 bit ... and which command actually show me the right memory status
I'm not sure but I think everything is okay with your box. "xm top" shows that your box run with 64Gb memory, and what "top" shows is that your
Dom0
OS runs with 32Gb memory, which means your DomU's take the other 32Gb.
Did
you up the memory settings in your virtual hosts configuration? After doing so you should see the Dom0 memory to go down when looking at it
with
"top".
Benedict is right. Remember that under Xen Dom0 is really just another VM with special privileges. In fact on all my host systems I restrict Dom0 to 1G of ram by adding a "dom0_mem=1024M" argument to the grub configuration. In practice Xen should reduce the amount of ram used by Dom0 dynamically as needed for VMs i.e. if your virtual machines use 48G of ram then dom0 should only show 16G left. A while ago this mechanism lead to instability though so a general recommendation was to set dom0 memory to a fixed 1G or 512M to not rely on this dynamic memory handling.
Check this out for example: http://lists.xensource.com/archives/html/xen-users/2008-06/msg00729.html
Regards, Dennis _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
From: "sylvan.dcunha@gmail.com" sylvan.dcunha@gmail.com
I had a Centos 5.5 OS running for about 6 months used as a Xen VM server prfectly running 3 Virtual machines Its a Sun Blade server with 8 core Xeon Proceesor with 32 GB Ram couple of days back I added another 32 gb ram . The bios shows the added ram that is now it shows me 64 GB but the Centos OS just recognizes 32gb only
No Xen experience but did you do this?
http://wiki.xensource.com/xenwiki/XenFaq#head-f13e54576168210e3e59c16a1f2096...
JD
-
benedict dcunha -
Dennis Jacobfeuerborn -
John Doe -
Leonard den Ottolander -
Pasi Kärkkäinen -
Simon Matter -
sylvan.dcunha＠gmail.com