Hi,
Running Centos 5 (32bit)... running a server with 4gb of RAM and it's shows up as:
[root@ubercore1 ~]# free total used free shared buffers cached Mem: 3369544 2549668 819876 0 29008 2382360 -/+ buffers/cache: 138300 3231244 Swap: 6291448 0 6291448
I know 4gb is the limit for a 32bit OS but i thought it would be able to use all of it? Is there another kernel i was supposed to install?
--N
Nick wrote:
The thing is, and I'm still trying to find a solution for one of our servers, the the memory addresses from 4Gb down are used for PCI, VGA, onboard stuff, and even though the memory isn't used, the addresses are, and so they are not passed to the OS.
Can you send some info on the motherboard you have?
Nick wrote:
What chipset does the motherboard have? Asus, Intel, MSI, etc?
Martin Marques wrote:
Asus and MSI don't make chipsets, nor do they make Dell server boards.
google leads me to dell.com where the 'tech specs' for this dual Xeon 5000/5100/5300 series 1U server... which says its a Intel 5000X chipset.
but thats neither here nor there, PAE is a universal issue for any x86 32bit system with 4GB+ ram, with PAE disabled, the BIOS, PCI, AGP or PCI-express, etc IO spaces consume anywheres from .5 to 1GB of the 32bit address space. PAE is a hardware workaround implemented in pretty much all Intel and AMD CPUs made in the last 5+ years, and allows the OS to access more than 4GB of physical address space. PAE introduces some hardware overhead because it involves larger page tables and another level of indirection in the TLB lookups.
CentOS 5 installs defaults to PAE off because there are some systems where PAE is crash-happy. Someone here has already explained how to enable PAE and in fact the original poster tried it and is happy with his full 4GB now.
John R Pierce wrote:
Well, looks like I'm having quite a bit of a problem then. :-(
I have PAE kernel running, and all I see is 3.2Gb of the 4Gb.
dmidecode gives me this (intel chip):
Handle 0x0000, DMI type 0, 20 bytes. BIOS Information Vendor: Intel Corp. Version: EV91510A.86A.0482.2006.0222.2350
And later this:
Handle 0x0041, DMI type 19, 15 bytes. Memory Array Mapped Address Starting Address: 0x00000000000 Ending Address: 0x000C77FFFFF Range Size: 3192 MB Physical Array Handle: 0x0040 Partition Width: 0
[snip]
Handle 0x0048, DMI type 17, 27 bytes. Memory Device Array Handle: 0x0040 Error Information Handle: 0x003F Total Width: 64 bits Data Width: 64 bits Size: 1024 MB Form Factor: DIMM Set: 2 Locator: J6H2 Bank Locator: CHANNEL B DIMM1 Type: DDR Type Detail: Synchronous Speed: 400 MHz (2.5 ns) Manufacturer: Manufacturer4 Serial Number: SerNum4 Asset Tag: AssetTagNum4 Part Number: PartNum4
Handle 0x0049, DMI type 20, 19 bytes. Memory Device Mapped Address Starting Address: 0x000C0000000 Ending Address: 0x000C6FFFFFF Range Size: 112 MB Physical Device Handle: 0x0048 Memory Array Mapped Address Handle: 0x0041 Partition Row Position: 2 Interleave Position: 2 Interleaved Data Depth: 2
As you can see, the last 1Gb bank is only used at 11%.
Any ideas on why I can't see all 4Gb?
Martin Marques wrote:
This thread may be a start: http://lkml.org/lkml/2006/7/26/204
It sounds like this is a problem with this particular chipset.
-- jeremy
Martin Marques wrote:
Could be an issue with the ASPI information being provided by the system BIOS. The missing 700MB or so of memory is at a high physical address somewhere, its up to the BIOS to tell the OS where this is.
according to that BIOS identifier code in your DMI data, your board is a Intel Desktop D9150GEV or its cousins, and you have BIOS version 0482, which is the last version (see http://support.intel.com/support/motherboards/desktop/sb/cs-010639.htm)
per the technical product specification for those boards, http://download.intel.com/support/motherboards/desktop/d915gev/sb/c6860201.p... PAE may not be supported. it doesn't say that explicitly, however, section 2.2.1 on page 51 says ...
The board utilizes 4 GB of addressable system memory. Typically the address space that is allocated for PCI Conventional bus add-in cards, PCI Express configuration space, BIOS (firmware hub), and chipset overhead resides above the top of DRAM (total system memory). On a system that has 4 GB of system memory installed, it is not possible to use all of the installed memory due to system address space being allocated for other system critical functions. These functions include the following:
• BIOS/firmware hub (2 MB) • Local APIC (19 MB) • Digital Media Interface (40 MB) • Front side bus interrupts (17 MB) • PCI Express configuration space (256 MB) • MCH base address registers, internal graphics ranges, PCI Express ports (up to 512 MB) • Memory-mapped I/O that is dynamically allocated for PCI Conventional and PCI Express add-in cards.
this implies that the 'overlapping' memory is NOT relocated to a higher address. This was a rather low end desktop board from circa 2004, intended strictly to support Win2000/XP in 32bit mode. with the P4 500 and 600 series CPUs and onboard Intel shared memory graphics.
On 7/26/07, Nick list@everywhereinternet.com wrote:
I've seen this before on laptops. This could be a limitation of the chipset itself. If that is the case, nothing can be done about it.
Another option is that you try to boot with the PAE kernel. That kernel is made to go over the 4GB boundary on 32bit.
Regards, Tim
Tim Verhoeven wrote:
yum install kernel-PAE
<reboot>
[root@ubercore1 ~]# uname -a Linux ubercore1 2.6.18-8.1.8.el5PAE #1 SMP Tue Jul 10 07:50:36 EDT 2007 i686 i686 i386 GNU/Linux [root@ubercore1 ~]# free total used free shared buffers cached Mem: 4149852 251852 3898000 0 12500 141792 -/+ buffers/cache: 97560 4052292 Swap: 6291448 0 6291448
Fixed!
Is there anything i should know about the PAE kernel... like it self destructs after migrating 2000 users over to it? ;)
On 7/26/07, Nick list@everywhereinternet.com wrote:
Nothing that I can think of. Only that using PAE mode incurs a small performance penalty because of extra address translations. But functionally it's identical to a normal kernel.
Regards, Tim
Tim Verhoeven wrote:
Another option is that you try to boot with the PAE kernel. That kernel is made to go over the 4GB boundary on 32bit.
Didn't work for me. :-(
[root@newweb ~]# free total used free shared buffers cached Mem: 3237236 1865956 1371280 0 263588 1279996 -/+ buffers/cache: 322372 2914864 Swap: 0 0 0 [root@newweb ~]# uname -a Linux newweb.matematica 2.6.18-8.1.8.el5PAE #1 SMP Tue Jul 10 07:50:36 EDT 2007 i686 i686 i386 GNU/Linux
Martin Marques spake the following on 7/26/2007 1:46 PM:
The board has to support it. It has to be able to map the underlying memory to a higher address space, and the bios has to notify the OS where this is. Are you using a desktop board as opposed to a server board?
Hi,
Anyone trying for 3>gb ram on an Asus p5b-vm should try their new bios 901 . 2007/07/18
Quote "P5B-VM BIOS 0901 Fix memory remapping function is not working properly issue."
Flashed mine and now 4 gb is mapped properly.
Shawn
Scott Silva wrote:
Yes:
Handle 0x0000, DMI type 0, 20 bytes. BIOS Information Vendor: Intel Corp. Version: EV91510A.86A.0482.2006.0222.2350
Martin Marques spake the following on 7/27/2007 6:43 AM:
Many desktop boards are designed for Windows, and linux isn't quite up to speed on them. You usually need to run Cutting edge distros like Fedora on desktop boards to get the most out of them.
Scott Silva escribió:
That board is 2 years old...
Hi Nick,
----- "Nick" list@everywhereinternet.com escreveu:
I have almost the same issue here :) what I can advice is to take a look on the reserved memory on boot (using dmesg), but I didn't know what can be done to change it. Take a look:
---> CentOS-5.0 (x86_64) on HP-DL380G4 (4GB)
[root@Suzuka ~]# dmesg |more Bootdata ok (command line is ro root=/dev/sys/root rhgb quiet) Linux version 2.6.18-8.1.8.el5xen (mockbuild@builder6.centos.org) (gcc version 4 .1.1 20070105 (Red Hat 4.1.1-52)) #1 SMP Tue Jul 10 07:06:45 EDT 2007 <... cut ...> Memory: 3772028k/3933628k available (2321k kernel code, 152780k reserved, 1312k data, 168k init)
The "free" command on this machine reports less memory because of Xen. [root@Suzuka ~]# free total used free shared buffers cached Mem: 3032064 3024120 7944 0 15472 313564 -/+ buffers/cache: 2695084 336980 Swap: 4194296 1280636 2913660
But "xentop" reports aprox. 4GB: xentop - 11:02:21 Xen 3.0.3-rc5-8.1.8.el 3 domains: 3 running, 0 blocked, 0 paused, 0 crashed, 0 dying, 0 shutdown Mem: 4128324k total, 4127496k used, 828k free CPUs: 4 @ 3400MHz
---> CentOS-4.5 (i386) on SuperMicro MoBo
[root@Sakhir ~]# dmesg |more Linux version 2.6.9-55.0.2.ELsmp (mockbuild@builder4.centos.org) (gcc version 3. 4.6 20060404 (Red Hat 3.4.6-8)) #1 SMP Tue Jun 26 14:30:58 EDT 2007 <... cut ...> Memory: 4149888k/4587520k available (1883k kernel code, 42852k reserved, 761k da ta, 188k init, 3276224k highmem)
[root@Sakhir ~]# free total used free shared buffers cached Mem: 4151980 4100544 51436 0 233316 1579404 -/+ buffers/cache: 2287824 1864156 Swap: 2104504 192 2104312
I don't know why, but on my HP machine CentOS always recognized less memory, with both CentOS4 and CentOS5 64bits, I had SLES9-x86_64 on that machine and it used all 4GB. And on the SuperMicro with 32bits OS I'm using all 4GB, but didn't try CentOS5 on it yet.
Hope this data helps someone on this issue :)
Antonio
Try running a pae enabled kernel - it might help (I'm not sure what the package name for centos is 'bigmem' or something like that maybe - as in kernel-bigmem-$VERSION). (or even better a 64-bit x86_64/amd64 kernel if the cpu supports it).
- Maciej
-
Antonio da Silva Martins Junior -
Clint Dilks -
Jeremy Rosengren -
John R Pierce -
Maciej Zenczykowski -
Martin Marques -
Nick -
Scott Silva -
Shawn -
Tim Verhoeven