[CentOS] High memory needs

Sat Sep 29 20:49:27 UTC 2012
Gordon Messmer <yinyang at eburg.com>

On 09/27/2012 09:34 AM, Les Mikesell wrote:
> That's an interesting difference on its own, since the underlying
> files are about 95M and 54M respectively.  Does the 32 bit kernel use
> some tricks to sparsely map files where the 64 bit one does it
> directly with page tables?

No, it's because glibc maps the whole file into memory on systems with 
greater than 32 bit memory address sizes.

http://illiterat.livejournal.com/4615.html?nojs=1
This blog discusses the topic briefly, but his description of 
M_MMAP_THRESHOLD is a little off.

Anyway, with a 64 bit address space, there's no reason not to mmap the 
entire file.  On a smaller space, mapping the entire file could consume 
some significant portion of the process' address space.  It depends on 
the size of that file, but let's say 5-10% on systems with a big 
locale-archive file.  That's a big cost for the feature.  On a 64 bit 
system, where address space is nearly unlimited, there's no reason to 
avoid mapping the whole file.  On BOTH systems, mapping the file alone 
doesn't actually consume physical memory.

Anyone using VIRT to make decisions about resource utilization is 
completely ignorant of its function.