I finally found a solution to our problem. I think some people running like us a combination CentOS 64 bits\Sun Grid Engine, could encounter the same situation. Here is a detailed explanation, hope it can be useful to someone!
The file /usr/lib/locale/locale-archive is a memory-mapped file used by glibc (the gnu C library). This file contains the languages used over the system (for instance, man pages). This memory-mapping allows to read in the file as if it were in memory, avoiding system calls used to perform disk-read operations, therefore it allows much faster access. Memory-mapped files (as well as shared libraries) are counted as part of the virtual memory of processes (see "top" command, "VIRT" field). Therefore, the part of the locale-archive file mapped to memory adds up to the virtual memory of every processes that makes use of glibc (basically everything), while this is actually only once in memory. In other words, for every processes, the virtual memory overestimates the real memory of the process by, at least, the part of the locale-archive file which is memory-mapped.
Apparently, on 32bits distros, and on most 64bits distros, only a part of the locale-archive file is mmapped: for instance on CentOS 32 bits: $ pmap $$ b7689000 2048K r---- /usr/lib/locale/locale-archive Only 2MB of the file are mmapped, while the file is actually ~54MB. Some distros only install a small subsets of languages, for instance on my Ubuntu 12.04, this file is only ~3MB.
I still don't get the reason, but CentOS x86_64 6.2 (and 6.3) mmappes the *entire* file: $ pmap $$ 00007f217fae4000 96836K r---- /usr/lib/locale/locale-archive Consequence: every processes that use glibc is overestimated by ~100MB (according to the virtual memory).
That, alone, is not a problem at all. It is known that the virtual memory is a bad estimation of the actual memory used, the fact that it is largely overestimated does not matter much in most cases.
The problem, however, is that if you deal with a computing cluster that uses Sun Grid Engine (we have version 6.2u5) it uses this value to check whether jobs exceed their memory limit or not. The consequence is a false impression that jobs require a huge amount of memory, SGE being miss-leaded, and thus killing them.
A solution could be to restrict the number of languages when installing glibc-common (which provides the locale-archive file), to have a smaller locale-archive file. This should be possible with: $ echo "%_install_langs <lang1>:<lang2>:<lang3>" >> /etc/rpm/macros.lang $ rpm -e glibc-common --nodeps $ rm /usr/lib/locale/locale-archive $ rpm -i <glibc-common.rpm> I don't know why but we couldn't have glibc-common (2.12-1.80) taking it into account, though. All languages were always installed. I read somewhere that it is done on purpose, not sure...
The solution that worked for us is a post-install fix that consists of removing all languages that we don't use. We checked with "locale" which languages are used on the system. Then we remove all the others: $ localedef --list-archive | grep -v -e "en_US" -e "de_DE" -e "en_GB" | xargs localedef --delete-from-archive $ mv /usr/lib/locale/locale-archive /usr/lib/locale/locale-archive.tmpl $ build-locale-archive After this, the size of the locale-archive file is ~4MB, and running a single Bash instance does not show "107MB" for SGE anymore :-) Et voilà!
Jérémie
2012/9/27 Les Mikesell lesmikesell@gmail.com:
On Thu, Sep 27, 2012 at 10:46 AM, Gordon Messmer yinyang@eburg.com wrote:
I understand it may not be very precise, however I still don't understant the difference compared to other x64 ditributions, under CentOS the value is 7 times higher!
This might explain it: https://bugzilla.redhat.com/show_bug.cgi?id=156477 The mmapped local-archive contains all languages even though only the ones you use are accessed from it. Other distros split them and the installers only install what you want.
64 bit system: 00007f8ae84b7000 102580K r---- /usr/lib/locale/locale-archive
32 bit: b7689000 2048K r---- /usr/lib/locale/locale-archive
That's an interesting difference on its own, since the underlying files are about 95M and 54M respectively. Does the 32 bit kernel use some tricks to sparsely map files where the 64 bit one does it directly with page tables?
-- Les Mikesell lesmikesell@gmail.com _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos