[CentOS] High memory needs

Thu Sep 27 15:46:35 UTC 2012
Gordon Messmer <yinyang at eburg.com>

On 09/27/2012 01:57 AM, Jérémie Dubois-Lacoste wrote:
> We have a computing cluster running Sun Grid Engine, which
> considers this value to check if a process exceeds the memory
> limit or not. So somehow I'm bound to consider it.
>
> I installed a machine from scratch with CentOS 6.2 x64, nothing
> else, I open a terminal, I run this simple bash script and VIRT
> goes beyond 100MB for it.
> I understand it may not be very precise, however I still don't
> understant the difference compared to other x64 ditributions,
> under CentOS the value is 7 times higher!

OK, so compare the memory map of the process on the two systems.  For 
example, I've included the output of 'pmap $$' on two CentOS 6 systems. 
  One is 32bit, the other is 64.  VIRT on the 32 bit system is ~5MB.  On 
the 64 bit system, VIRT is ~116MB.

Why?  Well, most of the VIRT size is a mmapped file,
/usr/lib/locale/locale-archive.  That file isn't resident in memory for 
the most part, but because the process addresses the contents of that 
file as if they were memory, rather than as if the content were a file, 
it's accounted as part of the VIRT size.

My login shell on the 64 bit host is only using 3.5MB of RAM total 
(including libraries shared with other processes), but its VIRT size is 
~116MB.

If your vendor is using VIRT to make decisions about available 
resources, they're doing their job profoundly wrong.  RES would be 
better, but still would represent far more memory use than is accurate, 
especially on systems with many active processes.


64 bit system:
[gordon at vagabond:~]$ pmap $$
23966:   bash
0000000000400000    872K r-x--  /usr/bin/bash
00000000006d9000      4K r----  /usr/bin/bash
00000000006da000     36K rw---  /usr/bin/bash
00000000006e3000     24K rw---    [ anon ]
00000000008e2000     36K rw---  /usr/bin/bash
000000000122d000   1836K rw---    [ anon ]
000000383c000000    128K r-x--  /usr/lib64/ld-2.15.so
000000383c21f000      4K r----  /usr/lib64/ld-2.15.so
000000383c220000      4K rw---  /usr/lib64/ld-2.15.so
000000383c221000      4K rw---    [ anon ]
000000383c400000   1712K r-x--  /usr/lib64/libc-2.15.so
000000383c5ac000   2048K -----  /usr/lib64/libc-2.15.so
000000383c7ac000     16K r----  /usr/lib64/libc-2.15.so
000000383c7b0000      8K rw---  /usr/lib64/libc-2.15.so
000000383c7b2000     20K rw---    [ anon ]
000000383cc00000     12K r-x--  /usr/lib64/libdl-2.15.so
000000383cc03000   2044K -----  /usr/lib64/libdl-2.15.so
000000383ce02000      4K r----  /usr/lib64/libdl-2.15.so
000000383ce03000      4K rw---  /usr/lib64/libdl-2.15.so
000000384d000000    148K r-x--  /usr/lib64/libtinfo.so.5.9
000000384d025000   2044K -----  /usr/lib64/libtinfo.so.5.9
000000384d224000     16K r----  /usr/lib64/libtinfo.so.5.9
000000384d228000      4K rw---  /usr/lib64/libtinfo.so.5.9
00007f8ae82aa000     48K r-x--  /usr/lib64/libnss_files-2.15.so
00007f8ae82b6000   2044K -----  /usr/lib64/libnss_files-2.15.so
00007f8ae84b5000      4K r----  /usr/lib64/libnss_files-2.15.so
00007f8ae84b6000      4K rw---  /usr/lib64/libnss_files-2.15.so
00007f8ae84b7000 102580K r----  /usr/lib/locale/locale-archive
00007f8aee8e4000     16K rw---    [ anon ]
00007f8aee8f5000      8K rw---    [ anon ]
00007f8aee8f7000     28K r--s-  /usr/lib64/gconv/gconv-modules.cache
00007f8aee8fe000      4K rw---    [ anon ]
00007fff5010a000    132K rw---    [ stack ]
00007fff501ff000      4K r-x--    [ anon ]
ffffffffff600000      4K r-x--    [ anon ]
  total           115904K


32 bit:
enetics at firewall:~$ pmap $$
32629:   -bash
00291000      4K r-x--    [ anon ]
00952000    120K r-x--  /lib/ld-2.12.so
00970000      4K r----  /lib/ld-2.12.so
00971000      4K rw---  /lib/ld-2.12.so
00978000   1584K r-x--  /lib/libc-2.12.so
00b04000      8K r----  /lib/libc-2.12.so
00b06000      4K rw---  /lib/libc-2.12.so
00b07000     12K rw---    [ anon ]
00b0c000     12K r-x--  /lib/libdl-2.12.so
00b0f000      4K r----  /lib/libdl-2.12.so
00b10000      4K rw---  /lib/libdl-2.12.so
00b39000     48K r-x--  /lib/libnss_files-2.12.so
00b45000      4K r----  /lib/libnss_files-2.12.so
00b46000      4K rw---  /lib/libnss_files-2.12.so
00b7d000     88K r-x--  /lib/libtinfo.so.5.7
00b93000     12K rw---  /lib/libtinfo.so.5.7
08047000    836K r-x--  /bin/bash
08118000     20K rw---  /bin/bash
0811d000     20K rw---    [ anon ]
08b43000    232K rw---    [ anon ]
b7682000     28K r--s-  /usr/lib/gconv/gconv-modules.cache
b7689000   2048K r----  /usr/lib/locale/locale-archive
b7889000      8K rw---    [ anon ]
b788f000     12K rw---    [ anon ]
bff3f000     84K rw---    [ stack ]
  total     5204K