[CentOS] memory problems

Fri Dec 9 15:51:53 UTC 2005
Tom De Blende <tom.deblende at dhl.com>

Hi,

Since we upgraded our server to 3.6, it runs out of memory. This is what 
I can find in syslog (multiple similar entries):

Dec  7 19:52:42 gcclo77 kernel: Out of Memory: Killed process 31375 
(nagios).
Dec  7 19:52:43 gcclo77 kernel: Mem-info:
Dec  7 19:52:43 gcclo77 kernel: Zone:DMA freepages:  2927 min:     0 
low:     0 high:     0
Dec  7 19:52:43 gcclo77 kernel: Zone:Normal freepages:   737 min:   766 
low:  4031 high:  5791
Dec  7 19:52:43 gcclo77 kernel: Zone:HighMem freepages:   126 min:   255 
low:   510 high:   765
Dec  7 19:52:43 gcclo77 kernel: Free pages:        3790 (   126 HighMem)
Dec  7 19:52:43 gcclo77 kernel: ( Active: 162420/241, inactive_laundry: 
0, inactive_clean: 0, free: 3790 )
Dec  7 19:52:43 gcclo77 kernel:   aa:0 ac:0 id:0 il:0 ic:0 fr:2927
Dec  7 19:52:43 gcclo77 kernel:   aa:157445 ac:1554 id:301 il:0 ic:0 fr:737
Dec  7 19:52:43 gcclo77 kernel:   aa:2791 ac:569 id:1 il:0 ic:0 fr:126
Dec  7 19:52:43 gcclo77 kernel: 3*4kB 2*8kB 4*16kB 3*32kB 4*64kB 2*128kB 
1*256kB 1*512kB 0*1024kB 1*2048kB 2*4096kB = 11708kB)
Dec  7 19:52:43 gcclo77 kernel: 49*4kB 0*8kB 0*16kB 0*32kB 1*64kB 
1*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 2948kB)
Dec  7 19:52:43 gcclo77 kernel: 0*4kB 1*8kB 1*16kB 1*32kB 1*64kB 1*128kB 
1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 504kB)
Dec  7 19:52:44 gcclo77 kernel: Swap cache: add 861742, delete 858950, 
find 7617227/7652707, race 0+36
Dec  7 19:52:44 gcclo77 kernel: 13082 pages of slabcache
Dec  7 19:52:44 gcclo77 kernel: 9852 pages of kernel stacks
Dec  7 19:52:44 gcclo77 kernel: 17191 lowmem pagetables, 28807 highmem 
pagetables
Dec  7 19:52:44 gcclo77 kernel: 32 bounce buffer pages, 32 are on the 
emergency list
Dec  7 19:52:44 gcclo77 kernel: Free swap:            0kB
Dec  7 19:52:44 gcclo77 kernel: 262138 pages of RAM
Dec  7 19:52:44 gcclo77 kernel: 32762 pages of HIGHMEM
Dec  7 19:52:44 gcclo77 kernel: 5784 reserved pages
Dec  7 19:52:44 gcclo77 kernel: 7450 pages shared
Dec  7 19:52:45 gcclo77 kernel: 2792 pages swap cached
Dec  7 19:52:45 gcclo77 kernel: Out of Memory: Killed process 31401 
(nagios).
Dec  7 19:52:45 gcclo77 kernel: Mem-info:
Dec  7 19:52:45 gcclo77 kernel: Zone:DMA freepages:  2927 min:     0 
low:     0 high:     0
Dec  7 19:52:45 gcclo77 kernel: Zone:Normal freepages:   753 min:   766 
low:  4031 high:  5791
Dec  7 19:52:45 gcclo77 kernel: Zone:HighMem freepages:   126 min:   255 
low:   510 high:   765
Dec  7 19:52:45 gcclo77 kernel: Free pages:        3806 (   126 HighMem)
Dec  7 19:52:45 gcclo77 kernel: ( Active: 162136/453, inactive_laundry: 
72, inactive_clean: 0, free: 3806 )
Dec  7 19:52:45 gcclo77 kernel:   aa:0 ac:0 id:0 il:0 ic:0 fr:2927
Dec  7 19:52:45 gcclo77 kernel:   aa:157502 ac:1554 id:172 il:72 ic:0 fr:753
Dec  7 19:52:46 gcclo77 kernel:   aa:2666 ac:439 id:256 il:0 ic:0 fr:126
Dec  7 19:52:46 gcclo77 kernel: 3*4kB 2*8kB 4*16kB 3*32kB 4*64kB 2*128kB 
1*256kB 1*512kB 0*1024kB 1*2048kB 2*4096kB = 11708kB)
Dec  7 19:52:46 gcclo77 kernel: 65*4kB 0*8kB 0*16kB 0*32kB 1*64kB 
1*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 3012kB)
Dec  7 19:52:46 gcclo77 kernel: 0*4kB 1*8kB 1*16kB 1*32kB 1*64kB 1*128kB 
1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 504kB)
Dec  7 19:52:46 gcclo77 kernel: Swap cache: add 861742, delete 858950, 
find 7617227/7652707, race 0+36
Dec  7 19:52:46 gcclo77 kernel: 13082 pages of slabcache
Dec  7 19:52:46 gcclo77 kernel: 9852 pages of kernel stacks
Dec  7 19:52:46 gcclo77 kernel: 17178 lowmem pagetables, 28807 highmem 
pagetables
Dec  7 19:52:46 gcclo77 kernel: 32 bounce buffer pages, 32 are on the 
emergency list
Dec  7 19:52:47 gcclo77 kernel: Free swap:            0kB
Dec  7 19:52:47 gcclo77 kernel: 262138 pages of RAM
Dec  7 19:52:47 gcclo77 kernel: 32762 pages of HIGHMEM
Dec  7 19:52:47 gcclo77 kernel: 5784 reserved pages
Dec  7 19:52:47 gcclo77 kernel: 7020 pages shared
Dec  7 19:52:47 gcclo77 kernel: 2792 pages swap cached

Anyone any ideas as to what causes this?

Thanks!
Tom