Hi list, We are experiencing a memory leak on our SquirrelMail server since the 5.3 update. The server is fully updated, only stock rpms. The httpd processes are eating all the memory and after swapping like hell, the server became unresponsive and we must hard-reboot it. The server is not that much loaded (max 10-15 concurrent users but with tons of mail in their inbox). Configuration's speaking, nothing was changed before and after the update. Is there someone else experiencing the same thing ? How can I search deeper the origin of the leak ? Thx, kfx Details follows: Screenshot from munin showing the memory usage's graph: http://uppix.net/e/a/7/cd2bdea933d57b0e28bde1f96b854.png [root at webmail ~]# uname -a Linux webmail 2.6.18-128.1.6.el5xen #1 SMP Wed Apr 1 09:53:14 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux [root at webmail ~]# yum list | grep -e php -e squirrel -e mysql -e httpd | grep installed httpd.x86_64 2.2.3-22.el5.centos installed mysql.x86_64 5.0.45-7.el5 installed mysql-server.x86_64 5.0.45-7.el5 installed php.x86_64 5.1.6-23.el5 installed php-cli.x86_64 5.1.6-23.el5 installed php-common.x86_64 5.1.6-23.el5 installed php-ldap.x86_64 5.1.6-23.el5 installed php-mbstring.x86_64 5.1.6-23.el5 installed php-mysql.x86_64 5.1.6-23.el5 installed php-pdo.x86_64 5.1.6-23.el5 installed php-pear.noarch 1:1.4.9-4.el5.1 installed php-pear-DB.noarch 1.7.13-1.el5.centos installed php-pear-MDB2.noarch 2.4.1-2.el5.centos installed php-pear-MDB2-Driver-mysql.noarch 1.4.1-3.el5.centos installed squirrelmail.noarch 1.4.8-5.el5.centos.3 installed Dmesg: Apr 2 17:18:28 s_local at webmail kernel:init invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0 Apr 2 17:18:28 s_local at webmail kernel: Apr 2 17:18:28 s_local at webmail kernel:Call Trace: Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff802bc998>] out_of_memory+0x8b/0x203 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff8020f657>] __alloc_pages+0x245/0x2ce Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff8021336e>] __do_page_cache_readahead+0xd0/0x21c Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff80262824>] __wait_on_bit_lock+0x5b/0x66 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff88081d4d>] :dm_mod:dm_any_congested+0x38/0x3f Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff80213c47>] filemap_nopage+0x148/0x322 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff80208db9>] __handle_mm_fault+0x440/0x11f6 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff802639f9>] _spin_lock_irqsave+0x9/0x14 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff802666ef>] do_page_fault+0xf7b/0x12e0 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff8025f82b>] error_exit+0x0/0x6e Apr 2 17:18:28 s_local at webmail kernel: Apr 2 17:18:28 s_local at webmail kernel:Mem-info: Apr 2 17:18:28 s_local at webmail kernel:DMA per-cpu: Apr 2 17:18:28 s_local at webmail kernel:cpu 0 hot: high 186, batch 31 used:84 Apr 2 17:18:28 s_local at webmail kernel:cpu 0 cold: high 62, batch 15 used:57 Apr 2 17:18:28 s_local at webmail kernel:DMA32 per-cpu: empty Apr 2 17:18:28 s_local at webmail kernel:Normal per-cpu: empty Apr 2 17:18:28 s_local at webmail kernel:HighMem per-cpu: empty Apr 2 17:18:28 s_local at webmail kernel:Free pages: 2912kB (0kB HighMem) Apr 2 17:18:28 s_local at webmail kernel:Active:73144 inactive:40971 dirty:0 writeback:0 unstable:0 free:728 slab:4076 mapped-file:2 mapped-anon:114411 pagetables:2857 Apr 2 17:18:28 s_local at webmail kernel:DMA free:2912kB min:2916kB low:3644kB high:4372kB active:292576kB inactive:163884kB present:532480kB pages_scanned:1521701 all_unreclaimable? yes Apr 2 17:18:28 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 17:18:28 s_local at webmail kernel:DMA32 free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 17:18:28 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 17:18:28 s_local at webmail kernel:Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 17:18:28 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 17:18:28 s_local at webmail kernel:HighMem free:0kB min:128kB low:128kB high:128kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 17:18:28 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 17:18:28 s_local at webmail kernel:DMA: 12*4kB 8*8kB 3*16kB 0*32kB 1*64kB 1*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 2912kB Apr 2 17:18:28 s_local at webmail kernel:DMA32: empty Apr 2 17:18:28 s_local at webmail kernel:Normal: empty Apr 2 17:18:28 s_local at webmail kernel:HighMem: empty Apr 2 17:18:28 s_local at webmail kernel:89 pagecache pages Apr 2 17:18:28 s_local at webmail kernel:Swap cache: add 176874, delete 176874, find 114990/119550, race 0+0 Apr 2 17:18:28 s_local at webmail kernel:Free swap = 0kB Apr 2 17:18:28 s_local at webmail kernel:Total swap = 557048kB Apr 2 17:18:28 s_local at webmail kernel:Free swap: 0kB Apr 2 17:18:28 s_local at webmail kernel:133120 pages of RAM Apr 2 17:18:28 s_local at webmail kernel:5993 reserved pages Apr 2 17:18:28 s_local at webmail kernel:4267 pages shared Apr 2 17:18:28 s_local at webmail kernel:0 pages swap cached Apr 2 17:18:28 s_local at webmail kernel:Out of memory: Killed process 1592 (httpd). Apr 2 17:18:28 s_local at webmail kernel:automount invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0 Apr 2 17:18:28 s_local at webmail kernel: Apr 2 17:18:28 s_local at webmail kernel:Call Trace: Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff802bc998>] out_of_memory+0x8b/0x203 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff8020f657>] __alloc_pages+0x245/0x2ce Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff8021336e>] __do_page_cache_readahead+0xd0/0x21c Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff80262824>] __wait_on_bit_lock+0x5b/0x66 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff88081d4d>] :dm_mod:dm_any_congested+0x38/0x3f Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff80213c47>] filemap_nopage+0x148/0x322 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff80208db9>] __handle_mm_fault+0x440/0x11f6 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff802639f9>] _spin_lock_irqsave+0x9/0x14 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff802666ef>] do_page_fault+0xf7b/0x12e0 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff8020622a>] hypercall_page+0x22a/0x1000 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff8020622a>] hypercall_page+0x22a/0x1000 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff8025f82b>] error_exit+0x0/0x6e Apr 2 17:18:28 s_local at webmail kernel: Apr 2 17:18:28 s_local at webmail kernel:Mem-info: Apr 2 17:18:28 s_local at webmail kernel:DMA per-cpu: Apr 2 17:18:28 s_local at webmail kernel:cpu 0 hot: high 186, batch 31 used:88 Apr 2 17:18:28 s_local at webmail kernel:cpu 0 cold: high 62, batch 15 used:57 Apr 2 17:18:28 s_local at webmail kernel:DMA32 per-cpu: empty Apr 2 17:18:28 s_local at webmail kernel:Normal per-cpu: empty Apr 2 17:18:28 s_local at webmail kernel:HighMem per-cpu: empty Apr 2 17:18:28 s_local at webmail kernel:Free pages: 2912kB (0kB HighMem) Apr 2 17:18:28 s_local at webmail kernel:Active:73160 inactive:40987 dirty:0 writeback:0 unstable:0 free:728 slab:4073 mapped-file:2 mapped-anon:114411 pagetables:2857 Apr 2 17:18:28 s_local at webmail kernel:DMA free:2912kB min:2916kB low:3644kB high:4372kB active:292640kB inactive:163948kB present:532480kB pages_scanned:5185565 all_unreclaimable? yes Apr 2 17:18:28 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 17:18:28 s_local at webmail kernel:DMA32 free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 17:18:28 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 17:18:28 s_local at webmail kernel:Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 17:18:28 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 17:18:28 s_local at webmail kernel:HighMem free:0kB min:128kB low:128kB high:128kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 17:18:28 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 17:18:28 s_local at webmail kernel:DMA: 12*4kB 8*8kB 3*16kB 0*32kB 1*64kB 1*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 2912kB Apr 2 17:18:28 s_local at webmail kernel:DMA32: empty Apr 2 17:18:28 s_local at webmail kernel:Normal: empty Apr 2 17:18:28 s_local at webmail kernel:HighMem: empty Apr 2 17:18:28 s_local at webmail kernel:89 pagecache pages Apr 2 17:18:28 s_local at webmail kernel:Swap cache: add 176874, delete 176874, find 114990/119550, race 0+0 Apr 2 17:18:28 s_local at webmail kernel:Free swap = 0kB Apr 2 17:18:28 s_local at webmail kernel:Total swap = 557048kB Apr 2 17:18:28 s_local at webmail kernel:Free swap: 0kB Apr 2 17:18:28 s_local at webmail kernel:133120 pages of RAM Apr 2 17:18:28 s_local at webmail kernel:5993 reserved pages Apr 2 17:18:28 s_local at webmail kernel:4235 pages shared Apr 2 17:18:28 s_local at webmail kernel:0 pages swap cached Apr 2 17:18:28 s_local at webmail kernel:audispd invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0 Apr 2 17:18:28 s_local at webmail kernel: Apr 2 17:18:28 s_local at webmail kernel:Call Trace: Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff802bc998>] out_of_memory+0x8b/0x203 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff8020f657>] __alloc_pages+0x245/0x2ce Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff8021336e>] __do_page_cache_readahead+0xd0/0x21c Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff80262824>] __wait_on_bit_lock+0x5b/0x66 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff88081d4d>] :dm_mod:dm_any_congested+0x38/0x3f Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff80213c47>] filemap_nopage+0x148/0x322 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff80208db9>] __handle_mm_fault+0x440/0x11f6 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff802666ef>] do_page_fault+0xf7b/0x12e0 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff8025f82b>] error_exit+0x0/0x6e Apr 2 17:18:28 s_local at webmail kernel: Apr 2 17:18:28 s_local at webmail kernel:Mem-info: Apr 2 17:18:28 s_local at webmail kernel:DMA per-cpu: Apr 2 17:18:28 s_local at webmail kernel:cpu 0 hot: high 186, batch 31 used:98 Apr 2 17:18:28 s_local at webmail kernel:cpu 0 cold: high 62, batch 15 used:57 Apr 2 17:18:28 s_local at webmail kernel:DMA32 per-cpu: empty Apr 2 17:18:28 s_local at webmail kernel:Normal per-cpu: empty Apr 2 17:18:28 s_local at webmail kernel:HighMem per-cpu: empty Apr 2 17:18:28 s_local at webmail kernel:Free pages: 2912kB (0kB HighMem) Apr 2 17:18:28 s_local at webmail kernel:Active:73115 inactive:40968 dirty:0 writeback:0 unstable:0 free:728 slab:4062 mapped-file:2 mapped-anon:114411 pagetables:2857 Apr 2 17:18:28 s_local at webmail kernel:DMA free:2912kB min:2916kB low:3644kB high:4372kB active:292460kB inactive:163872kB present:532480kB pages_scanned:8730229 all_unreclaimable? yes Apr 2 17:18:28 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 17:18:28 s_local at webmail kernel:DMA32 free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 17:18:28 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 17:18:28 s_local at webmail kernel:Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 17:18:28 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 17:18:28 s_local at webmail kernel:HighMem free:0kB min:128kB low:128kB high:128kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 17:18:28 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 17:18:28 s_local at webmail kernel:DMA: 12*4kB 8*8kB 3*16kB 0*32kB 1*64kB 1*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 2912kB Apr 2 17:18:28 s_local at webmail kernel:DMA32: empty Apr 2 17:18:28 s_local at webmail kernel:Normal: empty Apr 2 17:18:28 s_local at webmail kernel:HighMem: empty Apr 2 17:18:28 s_local at webmail kernel:89 pagecache pages Apr 2 17:18:28 s_local at webmail kernel:Swap cache: add 176874, delete 176874, find 114990/119550, race 0+0 Apr 2 17:18:28 s_local at webmail kernel:Free swap = 0kB Apr 2 17:18:28 s_local at webmail kernel:Total swap = 557048kB Apr 2 17:18:28 s_local at webmail kernel:Free swap: 0kB Apr 2 17:18:28 s_local at webmail kernel:133120 pages of RAM Apr 2 17:18:28 s_local at webmail kernel:5993 reserved pages Apr 2 17:18:28 s_local at webmail kernel:4299 pages shared Apr 2 17:18:28 s_local at webmail kernel:0 pages swap cached Apr 2 17:18:28 s_local at webmail kernel:kthread invoked oom-killer: gfp_mask=0xd0, order=0, oomkilladj=0 Apr 2 17:18:28 s_local at webmail kernel: Apr 2 17:18:28 s_local at webmail kernel:Call Trace: Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff802bc998>] out_of_memory+0x8b/0x203 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff8020f657>] __alloc_pages+0x245/0x2ce Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff8025e012>] cache_alloc_refill+0x269/0x4ba Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff8020afbc>] kmem_cache_alloc+0x50/0x6d Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff8021ef3f>] copy_process+0xa6/0x1a48 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff8020afce>] kmem_cache_alloc+0x62/0x6d Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff80263a0d>] _spin_lock_irq+0x9/0x14 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff80297d10>] alloc_pid+0x26c/0x292 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff80231fcf>] do_fork+0x69/0x1c1 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff80299dc0>] keventd_create_kthread+0x0/0xc4 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff8025fac5>] kernel_thread+0x81/0xde Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff80299dc0>] keventd_create_kthread+0x0/0xc4 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff8026df02>] monotonic_clock+0x35/0x7b Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff80233378>] kthread+0x0/0x132 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff8025fb22>] child_rip+0x0/0x12 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff80299ddd>] keventd_create_kthread+0x1d/0xc4 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff8022ec93>] __wake_up+0x38/0x4f Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff8024ee11>] run_workqueue+0x94/0xe4 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff8024b71a>] worker_thread+0x0/0x122 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff8024b80a>] worker_thread+0xf0/0x122 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff80286daf>] default_wake_function+0x0/0xe Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff80233476>] kthread+0xfe/0x132 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff8025fb2c>] child_rip+0xa/0x12 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff80233378>] kthread+0x0/0x132 Apr 2 17:18:28 s_local at webmail kernel:[<ffffffff8025fb22>] child_rip+0x0/0x12 Apr 2 17:18:28 s_local at webmail kernel: Apr 2 17:18:28 s_local at webmail kernel:Mem-info: Apr 2 17:18:28 s_local at webmail kernel:DMA per-cpu: Apr 2 17:18:28 s_local at webmail kernel:cpu 0 hot: high 186, batch 31 used:102 Apr 2 17:18:28 s_local at webmail kernel:cpu 0 cold: high 62, batch 15 used:57 Apr 2 17:18:28 s_local at webmail kernel:DMA32 per-cpu: empty Apr 2 17:18:28 s_local at webmail kernel:Normal per-cpu: empty Apr 2 17:18:28 s_local at webmail kernel:HighMem per-cpu: empty Apr 2 17:18:28 s_local at webmail kernel:Free pages: 2912kB (0kB HighMem) Apr 2 17:18:28 s_local at webmail kernel:Active:73053 inactive:40998 dirty:0 writeback:0 unstable:0 free:728 slab:4059 mapped-file:2 mapped-anon:114411 pagetables:2857 Apr 2 17:18:28 s_local at webmail kernel:DMA free:2912kB min:2916kB low:3644kB high:4372kB active:292212kB inactive:163992kB present:532480kB pages_scanned:12263677 all_unreclaimable? yes Apr 2 17:18:28 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 17:18:28 s_local at webmail kernel:DMA32 free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 17:18:28 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 17:18:28 s_local at webmail kernel:Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 17:18:28 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 17:18:28 s_local at webmail kernel:HighMem free:0kB min:128kB low:128kB high:128kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 17:18:28 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 17:18:28 s_local at webmail kernel:DMA: 12*4kB 8*8kB 3*16kB 0*32kB 1*64kB 1*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 2912kB Apr 2 17:18:28 s_local at webmail kernel:DMA32: empty Apr 2 17:18:28 s_local at webmail kernel:Normal: empty Apr 2 17:18:28 s_local at webmail kernel:HighMem: empty Apr 2 17:18:28 s_local at webmail kernel:89 pagecache pages Apr 2 17:18:28 s_local at webmail kernel:Swap cache: add 176874, delete 176874, find 114990/119550, race 0+0 Apr 2 17:18:28 s_local at webmail kernel:Free swap = 0kB Apr 2 17:18:28 s_local at webmail kernel:Total swap = 557048kB Apr 2 17:18:28 s_local at webmail kernel:Free swap: 0kB Apr 2 17:18:28 s_local at webmail kernel:133120 pages of RAM Apr 2 17:18:28 s_local at webmail kernel:5993 reserved pages Apr 2 17:18:28 s_local at webmail kernel:4331 pages shared Apr 2 17:18:28 s_local at webmail kernel:0 pages swap cached Apr 2 18:05:47 s_local at webmail kernel:init invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0 Apr 2 18:05:47 s_local at webmail kernel: Apr 2 18:05:47 s_local at webmail kernel:Call Trace: Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff802bc998>] out_of_memory+0x8b/0x203 Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff8020f657>] __alloc_pages+0x245/0x2ce Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff8021336e>] __do_page_cache_readahead+0xd0/0x21c Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff80262824>] __wait_on_bit_lock+0x5b/0x66 Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff88081d4d>] :dm_mod:dm_any_congested+0x38/0x3f Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff80213c47>] filemap_nopage+0x148/0x322 Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff80208db9>] __handle_mm_fault+0x440/0x11f6 Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff8020622a>] hypercall_page+0x22a/0x1000 Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff802639f9>] _spin_lock_irqsave+0x9/0x14 Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff802666ef>] do_page_fault+0xf7b/0x12e0 Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff8025f82b>] error_exit+0x0/0x6e Apr 2 18:05:47 s_local at webmail kernel: Apr 2 18:05:47 s_local at webmail kernel:Mem-info: Apr 2 18:05:47 s_local at webmail kernel:DMA per-cpu: Apr 2 18:05:47 s_local at webmail kernel:cpu 0 hot: high 186, batch 31 used:12 Apr 2 18:05:47 s_local at webmail kernel:cpu 0 cold: high 62, batch 15 used:57 Apr 2 18:05:47 s_local at webmail kernel:DMA32 per-cpu: empty Apr 2 18:05:47 s_local at webmail kernel:Normal per-cpu: empty Apr 2 18:05:47 s_local at webmail kernel:HighMem per-cpu: empty Apr 2 18:05:47 s_local at webmail kernel:Free pages: 2868kB (0kB HighMem) Apr 2 18:05:47 s_local at webmail kernel:Active:57202 inactive:56889 dirty:0 writeback:0 unstable:0 free:717 slab:4124 mapped-file:2 mapped-anon:114137 pagetables:3184 Apr 2 18:05:47 s_local at webmail kernel:DMA free:2868kB min:2916kB low:3644kB high:4372kB active:228808kB inactive:227556kB present:532480kB pages_scanned:708700 all_unreclaimable? yes Apr 2 18:05:47 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 18:05:47 s_local at webmail kernel:DMA32 free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 18:05:47 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 18:05:47 s_local at webmail kernel:Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 18:05:47 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 18:05:47 s_local at webmail kernel:HighMem free:0kB min:128kB low:128kB high:128kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 18:05:47 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 18:05:47 s_local at webmail kernel:DMA: 31*4kB 1*8kB 1*16kB 1*32kB 0*64kB 1*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 2868kB Apr 2 18:05:47 s_local at webmail kernel:DMA32: empty Apr 2 18:05:47 s_local at webmail kernel:Normal: empty Apr 2 18:05:47 s_local at webmail kernel:HighMem: empty Apr 2 18:05:47 s_local at webmail kernel:89 pagecache pages Apr 2 18:05:47 s_local at webmail kernel:Swap cache: add 248675, delete 248675, find 135538/146503, race 0+0 Apr 2 18:05:47 s_local at webmail kernel:Free swap = 0kB Apr 2 18:05:47 s_local at webmail kernel:Total swap = 557048kB Apr 2 18:05:47 s_local at webmail kernel:Free swap: 0kB Apr 2 18:05:47 s_local at webmail kernel:133120 pages of RAM Apr 2 18:05:47 s_local at webmail kernel:5993 reserved pages Apr 2 18:05:47 s_local at webmail kernel:4606 pages shared Apr 2 18:05:47 s_local at webmail kernel:0 pages swap cached Apr 2 18:05:47 s_local at webmail kernel:Out of memory: Killed process 21335 (httpd). Apr 2 18:05:47 s_local at webmail kernel:init invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0 Apr 2 18:05:47 s_local at webmail kernel: Apr 2 18:05:47 s_local at webmail kernel:Call Trace: Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff802bc998>] out_of_memory+0x8b/0x203 Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff8020f657>] __alloc_pages+0x245/0x2ce Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff8021336e>] __do_page_cache_readahead+0xd0/0x21c Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff80262824>] __wait_on_bit_lock+0x5b/0x66 Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff88081d4d>] :dm_mod:dm_any_congested+0x38/0x3f Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff80213c47>] filemap_nopage+0x148/0x322 Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff80208db9>] __handle_mm_fault+0x440/0x11f6 Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff8020622a>] hypercall_page+0x22a/0x1000 Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff802639f9>] _spin_lock_irqsave+0x9/0x14 Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff802666ef>] do_page_fault+0xf7b/0x12e0 Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff8025f82b>] error_exit+0x0/0x6e Apr 2 18:05:47 s_local at webmail kernel: Apr 2 18:05:47 s_local at webmail kernel:Mem-info: Apr 2 18:05:47 s_local at webmail kernel:DMA per-cpu: Apr 2 18:05:47 s_local at webmail kernel:cpu 0 hot: high 186, batch 31 used:19 Apr 2 18:05:47 s_local at webmail kernel:cpu 0 cold: high 62, batch 15 used:57 Apr 2 18:05:47 s_local at webmail kernel:DMA32 per-cpu: empty Apr 2 18:05:47 s_local at webmail kernel:Normal per-cpu: empty Apr 2 18:05:47 s_local at webmail kernel:HighMem per-cpu: empty Apr 2 18:05:47 s_local at webmail kernel:Free pages: 2868kB (0kB HighMem) Apr 2 18:05:47 s_local at webmail kernel:Active:73107 inactive:40952 dirty:0 writeback:0 unstable:0 free:717 slab:4118 mapped-file:2 mapped-anon:114137 pagetables:3184 Apr 2 18:05:47 s_local at webmail kernel:DMA free:2868kB min:2916kB low:3644kB high:4372kB active:292428kB inactive:163808kB present:532480kB pages_scanned:4108148 all_unreclaimable? yes Apr 2 18:05:47 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 18:05:47 s_local at webmail kernel:DMA32 free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 18:05:47 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 18:05:47 s_local at webmail kernel:Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 18:05:47 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 18:05:47 s_local at webmail kernel:HighMem free:0kB min:128kB low:128kB high:128kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 18:05:47 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 18:05:47 s_local at webmail kernel:DMA: 31*4kB 1*8kB 1*16kB 1*32kB 0*64kB 1*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 2868kB Apr 2 18:05:47 s_local at webmail kernel:DMA32: empty Apr 2 18:05:47 s_local at webmail kernel:Normal: empty Apr 2 18:05:47 s_local at webmail kernel:HighMem: empty Apr 2 18:05:47 s_local at webmail kernel:83 pagecache pages Apr 2 18:05:47 s_local at webmail kernel:Swap cache: add 248675, delete 248675, find 135538/146503, race 0+0 Apr 2 18:05:47 s_local at webmail kernel:Free swap = 0kB Apr 2 18:05:47 s_local at webmail kernel:Total swap = 557048kB Apr 2 18:05:47 s_local at webmail kernel:Free swap: 0kB Apr 2 18:05:47 s_local at webmail kernel:133120 pages of RAM Apr 2 18:05:47 s_local at webmail kernel:5993 reserved pages Apr 2 18:05:47 s_local at webmail kernel:4638 pages shared Apr 2 18:05:47 s_local at webmail kernel:0 pages swap cached Apr 2 18:05:47 s_local at webmail kernel:init invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0 Apr 2 18:05:47 s_local at webmail kernel: Apr 2 18:05:47 s_local at webmail kernel:Call Trace: Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff802bc998>] out_of_memory+0x8b/0x203 Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff8020f657>] __alloc_pages+0x245/0x2ce Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff8021336e>] __do_page_cache_readahead+0xd0/0x21c Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff80262824>] __wait_on_bit_lock+0x5b/0x66 Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff88081d4d>] :dm_mod:dm_any_congested+0x38/0x3f Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff80213c47>] filemap_nopage+0x148/0x322 Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff80208db9>] __handle_mm_fault+0x440/0x11f6 Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff8020622a>] hypercall_page+0x22a/0x1000 Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff802639f9>] _spin_lock_irqsave+0x9/0x14 Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff802666ef>] do_page_fault+0xf7b/0x12e0 Apr 2 18:05:47 s_local at webmail kernel:[<ffffffff8025f82b>] error_exit+0x0/0x6e Apr 2 18:05:47 s_local at webmail kernel: Apr 2 18:05:47 s_local at webmail kernel:Mem-info: Apr 2 18:05:47 s_local at webmail kernel:DMA per-cpu: Apr 2 18:05:47 s_local at webmail kernel:cpu 0 hot: high 186, batch 31 used:26 Apr 2 18:05:47 s_local at webmail kernel:cpu 0 cold: high 62, batch 15 used:57 Apr 2 18:05:47 s_local at webmail kernel:DMA32 per-cpu: empty Apr 2 18:05:47 s_local at webmail kernel:Normal per-cpu: empty Apr 2 18:05:47 s_local at webmail kernel:HighMem per-cpu: empty Apr 2 18:05:47 s_local at webmail kernel:Free pages: 2868kB (0kB HighMem) Apr 2 18:05:47 s_local at webmail kernel:Active:72986 inactive:41009 dirty:0 writeback:0 unstable:0 free:717 slab:4111 mapped-file:2 mapped-anon:114137 pagetables:3184 Apr 2 18:05:47 s_local at webmail kernel:DMA free:2868kB min:2916kB low:3644kB high:4372kB active:291944kB inactive:164036kB present:532480kB pages_scanned:7458532 all_unreclaimable? yes Apr 2 18:05:47 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 18:05:47 s_local at webmail kernel:DMA32 free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 18:05:47 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 18:05:47 s_local at webmail kernel:Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 18:05:47 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 18:05:47 s_local at webmail kernel:HighMem free:0kB min:128kB low:128kB high:128kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 18:05:47 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 18:05:47 s_local at webmail kernel:DMA: 31*4kB 1*8kB 1*16kB 1*32kB 0*64kB 1*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 2868kB Apr 2 18:05:47 s_local at webmail kernel:DMA32: empty Apr 2 18:05:47 s_local at webmail kernel:Normal: empty Apr 2 18:05:47 s_local at webmail kernel:HighMem: empty Apr 2 18:05:47 s_local at webmail kernel:83 pagecache pages Apr 2 18:05:47 s_local at webmail kernel:Swap cache: add 248675, delete 248675, find 135538/146503, race 0+0 Apr 2 18:05:47 s_local at webmail kernel:Free swap = 0kB Apr 2 18:05:47 s_local at webmail kernel:Total swap = 557048kB Apr 2 18:05:47 s_local at webmail kernel:Free swap: 0kB Apr 2 18:05:47 s_local at webmail kernel:133120 pages of RAM Apr 2 18:05:47 s_local at webmail kernel:5993 reserved pages Apr 2 18:05:47 s_local at webmail kernel:4702 pages shared Apr 2 18:05:47 s_local at webmail kernel:0 pages swap cached Apr 2 18:14:47 s_local at webmail kernel:httpd invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0 Apr 2 18:14:47 s_local at webmail kernel: Apr 2 18:14:47 s_local at webmail kernel:Call Trace: Apr 2 18:14:47 s_local at webmail kernel:[<ffffffff802bc998>] out_of_memory+0x8b/0x203 Apr 2 18:14:47 s_local at webmail kernel:[<ffffffff8020f657>] __alloc_pages+0x245/0x2ce Apr 2 18:14:47 s_local at webmail kernel:[<ffffffff8021336e>] __do_page_cache_readahead+0xd0/0x21c Apr 2 18:14:47 s_local at webmail kernel:[<ffffffff80262824>] __wait_on_bit_lock+0x5b/0x66 Apr 2 18:14:47 s_local at webmail kernel:[<ffffffff88081d4d>] :dm_mod:dm_any_congested+0x38/0x3f Apr 2 18:14:47 s_local at webmail kernel:[<ffffffff80213c47>] filemap_nopage+0x148/0x322 Apr 2 18:14:47 s_local at webmail kernel:[<ffffffff80208db9>] __handle_mm_fault+0x440/0x11f6 Apr 2 18:14:47 s_local at webmail kernel:[<ffffffff802639f9>] _spin_lock_irqsave+0x9/0x14 Apr 2 18:14:47 s_local at webmail kernel:[<ffffffff802666ef>] do_page_fault+0xf7b/0x12e0 Apr 2 18:14:47 s_local at webmail kernel:[<ffffffff80299fd8>] autoremove_wake_function+0x0/0x2e Apr 2 18:14:47 s_local at webmail kernel:[<ffffffff803164c8>] file_has_perm+0x48/0xa3 Apr 2 18:14:47 s_local at webmail kernel:[<ffffffff8025f82b>] error_exit+0x0/0x6e Apr 2 18:14:47 s_local at webmail kernel: Apr 2 18:14:47 s_local at webmail kernel:Mem-info: Apr 2 18:14:47 s_local at webmail kernel:DMA per-cpu: Apr 2 18:14:47 s_local at webmail kernel:cpu 0 hot: high 186, batch 31 used:25 Apr 2 18:14:47 s_local at webmail kernel:cpu 0 cold: high 62, batch 15 used:14 Apr 2 18:14:47 s_local at webmail kernel:DMA32 per-cpu: empty Apr 2 18:14:47 s_local at webmail kernel:Normal per-cpu: empty Apr 2 18:14:47 s_local at webmail kernel:HighMem per-cpu: empty Apr 2 18:14:47 s_local at webmail kernel:Free pages: 2876kB (0kB HighMem) Apr 2 18:14:47 s_local at webmail kernel:Active:54916 inactive:59014 dirty:0 writeback:0 unstable:0 free:719 slab:4154 mapped-file:5 mapped-anon:113909 pagetables:3325 Apr 2 18:14:47 s_local at webmail kernel:DMA free:2876kB min:2916kB low:3644kB high:4372kB active:219664kB inactive:236056kB present:532480kB pages_scanned:738327 all_unreclaimable? yes Apr 2 18:14:47 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 18:14:47 s_local at webmail kernel:DMA32 free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 18:14:47 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 18:14:47 s_local at webmail kernel:Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 18:14:47 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 18:14:47 s_local at webmail kernel:HighMem free:0kB min:128kB low:128kB high:128kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 18:14:47 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 18:14:47 s_local at webmail kernel:DMA: 1*4kB 11*8kB 0*16kB 3*32kB 0*64kB 1*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 2876kB Apr 2 18:14:47 s_local at webmail kernel:DMA32: empty Apr 2 18:14:47 s_local at webmail kernel:Normal: empty Apr 2 18:14:47 s_local at webmail kernel:HighMem: empty Apr 2 18:14:47 s_local at webmail kernel:126 pagecache pages Apr 2 18:14:47 s_local at webmail kernel:Swap cache: add 256186, delete 256186, find 137948/149847, race 0+0 Apr 2 18:14:47 s_local at webmail kernel:Free swap = 0kB Apr 2 18:14:47 s_local at webmail kernel:Total swap = 557048kB Apr 2 18:14:47 s_local at webmail kernel:Free swap: 0kB Apr 2 18:14:47 s_local at webmail kernel:133120 pages of RAM Apr 2 18:14:47 s_local at webmail kernel:5993 reserved pages Apr 2 18:14:47 s_local at webmail kernel:4863 pages shared Apr 2 18:14:47 s_local at webmail kernel:0 pages swap cached Apr 2 18:14:47 s_local at webmail kernel:Out of memory: Killed process 21332 (httpd). Apr 2 18:23:47 s_local at webmail kernel:automount invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0 Apr 2 18:23:47 s_local at webmail kernel: Apr 2 18:23:47 s_local at webmail kernel:Call Trace: Apr 2 18:23:47 s_local at webmail kernel:[<ffffffff802bc998>] out_of_memory+0x8b/0x203 Apr 2 18:23:47 s_local at webmail kernel:[<ffffffff8020f657>] __alloc_pages+0x245/0x2ce Apr 2 18:23:47 s_local at webmail kernel:[<ffffffff8021336e>] __do_page_cache_readahead+0xd0/0x21c Apr 2 18:23:47 s_local at webmail kernel:[<ffffffff80262824>] __wait_on_bit_lock+0x5b/0x66 Apr 2 18:23:47 s_local at webmail kernel:[<ffffffff88081d4d>] :dm_mod:dm_any_congested+0x38/0x3f Apr 2 18:23:47 s_local at webmail kernel:[<ffffffff80213c47>] filemap_nopage+0x148/0x322 Apr 2 18:23:47 s_local at webmail kernel:[<ffffffff80208db9>] __handle_mm_fault+0x440/0x11f6 Apr 2 18:23:47 s_local at webmail kernel:[<ffffffff802639f9>] _spin_lock_irqsave+0x9/0x14 Apr 2 18:23:47 s_local at webmail kernel:[<ffffffff802666ef>] do_page_fault+0xf7b/0x12e0 Apr 2 18:23:47 s_local at webmail kernel:[<ffffffff802ad687>] audit_syscall_entry+0xf2/0x1a1 Apr 2 18:23:47 s_local at webmail kernel:[<ffffffff8025f82b>] error_exit+0x0/0x6e Apr 2 18:23:47 s_local at webmail kernel: Apr 2 18:23:47 s_local at webmail kernel:Mem-info: Apr 2 18:23:47 s_local at webmail kernel:DMA per-cpu: Apr 2 18:23:47 s_local at webmail kernel:cpu 0 hot: high 186, batch 31 used:30 Apr 2 18:23:47 s_local at webmail kernel:cpu 0 cold: high 62, batch 15 used:14 Apr 2 18:23:47 s_local at webmail kernel:DMA32 per-cpu: empty Apr 2 18:23:47 s_local at webmail kernel:Normal per-cpu: empty Apr 2 18:23:47 s_local at webmail kernel:HighMem per-cpu: empty Apr 2 18:23:47 s_local at webmail kernel:Free pages: 2908kB (0kB HighMem) Apr 2 18:23:47 s_local at webmail kernel:Active:56901 inactive:56963 dirty:0 writeback:0 unstable:0 free:727 slab:4164 mapped-file:6 mapped-anon:113966 pagetables:3306 Apr 2 18:23:47 s_local at webmail kernel:DMA free:2908kB min:2916kB low:3644kB high:4372kB active:227604kB inactive:227852kB present:532480kB pages_scanned:807511 all_unreclaimable? yes Apr 2 18:23:47 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 18:23:47 s_local at webmail kernel:DMA32 free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 18:23:47 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 18:23:47 s_local at webmail kernel:Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 18:23:47 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 18:23:47 s_local at webmail kernel:HighMem free:0kB min:128kB low:128kB high:128kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Apr 2 18:23:47 s_local at webmail kernel:lowmem_reserve[]: 0 0 0 0 Apr 2 18:23:47 s_local at webmail kernel:DMA: 37*4kB 1*8kB 0*16kB 2*32kB 0*64kB 1*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 2908kB Apr 2 18:23:47 s_local at webmail kernel:DMA32: empty Apr 2 18:23:47 s_local at webmail kernel:Normal: empty Apr 2 18:23:47 s_local at webmail kernel:HighMem: empty Apr 2 18:23:47 s_local at webmail kernel:147 pagecache pages Apr 2 18:23:47 s_local at webmail kernel:Swap cache: add 267431, delete 267431, find 138909/152168, race 0+0 Apr 2 18:23:47 s_local at webmail kernel:Free swap = 0kB Apr 2 18:23:47 s_local at webmail kernel:Total swap = 557048kB Apr 2 18:23:47 s_local at webmail kernel:Free swap: 0kB Apr 2 18:23:47 s_local at webmail kernel:133120 pages of RAM Apr 2 18:23:47 s_local at webmail kernel:5993 reserved pages Apr 2 18:23:47 s_local at webmail kernel:4965 pages shared Apr 2 18:23:47 s_local at webmail kernel:0 pages swap cached Apr 2 18:23:47 s_local at webmail kernel:Out of memory: Killed process 21336 (httpd).
