The system stopped responding to ssh but still responded to ping. The logs showed this error: (its long)
Feb 4 00:08:08 server1 kernel: gdmgreeter: page allocation failure. order:0, mode:0x50 Feb 4 00:08:08 server1 kernel: [<c014f863>] __alloc_pages+0x28b/0x29d Feb 4 00:08:08 server1 kernel: [<c014b70f>] find_or_create_page+0x39/0x72 Feb 4 00:08:08 server1 kernel: [<c016eda7>] grow_dev_page+0x2a/0x1eb Feb 4 00:08:08 server1 kernel: [<c016f096>] __getblk_slow+0x12e/0x159 Feb 4 00:08:08 server1 kernel: [<c016f40e>] __getblk+0x3f/0x49 Feb 4 00:08:08 server1 kernel: [<c016f455>] __bread+0x9/0x22 Feb 4 00:08:08 server1 kernel: [<f88af239>] ext3_free_branches+0x91/0x1be [ext3] Feb 4 00:08:08 server1 kernel: [<f88af27e>] ext3_free_branches+0xd6/0x1be [ext3] Feb 4 00:08:08 server1 kernel: [<f88af702>] ext3_truncate+0x39c/0x460 [ext3] Feb 4 00:08:08 server1 kernel: [<c0317627>] __cond_resched+0x14/0x3b Feb 4 00:08:08 server1 kernel: [<f8862737>] journal_start+0x78/0x9e [jbd] Feb 4 00:08:08 server1 kernel: [<f88ad4b5>] ext3_delete_inode+0x60/0xaa [ext3] Feb 4 00:08:23 server1 kernel: [<f88ad455>] ext3_delete_inode+0x0/0xaa [ext3] Feb 4 00:08:23 server1 kernel: [<c018adbb>] generic_delete_inode+0xef/0x229 Feb 4 00:08:23 server1 nrpe[2751]: Error: Could not complete SSL handshake. 5 Feb 4 00:08:23 server1 nrpe[2753]: Error: Could not complete SSL handshake. 5 Feb 4 00:08:23 server1 kernel: [<c018b1e2>] iput+0x5f/0x61 Feb 4 00:08:23 server1 kernel: [<c018633c>] prune_dcache+0x50d/0x70c Feb 4 00:08:23 server1 kernel: [<c0186df1>] shrink_dcache_memory+0x16/0x2d Feb 4 00:08:23 server1 kernel: [<c015628b>] shrink_slab+0xf7/0x14c Feb 4 00:08:23 server1 kernel: [<c0157648>] try_to_free_pages+0xb9/0x189 Feb 4 00:08:23 server1 kernel: [<c014f78d>] __alloc_pages+0x1b5/0x29d Feb 4 00:08:23 server1 kernel: [<c014f88d>] __get_free_pages+0x18/0x24 Feb 4 00:08:23 server1 kernel: [<c0180962>] __pollwait+0x2d/0x94 Feb 4 00:08:23 server1 kernel: [<c031269a>] unix_poll+0x24/0xa3 Feb 4 00:08:23 server1 kernel: [<c02b40f4>] sock_poll+0x12/0x14 Feb 4 00:08:23 server1 kernel: [<c018127a>] do_pollfd+0x47/0x81 Feb 4 00:08:23 server1 kernel: [<c01812fd>] do_poll+0x49/0xab Feb 4 00:08:23 server1 kernel: [<c018152c>] sys_poll+0x1cd/0x281 Feb 4 00:08:23 server1 kernel: [<c0180935>] __pollwait+0x0/0x94 Feb 4 00:08:23 server1 kernel: [<c0129189>] sys_gettimeofday+0x53/0xac Feb 4 00:08:23 server1 kernel: [<c0318eeb>] syscall_call+0x7/0xb Feb 4 00:08:23 server1 kernel: Mem-info: Feb 4 00:08:23 server1 kernel: DMA per-cpu: Feb 4 00:08:23 server1 kernel: cpu 0 hot: low 2, high 6, batch 1 Feb 4 00:08:23 server1 kernel: cpu 0 cold: low 0, high 2, batch 1 Feb 4 00:08:23 server1 kernel: Normal per-cpu: Feb 4 00:08:23 server1 kernel: cpu 0 hot: low 32, high 96, batch 16 Feb 4 00:08:23 server1 kernel: cpu 0 cold: low 0, high 32, batch 16 Feb 4 00:08:23 server1 kernel: HighMem per-cpu: Feb 4 00:08:23 server1 kernel: cpu 0 hot: low 10, high 30, batch 5 Feb 4 00:08:23 server1 kernel: cpu 0 cold: low 0, high 10, batch 5 Feb 4 00:08:23 server1 kernel: Feb 4 00:08:23 server1 kernel: Free pages: 49280kB (49280kB HighMem) Feb 4 00:08:23 server1 kernel: Active:28662 inactive:196240 dirty:60 writeback:0 unstable:0 free:12320 slab:11019 mapped:9351 pagetables:253 Feb 4 00:08:23 server1 kernel: DMA free:0kB min:16kB low:32kB high:48kB active:36kB inactive:12184kB present:16384kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:23 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:23 server1 kernel: Normal free:0kB min:936kB low:1872kB high:2808kB active:93152kB inactive:747560kB present:901120kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:23 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:23 server1 kernel: HighMem free:49280kB min:128kB low:256kB high:384kB active:21460kB inactive:25216kB present:98240kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:23 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:23 server1 kernel: DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB Feb 4 00:08:23 server1 kernel: Normal: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB Feb 4 00:08:23 server1 kernel: HighMem: 1614*4kB 1807*8kB 1105*16kB 290*32kB 20*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 49280kB Feb 4 00:08:23 server1 kernel: Swap cache: add 40, delete 40, find 0/0, race 0+0 Feb 4 00:08:23 server1 kernel: 0 bounce buffer pages Feb 4 00:08:23 server1 kernel: Free swap: 1052088kB Feb 4 00:08:23 server1 kernel: 253936 pages of RAM Feb 4 00:08:23 server1 kernel: 24560 pages of HIGHMEM Feb 4 00:08:23 server1 kernel: 3209 reserved pages Feb 4 00:08:23 server1 kernel: 199769 pages shared Feb 4 00:08:23 server1 kernel: 0 pages swap cached Feb 4 00:08:23 server1 kernel: kswapd0: page allocation failure. order:0, mode:0x50 Feb 4 00:08:23 server1 kernel: [<c014f863>] __alloc_pages+0x28b/0x29d Feb 4 00:08:23 server1 kernel: [<c014b70f>] find_or_create_page+0x39/0x72 Feb 4 00:08:23 server1 kernel: [<c016eda7>] grow_dev_page+0x2a/0x1eb Feb 4 00:08:23 server1 kernel: [<c016f096>] __getblk_slow+0x12e/0x159 Feb 4 00:08:23 server1 kernel: [<c016f40e>] __getblk+0x3f/0x49 Feb 4 00:08:23 server1 kernel: [<c016f455>] __bread+0x9/0x22 Feb 4 00:08:23 server1 kernel: [<f88af239>] ext3_free_branches+0x91/0x1be [ext3] Feb 4 00:08:23 server1 kernel: [<f88af27e>] ext3_free_branches+0xd6/0x1be [ext3] Feb 4 00:08:23 server1 kernel: [<f88af702>] ext3_truncate+0x39c/0x460 [ext3] Feb 4 00:08:23 server1 kernel: [<c0317627>] __cond_resched+0x14/0x3b Feb 4 00:08:23 server1 kernel: [<f8862737>] journal_start+0x78/0x9e [jbd] Feb 4 00:08:23 server1 kernel: [<f88ad4b5>] ext3_delete_inode+0x60/0xaa [ext3] Feb 4 00:08:23 server1 kernel: [<f88ad455>] ext3_delete_inode+0x0/0xaa [ext3] Feb 4 00:08:23 server1 kernel: [<c018adbb>] generic_delete_inode+0xef/0x229 Feb 4 00:08:23 server1 kernel: [<c018b1e2>] iput+0x5f/0x61 Feb 4 00:08:23 server1 kernel: [<c018633c>] prune_dcache+0x50d/0x70c Feb 4 00:08:23 server1 kernel: [<c0186df1>] shrink_dcache_memory+0x16/0x2d Feb 4 00:08:23 server1 kernel: [<c015628b>] shrink_slab+0xf7/0x14c Feb 4 00:08:23 server1 kernel: [<c01578cb>] balance_pgdat+0x1b3/0x2cb Feb 4 00:08:23 server1 kernel: [<c0157a9c>] kswapd+0xb9/0xbb Feb 4 00:08:23 server1 kernel: [<c0121853>] autoremove_wake_function+0x0/0x2d Feb 4 00:08:23 server1 kernel: [<c0318e06>] ret_from_fork+0x6/0x14 Feb 4 00:08:23 server1 kernel: [<c0121853>] autoremove_wake_function+0x0/0x2d Feb 4 00:08:23 server1 kernel: [<c01579e3>] kswapd+0x0/0xbb Feb 4 00:08:23 server1 kernel: [<c01041dd>] kernel_thread_helper+0x5/0xb Feb 4 00:08:23 server1 kernel: Mem-info: Feb 4 00:08:23 server1 kernel: DMA per-cpu: Feb 4 00:08:23 server1 kernel: cpu 0 hot: low 2, high 6, batch 1 Feb 4 00:08:23 server1 kernel: cpu 0 cold: low 0, high 2, batch 1 Feb 4 00:08:23 server1 kernel: Normal per-cpu: Feb 4 00:08:23 server1 kernel: cpu 0 hot: low 32, high 96, batch 16 Feb 4 00:08:23 server1 kernel: cpu 0 cold: low 0, high 32, batch 16 Feb 4 00:08:23 server1 kernel: HighMem per-cpu: Feb 4 00:08:23 server1 kernel: cpu 0 hot: low 10, high 30, batch 5 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 10, batch 5 Feb 4 00:08:24 server1 kernel: Feb 4 00:08:24 server1 kernel: Free pages: 49280kB (49280kB HighMem) Feb 4 00:08:24 server1 kernel: Active:28663 inactive:196239 dirty:60 writeback:0 unstable:0 free:12320 slab:11019 mapped:9351 pagetables:253 Feb 4 00:08:24 server1 kernel: DMA free:0kB min:16kB low:32kB high:48kB active:36kB inactive:12184kB present:16384kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: Normal free:0kB min:936kB low:1872kB high:2808kB active:93156kB inactive:747556kB present:901120kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: HighMem free:49280kB min:128kB low:256kB high:384kB active:21460kB inactive:25216kB present:98240kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB Feb 4 00:08:24 server1 kernel: Normal: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB Feb 4 00:08:24 server1 kernel: HighMem: 1614*4kB 1807*8kB 1105*16kB 290*32kB 20*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 49280kB Feb 4 00:08:24 server1 kernel: Swap cache: add 40, delete 40, find 0/0, race 0+0 Feb 4 00:08:24 server1 kernel: 0 bounce buffer pages Feb 4 00:08:24 server1 kernel: Free swap: 1052088kB Feb 4 00:08:24 server1 kernel: 253936 pages of RAM Feb 4 00:08:24 server1 kernel: 24560 pages of HIGHMEM Feb 4 00:08:24 server1 kernel: 3209 reserved pages Feb 4 00:08:24 server1 kernel: 199769 pages shared Feb 4 00:08:24 server1 kernel: 0 pages swap cached Feb 4 00:08:24 server1 kernel: nfsd: page allocation failure. order:0, mode:0x50 Feb 4 00:08:24 server1 kernel: [<c014f863>] __alloc_pages+0x28b/0x29d Feb 4 00:08:24 server1 kernel: [<c014b70f>] find_or_create_page+0x39/0x72 Feb 4 00:08:24 server1 kernel: [<c016eda7>] grow_dev_page+0x2a/0x1eb Feb 4 00:08:24 server1 kernel: [<c016f096>] __getblk_slow+0x12e/0x159 Feb 4 00:08:24 server1 kernel: [<c016f40e>] __getblk+0x3f/0x49 Feb 4 00:08:24 server1 kernel: [<c016f455>] __bread+0x9/0x22 Feb 4 00:08:24 server1 kernel: [<f88af239>] ext3_free_branches+0x91/0x1be [ext3] Feb 4 00:08:24 server1 kernel: [<f88af27e>] ext3_free_branches+0xd6/0x1be [ext3] Feb 4 00:08:24 server1 kernel: [<f88af27e>] ext3_free_branches+0xd6/0x1be [ext3] Feb 4 00:08:24 server1 kernel: [<f88af736>] ext3_truncate+0x3d0/0x460 [ext3] Feb 4 00:08:24 server1 kernel: [<c0317627>] __cond_resched+0x14/0x3b Feb 4 00:08:24 server1 kernel: [<f88ad4b5>] ext3_delete_inode+0x60/0xaa [ext3] Feb 4 00:08:24 server1 kernel: [<f88ad455>] ext3_delete_inode+0x0/0xaa [ext3] Feb 4 00:08:24 server1 kernel: [<c018adbb>] generic_delete_inode+0xef/0x229 Feb 4 00:08:24 server1 kernel: [<c018b1e2>] iput+0x5f/0x61 Feb 4 00:08:24 server1 kernel: [<c018633c>] prune_dcache+0x50d/0x70c Feb 4 00:08:24 server1 kernel: [<c0186df1>] shrink_dcache_memory+0x16/0x2d Feb 4 00:08:24 server1 kernel: [<c015628b>] shrink_slab+0xf7/0x14c Feb 4 00:08:24 server1 kernel: [<c0157648>] try_to_free_pages+0xb9/0x189 Feb 4 00:08:24 server1 kernel: [<c014f78d>] __alloc_pages+0x1b5/0x29d Feb 4 00:08:24 server1 kernel: [<c014ce77>] generic_file_buffered_write+0x172/0x4db Feb 4 00:08:24 server1 kernel: [<c01299d0>] current_fs_time+0x44/0x4c Feb 4 00:08:24 server1 kernel: [<c014d51d>] __generic_file_aio_write_nolock+0x33d/0x36b Feb 4 00:08:24 server1 kernel: [<c014d64e>] __generic_file_write_nolock+0x84/0x99 Feb 4 00:08:24 server1 kernel: [<c011f422>] recalc_task_prio+0x128/0x133 Feb 4 00:08:24 server1 kernel: [<c0121853>] autoremove_wake_function+0x0/0x2d Feb 4 00:08:24 server1 kernel: [<c014d971>] generic_file_writev+0x54/0xb3 Feb 4 00:08:24 server1 kernel: [<c014d91d>] generic_file_writev+0x0/0xb3 Feb 4 00:08:24 server1 kernel: [<c016c605>] do_readv_writev+0x1c5/0x245 Feb 4 00:08:24 server1 kernel: [<c016c0f2>] do_sync_write+0x0/0xc9 Feb 4 00:08:24 server1 kernel: [<c0317627>] __cond_resched+0x14/0x3b Feb 4 00:08:24 server1 kernel: [<c016b335>] __dentry_open+0xca/0x16a Feb 4 00:08:24 server1 kernel: [<c016b266>] dentry_open+0x48/0x4d Feb 4 00:08:24 server1 kernel: [<c016c703>] vfs_writev+0x3e/0x43 Feb 4 00:08:24 server1 kernel: [<f8b39600>] nfsd_write+0xeb/0x28f [nfsd] Feb 4 00:08:24 server1 kernel: [<c0317627>] __cond_resched+0x14/0x3b Feb 4 00:08:24 server1 kernel: [<f8b40ee2>] nfsd3_proc_write+0xbf/0xd5 [nfsd] Feb 4 00:08:24 server1 kernel: [<f8b42f90>] nfs3svc_decode_writeargs+0x0/0x243 [nfsd] Feb 4 00:08:24 server1 kernel: [<f8b35947>] nfsd_dispatch+0xba/0x16f [nfsd] Feb 4 00:08:24 server1 kernel: [<f8ace74c>] svc_process+0x432/0x6e2 [sunrpc] Feb 4 00:08:24 server1 kernel: [<f8b355eb>] nfsd+0x2a7/0x549 [nfsd] Feb 4 00:08:24 server1 kernel: [<f8b35344>] nfsd+0x0/0x549 [nfsd] Feb 4 00:08:24 server1 kernel: [<c01041dd>] kernel_thread_helper+0x5/0xb Feb 4 00:08:24 server1 kernel: Mem-info: Feb 4 00:08:24 server1 kernel: DMA per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 2, high 6, batch 1 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 2, batch 1 Feb 4 00:08:24 server1 kernel: Normal per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 32, high 96, batch 16 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 32, batch 16 Feb 4 00:08:24 server1 kernel: HighMem per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 10, high 30, batch 5 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 10, batch 5 Feb 4 00:08:24 server1 kernel: Feb 4 00:08:24 server1 kernel: Free pages: 49280kB (49280kB HighMem) Feb 4 00:08:24 server1 kernel: Active:28663 inactive:196239 dirty:60 writeback:0 unstable:0 free:12320 slab:11019 mapped:9351 pagetables:253 Feb 4 00:08:24 server1 kernel: DMA free:0kB min:16kB low:32kB high:48kB active:36kB inactive:12184kB present:16384kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: Normal free:0kB min:936kB low:1872kB high:2808kB active:93156kB inactive:747556kB present:901120kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: HighMem free:49280kB min:128kB low:256kB high:384kB active:21460kB inactive:25216kB present:98240kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB Feb 4 00:08:24 server1 kernel: Normal: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB Feb 4 00:08:24 server1 kernel: HighMem: 1614*4kB 1807*8kB 1105*16kB 290*32kB 20*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 49280kB Feb 4 00:08:24 server1 kernel: Swap cache: add 40, delete 40, find 0/0, race 0+0 Feb 4 00:08:24 server1 kernel: 0 bounce buffer pages Feb 4 00:08:24 server1 kernel: Free swap: 1052088kB Feb 4 00:08:24 server1 kernel: 253936 pages of RAM Feb 4 00:08:24 server1 kernel: 24560 pages of HIGHMEM Feb 4 00:08:24 server1 kernel: 3209 reserved pages Feb 4 00:08:24 server1 kernel: 199769 pages shared Feb 4 00:08:24 server1 kernel: 0 pages swap cached Feb 4 00:08:24 server1 kernel: nrpe: page allocation failure. order:0, mode:0x50 Feb 4 00:08:24 server1 kernel: [<c014f863>] __alloc_pages+0x28b/0x29d Feb 4 00:08:24 server1 kernel: [<c014b70f>] find_or_create_page+0x39/0x72 Feb 4 00:08:24 server1 kernel: [<c016eda7>] grow_dev_page+0x2a/0x1eb Feb 4 00:08:24 server1 kernel: [<c016f096>] __getblk_slow+0x12e/0x159 Feb 4 00:08:24 server1 kernel: [<c016f40e>] __getblk+0x3f/0x49 Feb 4 00:08:24 server1 kernel: [<c016f455>] __bread+0x9/0x22 Feb 4 00:08:24 server1 kernel: [<f88af239>] ext3_free_branches+0x91/0x1be [ext3] Feb 4 00:08:24 server1 kernel: [<f88af27e>] ext3_free_branches+0xd6/0x1be [ext3] Feb 4 00:08:24 server1 kernel: [<f88af702>] ext3_truncate+0x39c/0x460 [ext3] Feb 4 00:08:24 server1 kernel: [<c0317627>] __cond_resched+0x14/0x3b Feb 4 00:08:24 server1 kernel: [<f8862737>] journal_start+0x78/0x9e [jbd] Feb 4 00:08:24 server1 kernel: [<f88ad4b5>] ext3_delete_inode+0x60/0xaa [ext3] Feb 4 00:08:24 server1 kernel: [<f88ad455>] ext3_delete_inode+0x0/0xaa [ext3] Feb 4 00:08:24 server1 kernel: [<c018adbb>] generic_delete_inode+0xef/0x229 Feb 4 00:08:24 server1 kernel: [<c018b1e2>] iput+0x5f/0x61 Feb 4 00:08:24 server1 kernel: [<c018633c>] prune_dcache+0x50d/0x70c Feb 4 00:08:24 server1 kernel: [<c0186df1>] shrink_dcache_memory+0x16/0x2d Feb 4 00:08:24 server1 kernel: [<c015628b>] shrink_slab+0xf7/0x14c Feb 4 00:08:24 server1 kernel: [<c0157648>] try_to_free_pages+0xb9/0x189 Feb 4 00:08:24 server1 kernel: [<c014f78d>] __alloc_pages+0x1b5/0x29d Feb 4 00:08:24 server1 kernel: [<c014f88d>] __get_free_pages+0x18/0x24 Feb 4 00:08:24 server1 kernel: [<c0180962>] __pollwait+0x2d/0x94 Feb 4 00:08:24 server1 kernel: [<c02dd665>] tcp_poll+0x31/0x144 Feb 4 00:08:24 server1 kernel: [<c0188fc8>] destroy_inode+0x3d/0x4c Feb 4 00:08:24 server1 kernel: [<c018b1e2>] iput+0x5f/0x61 Feb 4 00:08:24 server1 kernel: [<c02b40f4>] sock_poll+0x12/0x14 Feb 4 00:08:24 server1 kernel: [<c0180cca>] do_select+0x25c/0x378 Feb 4 00:08:24 server1 kernel: [<c0180935>] __pollwait+0x0/0x94 Feb 4 00:08:24 server1 kernel: [<c01810d9>] sys_select+0x2e0/0x43a Feb 4 00:08:24 server1 kernel: [<c02b520c>] sys_socketcall+0x10c/0x1fb Feb 4 00:08:24 server1 kernel: [<c0318eeb>] syscall_call+0x7/0xb Feb 4 00:08:24 server1 kernel: Mem-info: Feb 4 00:08:24 server1 kernel: DMA per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 2, high 6, batch 1 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 2, batch 1 Feb 4 00:08:24 server1 kernel: Normal per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 32, high 96, batch 16 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 32, batch 16 Feb 4 00:08:24 server1 kernel: HighMem per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 10, high 30, batch 5 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 10, batch 5 Feb 4 00:08:24 server1 kernel: Feb 4 00:08:24 server1 kernel: Free pages: 49280kB (49280kB HighMem) Feb 4 00:08:24 server1 kernel: Active:28664 inactive:196239 dirty:0 writeback:20 unstable:0 free:12320 slab:11019 mapped:9351 pagetables:253 Feb 4 00:08:24 server1 kernel: DMA free:0kB min:16kB low:32kB high:48kB active:40kB inactive:12184kB present:16384kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: Normal free:0kB min:936kB low:1872kB high:2808kB active:93156kB inactive:747556kB present:901120kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: HighMem free:49280kB min:128kB low:256kB high:384kB active:21460kB inactive:25216kB present:98240kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB Feb 4 00:08:24 server1 kernel: Normal: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB Feb 4 00:08:24 server1 kernel: HighMem: 1614*4kB 1807*8kB 1105*16kB 290*32kB 20*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 49280kB Feb 4 00:08:24 server1 kernel: Swap cache: add 40, delete 40, find 0/0, race 0+0 Feb 4 00:08:24 server1 kernel: 0 bounce buffer pages Feb 4 00:08:24 server1 kernel: Free swap: 1052088kB Feb 4 00:08:24 server1 kernel: 253936 pages of RAM Feb 4 00:08:24 server1 kernel: 24560 pages of HIGHMEM Feb 4 00:08:24 server1 kernel: 3209 reserved pages Feb 4 00:08:24 server1 kernel: 199768 pages shared Feb 4 00:08:24 server1 kernel: 0 pages swap cached Feb 4 00:08:24 server1 kernel: nfsd: page allocation failure. order:0, mode:0x50 Feb 4 00:08:24 server1 kernel: [<c014f863>] __alloc_pages+0x28b/0x29d Feb 4 00:08:24 server1 kernel: [<c014b70f>] find_or_create_page+0x39/0x72 Feb 4 00:08:24 server1 kernel: [<c016eda7>] grow_dev_page+0x2a/0x1eb Feb 4 00:08:24 server1 kernel: [<c016f096>] __getblk_slow+0x12e/0x159 Feb 4 00:08:24 server1 kernel: [<c016f40e>] __getblk+0x3f/0x49 Feb 4 00:08:24 server1 kernel: [<c016f455>] __bread+0x9/0x22 Feb 4 00:08:24 server1 kernel: [<f88af239>] ext3_free_branches+0x91/0x1be [ext3] Feb 4 00:08:24 server1 kernel: [<f88af27e>] ext3_free_branches+0xd6/0x1be [ext3] Feb 4 00:08:24 server1 kernel: [<f88af27e>] ext3_free_branches+0xd6/0x1be [ext3] Feb 4 00:08:24 server1 kernel: [<f88af736>] ext3_truncate+0x3d0/0x460 [ext3] Feb 4 00:08:24 server1 kernel: [<c0317627>] __cond_resched+0x14/0x3b Feb 4 00:08:24 server1 kernel: [<f8862737>] journal_start+0x78/0x9e [jbd] Feb 4 00:08:24 server1 kernel: [<f88ad4b5>] ext3_delete_inode+0x60/0xaa [ext3] Feb 4 00:08:24 server1 kernel: [<f88ad455>] ext3_delete_inode+0x0/0xaa [ext3] Feb 4 00:08:24 server1 kernel: [<c018adbb>] generic_delete_inode+0xef/0x229 Feb 4 00:08:24 server1 kernel: [<c018b1e2>] iput+0x5f/0x61 Feb 4 00:08:24 server1 kernel: [<c018633c>] prune_dcache+0x50d/0x70c Feb 4 00:08:24 server1 kernel: [<c0186df1>] shrink_dcache_memory+0x16/0x2d Feb 4 00:08:24 server1 kernel: [<c015628b>] shrink_slab+0xf7/0x14c Feb 4 00:08:24 server1 kernel: [<c0157648>] try_to_free_pages+0xb9/0x189 Feb 4 00:08:24 server1 kernel: [<c014f78d>] __alloc_pages+0x1b5/0x29d Feb 4 00:08:24 server1 kernel: [<c014ce77>] generic_file_buffered_write+0x172/0x4db Feb 4 00:08:24 server1 kernel: [<c01299d0>] current_fs_time+0x44/0x4c Feb 4 00:08:24 server1 kernel: [<c014d51d>] __generic_file_aio_write_nolock+0x33d/0x36b Feb 4 00:08:24 server1 kernel: [<c014d64e>] __generic_file_write_nolock+0x84/0x99 Feb 4 00:08:24 server1 kernel: [<c011f422>] recalc_task_prio+0x128/0x133 Feb 4 00:08:24 server1 kernel: [<c0121853>] autoremove_wake_function+0x0/0x2d Feb 4 00:08:24 server1 kernel: [<c014d971>] generic_file_writev+0x54/0xb3 Feb 4 00:08:24 server1 kernel: [<c014d91d>] generic_file_writev+0x0/0xb3 Feb 4 00:08:24 server1 kernel: [<c016c605>] do_readv_writev+0x1c5/0x245 Feb 4 00:08:24 server1 kernel: [<c016c0f2>] do_sync_write+0x0/0xc9 Feb 4 00:08:24 server1 kernel: [<c0317627>] __cond_resched+0x14/0x3b Feb 4 00:08:24 server1 kernel: [<c016b335>] __dentry_open+0xca/0x16a Feb 4 00:08:24 server1 kernel: [<c016b266>] dentry_open+0x48/0x4d Feb 4 00:08:24 server1 kernel: [<c016c703>] vfs_writev+0x3e/0x43 Feb 4 00:08:24 server1 kernel: [<f8b39600>] nfsd_write+0xeb/0x28f [nfsd] Feb 4 00:08:24 server1 kernel: [<c0317627>] __cond_resched+0x14/0x3b Feb 4 00:08:24 server1 kernel: [<f8b40ee2>] nfsd3_proc_write+0xbf/0xd5 [nfsd] Feb 4 00:08:24 server1 kernel: [<f8b42f90>] nfs3svc_decode_writeargs+0x0/0x243 [nfsd] Feb 4 00:08:24 server1 kernel: [<f8b35947>] nfsd_dispatch+0xba/0x16f [nfsd] Feb 4 00:08:24 server1 kernel: [<f8ace74c>] svc_process+0x432/0x6e2 [sunrpc] Feb 4 00:08:24 server1 kernel: [<f8b355eb>] nfsd+0x2a7/0x549 [nfsd] Feb 4 00:08:24 server1 kernel: [<f8b35344>] nfsd+0x0/0x549 [nfsd] Feb 4 00:08:24 server1 kernel: [<c01041dd>] kernel_thread_helper+0x5/0xb Feb 4 00:08:24 server1 kernel: Mem-info: Feb 4 00:08:24 server1 kernel: DMA per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 2, high 6, batch 1 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 2, batch 1 Feb 4 00:08:24 server1 kernel: Normal per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 32, high 96, batch 16 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 32, batch 16 Feb 4 00:08:24 server1 kernel: HighMem per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 10, high 30, batch 5 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 10, batch 5 Feb 4 00:08:24 server1 kernel: Feb 4 00:08:24 server1 kernel: Free pages: 66820kB (66820kB HighMem) Feb 4 00:08:24 server1 kernel: Active:29068 inactive:191437 dirty:121 writeback:0 unstable:0 free:16705 slab:11020 mapped:9351 pagetables:253 Feb 4 00:08:24 server1 kernel: DMA free:0kB min:16kB low:32kB high:48kB active:36kB inactive:12168kB present:16384kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: Normal free:0kB min:936kB low:1872kB high:2808kB active:94776kB inactive:745908kB present:901120kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: HighMem free:66820kB min:128kB low:256kB high:384kB active:21460kB inactive:7672kB present:98240kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB Feb 4 00:08:24 server1 kernel: Normal: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB Feb 4 00:08:24 server1 kernel: HighMem: 1613*4kB 1804*8kB 1105*16kB 483*32kB 168*64kB 16*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 66820kB Feb 4 00:08:24 server1 kernel: Swap cache: add 40, delete 40, find 0/0, race 0+0 Feb 4 00:08:24 server1 kernel: 0 bounce buffer pages Feb 4 00:08:24 server1 kernel: Free swap: 1052088kB Feb 4 00:08:24 server1 kernel: 253936 pages of RAM Feb 4 00:08:24 server1 kernel: 24560 pages of HIGHMEM Feb 4 00:08:24 server1 kernel: 3209 reserved pages Feb 4 00:08:24 server1 kernel: 199786 pages shared Feb 4 00:08:24 server1 kernel: 0 pages swap cached Feb 4 00:08:24 server1 kernel: gdmgreeter: page allocation failure. order:0, mode:0x50 Feb 4 00:08:24 server1 kernel: [<c014f863>] __alloc_pages+0x28b/0x29d Feb 4 00:08:24 server1 kernel: [<c014b70f>] find_or_create_page+0x39/0x72 Feb 4 00:08:24 server1 kernel: [<c016eda7>] grow_dev_page+0x2a/0x1eb Feb 4 00:08:24 server1 kernel: [<c016f096>] __getblk_slow+0x12e/0x159 Feb 4 00:08:24 server1 kernel: [<c016f40e>] __getblk+0x3f/0x49 Feb 4 00:08:24 server1 kernel: [<c016f455>] __bread+0x9/0x22 Feb 4 00:08:24 server1 kernel: [<f88af239>] ext3_free_branches+0x91/0x1be [ext3] Feb 4 00:08:24 server1 kernel: [<f88af27e>] ext3_free_branches+0xd6/0x1be [ext3] Feb 4 00:08:24 server1 kernel: [<f88af702>] ext3_truncate+0x39c/0x460 [ext3] Feb 4 00:08:24 server1 kernel: [<c0317627>] __cond_resched+0x14/0x3b Feb 4 00:08:24 server1 kernel: [<f8862737>] journal_start+0x78/0x9e [jbd] Feb 4 00:08:24 server1 kernel: [<f88ad4b5>] ext3_delete_inode+0x60/0xaa [ext3] Feb 4 00:08:24 server1 kernel: [<f88ad455>] ext3_delete_inode+0x0/0xaa [ext3] Feb 4 00:08:24 server1 kernel: [<c018adbb>] generic_delete_inode+0xef/0x229 Feb 4 00:08:24 server1 kernel: [<c018b1e2>] iput+0x5f/0x61 Feb 4 00:08:24 server1 kernel: [<c018633c>] prune_dcache+0x50d/0x70c Feb 4 00:08:24 server1 kernel: [<c0186df1>] shrink_dcache_memory+0x16/0x2d Feb 4 00:08:24 server1 kernel: [<c015628b>] shrink_slab+0xf7/0x14c Feb 4 00:08:24 server1 kernel: [<c0157648>] try_to_free_pages+0xb9/0x189 Feb 4 00:08:24 server1 kernel: [<c014f78d>] __alloc_pages+0x1b5/0x29d Feb 4 00:08:24 server1 kernel: [<c014f88d>] __get_free_pages+0x18/0x24 Feb 4 00:08:24 server1 kernel: [<c0180962>] __pollwait+0x2d/0x94 Feb 4 00:08:24 server1 kernel: [<c031269a>] unix_poll+0x24/0xa3 Feb 4 00:08:24 server1 kernel: [<c02b40f4>] sock_poll+0x12/0x14 Feb 4 00:08:24 server1 kernel: [<c018127a>] do_pollfd+0x47/0x81 Feb 4 00:08:24 server1 kernel: [<c01812fd>] do_poll+0x49/0xab Feb 4 00:08:24 server1 kernel: [<c018152c>] sys_poll+0x1cd/0x281 Feb 4 00:08:24 server1 kernel: [<c0180935>] __pollwait+0x0/0x94 Feb 4 00:08:24 server1 kernel: [<c0129189>] sys_gettimeofday+0x53/0xac Feb 4 00:08:24 server1 kernel: [<c0318eeb>] syscall_call+0x7/0xb Feb 4 00:08:24 server1 kernel: Mem-info: Feb 4 00:08:24 server1 kernel: DMA per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 2, high 6, batch 1 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 2, batch 1 Feb 4 00:08:24 server1 kernel: Normal per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 32, high 96, batch 16 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 32, batch 16 Feb 4 00:08:24 server1 kernel: HighMem per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 10, high 30, batch 5 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 10, batch 5 Feb 4 00:08:24 server1 kernel: Feb 4 00:08:24 server1 kernel: Free pages: 66820kB (66820kB HighMem) Feb 4 00:08:24 server1 kernel: Active:29068 inactive:191437 dirty:121 writeback:0 unstable:0 free:16705 slab:11020 mapped:9351 pagetables:253 Feb 4 00:08:24 server1 kernel: DMA free:0kB min:16kB low:32kB high:48kB active:36kB inactive:12168kB present:16384kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: Normal free:0kB min:936kB low:1872kB high:2808kB active:94776kB inactive:745908kB present:901120kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: HighMem free:66820kB min:128kB low:256kB high:384kB active:21460kB inactive:7672kB present:98240kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB Feb 4 00:08:24 server1 kernel: Normal: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB Feb 4 00:08:24 server1 kernel: HighMem: 1613*4kB 1804*8kB 1105*16kB 483*32kB 168*64kB 16*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 66820kB Feb 4 00:08:24 server1 kernel: Swap cache: add 40, delete 40, find 0/0, race 0+0 Feb 4 00:08:24 server1 kernel: 0 bounce buffer pages Feb 4 00:08:24 server1 kernel: Free swap: 1052088kB Feb 4 00:08:24 server1 kernel: 253936 pages of RAM Feb 4 00:08:24 server1 kernel: 24560 pages of HIGHMEM Feb 4 00:08:24 server1 kernel: 3209 reserved pages Feb 4 00:08:24 server1 kernel: 199786 pages shared Feb 4 00:08:24 server1 kernel: 0 pages swap cached Feb 4 00:08:24 server1 kernel: syslogd: page allocation failure. order:0, mode:0x50 Feb 4 00:08:24 server1 kernel: [<c014f863>] __alloc_pages+0x28b/0x29d Feb 4 00:08:24 server1 kernel: [<c014b70f>] find_or_create_page+0x39/0x72 Feb 4 00:08:24 server1 kernel: [<c016eda7>] grow_dev_page+0x2a/0x1eb Feb 4 00:08:24 server1 kernel: [<c016f096>] __getblk_slow+0x12e/0x159 Feb 4 00:08:24 server1 kernel: [<c016f40e>] __getblk+0x3f/0x49 Feb 4 00:08:24 server1 kernel: [<c016f455>] __bread+0x9/0x22 Feb 4 00:08:24 server1 kernel: [<f88af239>] ext3_free_branches+0x91/0x1be [ext3] Feb 4 00:08:24 server1 kernel: [<f88af27e>] ext3_free_branches+0xd6/0x1be [ext3] Feb 4 00:08:24 server1 kernel: [<f88af702>] ext3_truncate+0x39c/0x460 [ext3] Feb 4 00:08:24 server1 kernel: [<c0317627>] __cond_resched+0x14/0x3b Feb 4 00:08:24 server1 kernel: [<f8862737>] journal_start+0x78/0x9e [jbd] Feb 4 00:08:24 server1 kernel: [<f88ad4b5>] ext3_delete_inode+0x60/0xaa [ext3] Feb 4 00:08:24 server1 kernel: [<f88ad455>] ext3_delete_inode+0x0/0xaa [ext3] Feb 4 00:08:24 server1 kernel: [<c018adbb>] generic_delete_inode+0xef/0x229 Feb 4 00:08:24 server1 kernel: [<c018b1e2>] iput+0x5f/0x61 Feb 4 00:08:24 server1 kernel: [<c018633c>] prune_dcache+0x50d/0x70c Feb 4 00:08:24 server1 kernel: [<c0186df1>] shrink_dcache_memory+0x16/0x2d Feb 4 00:08:24 server1 kernel: [<c015628b>] shrink_slab+0xf7/0x14c Feb 4 00:08:24 server1 kernel: [<c0157648>] try_to_free_pages+0xb9/0x189 Feb 4 00:08:24 server1 kernel: [<c014f78d>] __alloc_pages+0x1b5/0x29d Feb 4 00:08:24 server1 kernel: [<c014f88d>] __get_free_pages+0x18/0x24 Feb 4 00:08:24 server1 kernel: [<c0180962>] __pollwait+0x2d/0x94 Feb 4 00:08:24 server1 kernel: [<c02ba096>] datagram_poll+0x25/0xcf Feb 4 00:08:24 server1 kernel: [<c02b40f4>] sock_poll+0x12/0x14 Feb 4 00:08:24 server1 kernel: [<c0180cca>] do_select+0x25c/0x378 Feb 4 00:08:24 server1 kernel: [<c0180935>] __pollwait+0x0/0x94 Feb 4 00:08:24 server1 kernel: [<c01810d9>] sys_select+0x2e0/0x43a Feb 4 00:08:24 server1 kernel: [<c016db6d>] sys_fsync+0xce/0xd6 Feb 4 00:08:24 server1 kernel: [<c0318eeb>] syscall_call+0x7/0xb Feb 4 00:08:24 server1 kernel: Mem-info: Feb 4 00:08:24 server1 kernel: DMA per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 2, high 6, batch 1 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 2, batch 1 Feb 4 00:08:24 server1 kernel: Normal per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 32, high 96, batch 16 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 32, batch 16 Feb 4 00:08:24 server1 kernel: HighMem per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 10, high 30, batch 5 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 10, batch 5 Feb 4 00:08:24 server1 kernel: Feb 4 00:08:24 server1 kernel: Free pages: 66820kB (66820kB HighMem) Feb 4 00:08:24 server1 kernel: Active:29068 inactive:191437 dirty:121 writeback:0 unstable:0 free:16705 slab:11020 mapped:9351 pagetables:253 Feb 4 00:08:24 server1 kernel: DMA free:0kB min:16kB low:32kB high:48kB active:36kB inactive:12168kB present:16384kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: Normal free:0kB min:936kB low:1872kB high:2808kB active:94776kB inactive:745908kB present:901120kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: HighMem free:66820kB min:128kB low:256kB high:384kB active:21460kB inactive:7672kB present:98240kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB Feb 4 00:08:24 server1 kernel: Normal: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB Feb 4 00:08:24 server1 kernel: HighMem: 1613*4kB 1804*8kB 1105*16kB 483*32kB 168*64kB 16*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 66820kB Feb 4 00:08:24 server1 kernel: Swap cache: add 40, delete 40, find 0/0, race 0+0 Feb 4 00:08:24 server1 kernel: 0 bounce buffer pages Feb 4 00:08:24 server1 kernel: Free swap: 1052088kB Feb 4 00:08:24 server1 kernel: 253936 pages of RAM Feb 4 00:08:24 server1 kernel: 24560 pages of HIGHMEM Feb 4 00:08:24 server1 kernel: 3209 reserved pages Feb 4 00:08:24 server1 kernel: 199786 pages shared Feb 4 00:08:24 server1 kernel: 0 pages swap cached Feb 4 00:08:24 server1 kernel: nrpe: page allocation failure. order:0, mode:0x50 Feb 4 00:08:24 server1 kernel: [<c014f863>] __alloc_pages+0x28b/0x29d Feb 4 00:08:24 server1 kernel: [<c014b70f>] find_or_create_page+0x39/0x72 Feb 4 00:08:24 server1 kernel: [<c016eda7>] grow_dev_page+0x2a/0x1eb Feb 4 00:08:24 server1 kernel: [<c016f096>] __getblk_slow+0x12e/0x159 Feb 4 00:08:24 server1 kernel: [<c016f40e>] __getblk+0x3f/0x49 Feb 4 00:08:24 server1 kernel: [<c016f455>] __bread+0x9/0x22 Feb 4 00:08:24 server1 kernel: [<f88af239>] ext3_free_branches+0x91/0x1be [ext3] Feb 4 00:08:24 server1 kernel: [<f88af27e>] ext3_free_branches+0xd6/0x1be [ext3] Feb 4 00:08:24 server1 kernel: [<f88af702>] ext3_truncate+0x39c/0x460 [ext3] Feb 4 00:08:24 server1 kernel: [<c0317627>] __cond_resched+0x14/0x3b Feb 4 00:08:24 server1 kernel: [<f8862737>] journal_start+0x78/0x9e [jbd] Feb 4 00:08:24 server1 kernel: [<f88ad4b5>] ext3_delete_inode+0x60/0xaa [ext3] Feb 4 00:08:24 server1 kernel: [<f88ad455>] ext3_delete_inode+0x0/0xaa [ext3] Feb 4 00:08:24 server1 kernel: [<c018adbb>] generic_delete_inode+0xef/0x229 Feb 4 00:08:24 server1 kernel: [<c018b1e2>] iput+0x5f/0x61 Feb 4 00:08:24 server1 kernel: [<c018633c>] prune_dcache+0x50d/0x70c Feb 4 00:08:24 server1 kernel: [<c0186df1>] shrink_dcache_memory+0x16/0x2d Feb 4 00:08:24 server1 kernel: [<c015628b>] shrink_slab+0xf7/0x14c Feb 4 00:08:24 server1 kernel: [<c0157648>] try_to_free_pages+0xb9/0x189 Feb 4 00:08:24 server1 kernel: [<c014f78d>] __alloc_pages+0x1b5/0x29d Feb 4 00:08:24 server1 kernel: [<c014f88d>] __get_free_pages+0x18/0x24 Feb 4 00:08:24 server1 kernel: [<c0180962>] __pollwait+0x2d/0x94 Feb 4 00:08:24 server1 kernel: [<c02dd665>] tcp_poll+0x31/0x144 Feb 4 00:08:24 server1 kernel: [<c0188fc8>] destroy_inode+0x3d/0x4c Feb 4 00:08:24 server1 kernel: [<c018b1e2>] iput+0x5f/0x61 Feb 4 00:08:24 server1 kernel: [<c02b40f4>] sock_poll+0x12/0x14 Feb 4 00:08:24 server1 kernel: [<c0180cca>] do_select+0x25c/0x378 Feb 4 00:08:24 server1 kernel: [<c0180935>] __pollwait+0x0/0x94 Feb 4 00:08:24 server1 kernel: [<c01810d9>] sys_select+0x2e0/0x43a Feb 4 00:08:24 server1 kernel: [<c02b520c>] sys_socketcall+0x10c/0x1fb Feb 4 00:08:24 server1 kernel: [<c0318eeb>] syscall_call+0x7/0xb Feb 4 00:08:24 server1 kernel: Mem-info: Feb 4 00:08:24 server1 kernel: DMA per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 2, high 6, batch 1 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 2, batch 1 Feb 4 00:08:24 server1 kernel: Normal per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 32, high 96, batch 16 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 32, batch 16 Feb 4 00:08:24 server1 kernel: HighMem per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 10, high 30, batch 5 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 10, batch 5 Feb 4 00:08:24 server1 kernel: Feb 4 00:08:24 server1 kernel: Free pages: 66820kB (66820kB HighMem) Feb 4 00:08:24 server1 kernel: Active:29068 inactive:191437 dirty:121 writeback:0 unstable:0 free:16705 slab:11020 mapped:9351 pagetables:253 Feb 4 00:08:24 server1 kernel: DMA free:0kB min:16kB low:32kB high:48kB active:36kB inactive:12168kB present:16384kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: Normal free:0kB min:936kB low:1872kB high:2808kB active:94776kB inactive:745908kB present:901120kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: HighMem free:66820kB min:128kB low:256kB high:384kB active:21460kB inactive:7672kB present:98240kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB Feb 4 00:08:24 server1 kernel: Normal: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB Feb 4 00:08:24 server1 kernel: HighMem: 1613*4kB 1804*8kB 1105*16kB 483*32kB 168*64kB 16*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 66820kB Feb 4 00:08:24 server1 kernel: Swap cache: add 40, delete 40, find 0/0, race 0+0 Feb 4 00:08:24 server1 kernel: 0 bounce buffer pages Feb 4 00:08:24 server1 kernel: Free swap: 1052088kB Feb 4 00:08:24 server1 kernel: 253936 pages of RAM Feb 4 00:08:24 server1 kernel: 24560 pages of HIGHMEM Feb 4 00:08:24 server1 kernel: 3209 reserved pages Feb 4 00:08:24 server1 kernel: 199786 pages shared Feb 4 00:08:24 server1 kernel: 0 pages swap cached Feb 4 00:08:24 server1 kernel: kswapd0: page allocation failure. order:0, mode:0x50 Feb 4 00:08:24 server1 kernel: [<c014f863>] __alloc_pages+0x28b/0x29d Feb 4 00:08:24 server1 kernel: [<c014b70f>] find_or_create_page+0x39/0x72 Feb 4 00:08:24 server1 kernel: [<c016eda7>] grow_dev_page+0x2a/0x1eb Feb 4 00:08:24 server1 kernel: [<c016f096>] __getblk_slow+0x12e/0x159 Feb 4 00:08:24 server1 kernel: [<c016f40e>] __getblk+0x3f/0x49 Feb 4 00:08:24 server1 kernel: [<c016f455>] __bread+0x9/0x22 Feb 4 00:08:24 server1 kernel: [<f88af239>] ext3_free_branches+0x91/0x1be [ext3] Feb 4 00:08:24 server1 kernel: [<f88af27e>] ext3_free_branches+0xd6/0x1be [ext3] Feb 4 00:08:24 server1 kernel: [<f88af702>] ext3_truncate+0x39c/0x460 [ext3] Feb 4 00:08:24 server1 kernel: [<c0317627>] __cond_resched+0x14/0x3b Feb 4 00:08:24 server1 kernel: [<f8862737>] journal_start+0x78/0x9e [jbd] Feb 4 00:08:24 server1 kernel: [<f88ad4b5>] ext3_delete_inode+0x60/0xaa [ext3] Feb 4 00:08:24 server1 kernel: [<f88ad455>] ext3_delete_inode+0x0/0xaa [ext3] Feb 4 00:08:24 server1 kernel: [<c018adbb>] generic_delete_inode+0xef/0x229 Feb 4 00:08:24 server1 kernel: [<c018b1e2>] iput+0x5f/0x61 Feb 4 00:08:24 server1 kernel: [<c018633c>] prune_dcache+0x50d/0x70c Feb 4 00:08:24 server1 kernel: [<c0186df1>] shrink_dcache_memory+0x16/0x2d Feb 4 00:08:24 server1 kernel: [<c015628b>] shrink_slab+0xf7/0x14c Feb 4 00:08:24 server1 kernel: [<c01578cb>] balance_pgdat+0x1b3/0x2cb Feb 4 00:08:24 server1 kernel: [<c0157a9c>] kswapd+0xb9/0xbb Feb 4 00:08:24 server1 kernel: [<c0121853>] autoremove_wake_function+0x0/0x2d Feb 4 00:08:24 server1 kernel: [<c0318e06>] ret_from_fork+0x6/0x14 Feb 4 00:08:24 server1 kernel: [<c0121853>] autoremove_wake_function+0x0/0x2d Feb 4 00:08:24 server1 kernel: [<c01579e3>] kswapd+0x0/0xbb Feb 4 00:08:24 server1 kernel: [<c01041dd>] kernel_thread_helper+0x5/0xb Feb 4 00:08:24 server1 kernel: Mem-info: Feb 4 00:08:24 server1 kernel: DMA per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 2, high 6, batch 1 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 2, batch 1 Feb 4 00:08:24 server1 kernel: Normal per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 32, high 96, batch 16 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 32, batch 16 Feb 4 00:08:24 server1 kernel: HighMem per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 10, high 30, batch 5 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 10, batch 5 Feb 4 00:08:24 server1 kernel: Feb 4 00:08:24 server1 kernel: Free pages: 66820kB (66820kB HighMem) Feb 4 00:08:24 server1 kernel: Active:29068 inactive:191449 dirty:0 writeback:55 unstable:0 free:16705 slab:11020 mapped:9351 pagetables:253 Feb 4 00:08:24 server1 kernel: DMA free:0kB min:16kB low:32kB high:48kB active:36kB inactive:12188kB present:16384kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: Normal free:0kB min:936kB low:1872kB high:2808kB active:94776kB inactive:745936kB present:901120kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: HighMem free:66820kB min:128kB low:256kB high:384kB active:21460kB inactive:7672kB present:98240kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB Feb 4 00:08:24 server1 kernel: Normal: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB Feb 4 00:08:24 server1 kernel: HighMem: 1613*4kB 1804*8kB 1105*16kB 483*32kB 168*64kB 16*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 66820kB Feb 4 00:08:24 server1 kernel: Swap cache: add 40, delete 40, find 0/0, race 0+0 Feb 4 00:08:24 server1 kernel: 0 bounce buffer pages Feb 4 00:08:24 server1 kernel: Free swap: 1052088kB Feb 4 00:08:24 server1 kernel: 253936 pages of RAM Feb 4 00:08:24 server1 kernel: 24560 pages of HIGHMEM Feb 4 00:08:24 server1 kernel: 3209 reserved pages Feb 4 00:08:24 server1 kernel: 199774 pages shared Feb 4 00:08:24 server1 kernel: 0 pages swap cached Feb 4 00:08:24 server1 kernel: nrpe: page allocation failure. order:0, mode:0x50 Feb 4 00:08:24 server1 kernel: [<c014f863>] __alloc_pages+0x28b/0x29d Feb 4 00:08:24 server1 kernel: [<c014b70f>] find_or_create_page+0x39/0x72 Feb 4 00:08:24 server1 kernel: [<c016eda7>] grow_dev_page+0x2a/0x1eb Feb 4 00:08:24 server1 kernel: [<c016f096>] __getblk_slow+0x12e/0x159 Feb 4 00:08:24 server1 kernel: [<c016f40e>] __getblk+0x3f/0x49 Feb 4 00:08:24 server1 kernel: [<c016f455>] __bread+0x9/0x22 Feb 4 00:08:24 server1 kernel: [<f88af239>] ext3_free_branches+0x91/0x1be [ext3] Feb 4 00:08:24 server1 kernel: [<f88af27e>] ext3_free_branches+0xd6/0x1be [ext3] Feb 4 00:08:24 server1 kernel: [<f88af702>] ext3_truncate+0x39c/0x460 [ext3] Feb 4 00:08:24 server1 kernel: [<c0317627>] __cond_resched+0x14/0x3b Feb 4 00:08:24 server1 kernel: [<f8862737>] journal_start+0x78/0x9e [jbd] Feb 4 00:08:24 server1 kernel: [<f88ad4b5>] ext3_delete_inode+0x60/0xaa [ext3] Feb 4 00:08:24 server1 kernel: [<f88ad455>] ext3_delete_inode+0x0/0xaa [ext3] Feb 4 00:08:24 server1 kernel: [<c018adbb>] generic_delete_inode+0xef/0x229 Feb 4 00:08:24 server1 kernel: [<c018b1e2>] iput+0x5f/0x61 Feb 4 00:08:24 server1 kernel: [<c018633c>] prune_dcache+0x50d/0x70c Feb 4 00:08:24 server1 kernel: [<c0186df1>] shrink_dcache_memory+0x16/0x2d Feb 4 00:08:24 server1 kernel: [<c015628b>] shrink_slab+0xf7/0x14c Feb 4 00:08:24 server1 kernel: [<c0157648>] try_to_free_pages+0xb9/0x189 Feb 4 00:08:24 server1 kernel: [<c014f78d>] __alloc_pages+0x1b5/0x29d Feb 4 00:08:24 server1 kernel: [<c014f88d>] __get_free_pages+0x18/0x24 Feb 4 00:08:24 server1 kernel: [<c0180962>] __pollwait+0x2d/0x94 Feb 4 00:08:24 server1 kernel: [<c02dd665>] tcp_poll+0x31/0x144 Feb 4 00:08:24 server1 kernel: [<c0188fc8>] destroy_inode+0x3d/0x4c Feb 4 00:08:24 server1 kernel: [<c018b1e2>] iput+0x5f/0x61 Feb 4 00:08:24 server1 kernel: [<c02b40f4>] sock_poll+0x12/0x14 Feb 4 00:08:24 server1 kernel: [<c0180cca>] do_select+0x25c/0x378 Feb 4 00:08:24 server1 kernel: [<c0180935>] __pollwait+0x0/0x94 Feb 4 00:08:24 server1 kernel: [<c01810d9>] sys_select+0x2e0/0x43a Feb 4 00:08:24 server1 kernel: [<c02b520c>] sys_socketcall+0x10c/0x1fb Feb 4 00:08:24 server1 kernel: [<c0318eeb>] syscall_call+0x7/0xb Feb 4 00:08:24 server1 kernel: Mem-info: Feb 4 00:08:24 server1 kernel: DMA per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 2, high 6, batch 1 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 2, batch 1 Feb 4 00:08:24 server1 kernel: Normal per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 32, high 96, batch 16 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 32, batch 16 Feb 4 00:08:24 server1 kernel: HighMem per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 10, high 30, batch 5 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 10, batch 5 Feb 4 00:08:24 server1 kernel: Feb 4 00:08:24 server1 kernel: Free pages: 68140kB (68140kB HighMem) Feb 4 00:08:24 server1 kernel: Active:29706 inactive:190479 dirty:144 writeback:0 unstable:0 free:17035 slab:11015 mapped:9351 pagetables:253 Feb 4 00:08:24 server1 kernel: DMA free:0kB min:16kB low:32kB high:48kB active:40kB inactive:12172kB present:16384kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: Normal free:0kB min:936kB low:1872kB high:2808kB active:97324kB inactive:743392kB present:901120kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: HighMem free:68140kB min:128kB low:256kB high:384kB active:21460kB inactive:6352kB present:98240kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB Feb 4 00:08:24 server1 kernel: Normal: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB Feb 4 00:08:24 server1 kernel: HighMem: 1657*4kB 1851*8kB 1113*16kB 489*32kB 171*64kB 18*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 68140kB Feb 4 00:08:24 server1 kernel: Swap cache: add 40, delete 40, find 0/0, race 0+0 Feb 4 00:08:24 server1 kernel: 0 bounce buffer pages Feb 4 00:08:24 server1 kernel: Free swap: 1052088kB Feb 4 00:08:24 server1 kernel: 253936 pages of RAM Feb 4 00:08:24 server1 kernel: 24560 pages of HIGHMEM Feb 4 00:08:24 server1 kernel: 3209 reserved pages Feb 4 00:08:24 server1 kernel: 200567 pages shared Feb 4 00:08:24 server1 kernel: 0 pages swap cached Feb 4 00:08:24 server1 kernel: syslogd: page allocation failure. order:0, mode:0x50 Feb 4 00:08:24 server1 kernel: [<c014f863>] __alloc_pages+0x28b/0x29d Feb 4 00:08:24 server1 kernel: [<c014b70f>] find_or_create_page+0x39/0x72 Feb 4 00:08:24 server1 kernel: [<c016eda7>] grow_dev_page+0x2a/0x1eb Feb 4 00:08:24 server1 kernel: [<c016f096>] __getblk_slow+0x12e/0x159 Feb 4 00:08:24 server1 kernel: [<c016f40e>] __getblk+0x3f/0x49 Feb 4 00:08:24 server1 kernel: [<c016f455>] __bread+0x9/0x22 Feb 4 00:08:24 server1 kernel: [<f88af239>] ext3_free_branches+0x91/0x1be [ext3] Feb 4 00:08:24 server1 kernel: [<f88af27e>] ext3_free_branches+0xd6/0x1be [ext3] Feb 4 00:08:24 server1 kernel: [<f88af702>] ext3_truncate+0x39c/0x460 [ext3] Feb 4 00:08:24 server1 kernel: [<c0317627>] __cond_resched+0x14/0x3b Feb 4 00:08:24 server1 kernel: [<f8862737>] journal_start+0x78/0x9e [jbd] Feb 4 00:08:24 server1 kernel: [<f88ad4b5>] ext3_delete_inode+0x60/0xaa [ext3] Feb 4 00:08:24 server1 kernel: [<f88ad455>] ext3_delete_inode+0x0/0xaa [ext3] Feb 4 00:08:24 server1 kernel: [<c018adbb>] generic_delete_inode+0xef/0x229 Feb 4 00:08:24 server1 kernel: [<c018b1e2>] iput+0x5f/0x61 Feb 4 00:08:24 server1 kernel: [<c018633c>] prune_dcache+0x50d/0x70c Feb 4 00:08:24 server1 kernel: [<c0186df1>] shrink_dcache_memory+0x16/0x2d Feb 4 00:08:24 server1 kernel: [<c015628b>] shrink_slab+0xf7/0x14c Feb 4 00:08:24 server1 kernel: [<c0157648>] try_to_free_pages+0xb9/0x189 Feb 4 00:08:24 server1 kernel: [<c014f78d>] __alloc_pages+0x1b5/0x29d Feb 4 00:08:24 server1 kernel: [<c014f88d>] __get_free_pages+0x18/0x24 Feb 4 00:08:24 server1 kernel: [<c0180962>] __pollwait+0x2d/0x94 Feb 4 00:08:24 server1 kernel: [<c02ba096>] datagram_poll+0x25/0xcf Feb 4 00:08:24 server1 kernel: [<c02b40f4>] sock_poll+0x12/0x14 Feb 4 00:08:24 server1 kernel: [<c0180cca>] do_select+0x25c/0x378 Feb 4 00:08:24 server1 kernel: [<c0180935>] __pollwait+0x0/0x94 Feb 4 00:08:24 server1 kernel: [<c01810d9>] sys_select+0x2e0/0x43a Feb 4 00:08:24 server1 kernel: [<c016db6d>] sys_fsync+0xce/0xd6 Feb 4 00:08:24 server1 kernel: [<c0318eeb>] syscall_call+0x7/0xb Feb 4 00:08:24 server1 kernel: Mem-info: Feb 4 00:08:24 server1 kernel: DMA per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 2, high 6, batch 1 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 2, batch 1 Feb 4 00:08:24 server1 kernel: Normal per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 32, high 96, batch 16 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 32, batch 16 Feb 4 00:08:24 server1 kernel: HighMem per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 10, high 30, batch 5 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 10, batch 5 Feb 4 00:08:24 server1 kernel: Feb 4 00:08:24 server1 kernel: Free pages: 68140kB (68140kB HighMem) Feb 4 00:08:24 server1 kernel: Active:29706 inactive:190479 dirty:144 writeback:0 unstable:0 free:17035 slab:11015 mapped:9351 pagetables:253 Feb 4 00:08:24 server1 kernel: DMA free:0kB min:16kB low:32kB high:48kB active:40kB inactive:12172kB present:16384kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: Normal free:0kB min:936kB low:1872kB high:2808kB active:97324kB inactive:743392kB present:901120kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: HighMem free:68140kB min:128kB low:256kB high:384kB active:21460kB inactive:6352kB present:98240kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB Feb 4 00:08:24 server1 kernel: Normal: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB Feb 4 00:08:24 server1 kernel: HighMem: 1657*4kB 1851*8kB 1113*16kB 489*32kB 171*64kB 18*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 68140kB Feb 4 00:08:24 server1 kernel: Swap cache: add 40, delete 40, find 0/0, race 0+0 Feb 4 00:08:24 server1 kernel: 0 bounce buffer pages Feb 4 00:08:24 server1 kernel: Free swap: 1052088kB Feb 4 00:08:24 server1 kernel: 253936 pages of RAM Feb 4 00:08:24 server1 kernel: 24560 pages of HIGHMEM Feb 4 00:08:24 server1 kernel: 3209 reserved pages Feb 4 00:08:24 server1 kernel: 200567 pages shared Feb 4 00:08:24 server1 kernel: 0 pages swap cached Feb 4 00:08:24 server1 kernel: kswapd0: page allocation failure. order:0, mode:0x850 Feb 4 00:08:24 server1 kernel: [<c014f863>] __alloc_pages+0x28b/0x29d Feb 4 00:08:24 server1 kernel: [<c014f88d>] __get_free_pages+0x18/0x24 Feb 4 00:08:24 server1 kernel: [<c0152e22>] kmem_getpages+0x15/0x94 Feb 4 00:08:24 server1 kernel: [<c0153adf>] cache_grow+0x107/0x233 Feb 4 00:08:24 server1 kernel: [<c0153e02>] cache_alloc_refill+0x1f7/0x227 Feb 4 00:08:24 server1 kernel: [<c0154337>] __kmalloc+0x6b/0x7d Feb 4 00:08:24 server1 kernel: [<f886c54c>] __jbd_kmalloc+0x16/0x17 [jbd] Feb 4 00:08:24 server1 kernel: [<f8863833>] journal_get_undo_access+0x58/0x122 [jbd] Feb 4 00:08:24 server1 kernel: [<f88aa437>] ext3_free_blocks_sb+0x17e/0x4d5 [ext3] Feb 4 00:08:24 server1 kernel: [<f88aa7bb>] ext3_free_blocks+0x2d/0x84 [ext3] Feb 4 00:08:24 server1 kernel: [<f88af141>] ext3_free_data+0x90/0xf7 [ext3] Feb 4 00:08:24 server1 kernel: [<f88af27e>] ext3_free_branches+0xd6/0x1be [ext3] Feb 4 00:08:24 server1 kernel: [<f88af27e>] ext3_free_branches+0xd6/0x1be [ext3] Feb 4 00:08:24 server1 kernel: [<f88af702>] ext3_truncate+0x39c/0x460 [ext3] Feb 4 00:08:24 server1 kernel: [<c0317627>] __cond_resched+0x14/0x3b Feb 4 00:08:24 server1 kernel: [<f8862737>] journal_start+0x78/0x9e [jbd] Feb 4 00:08:24 server1 kernel: [<f88ad4b5>] ext3_delete_inode+0x60/0xaa [ext3] Feb 4 00:08:24 server1 kernel: [<f88ad455>] ext3_delete_inode+0x0/0xaa [ext3] Feb 4 00:08:24 server1 kernel: [<c018adbb>] generic_delete_inode+0xef/0x229 Feb 4 00:08:24 server1 kernel: [<c018b1e2>] iput+0x5f/0x61 Feb 4 00:08:24 server1 kernel: [<c018633c>] prune_dcache+0x50d/0x70c Feb 4 00:08:24 server1 kernel: [<c0186df1>] shrink_dcache_memory+0x16/0x2d Feb 4 00:08:24 server1 kernel: [<c015628b>] shrink_slab+0xf7/0x14c Feb 4 00:08:24 server1 kernel: [<c01578cb>] balance_pgdat+0x1b3/0x2cb Feb 4 00:08:24 server1 kernel: [<c0157a9c>] kswapd+0xb9/0xbb Feb 4 00:08:24 server1 kernel: [<c0121853>] autoremove_wake_function+0x0/0x2d Feb 4 00:08:24 server1 kernel: [<c0318e06>] ret_from_fork+0x6/0x14 Feb 4 00:08:24 server1 kernel: [<c0121853>] autoremove_wake_function+0x0/0x2d Feb 4 00:08:24 server1 kernel: [<c01579e3>] kswapd+0x0/0xbb Feb 4 00:08:24 server1 kernel: [<c01041dd>] kernel_thread_helper+0x5/0xb Feb 4 00:08:24 server1 kernel: Mem-info: Feb 4 00:08:24 server1 kernel: DMA per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 2, high 6, batch 1 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 2, batch 1 Feb 4 00:08:24 server1 kernel: Normal per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 32, high 96, batch 16 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 32, batch 16 Feb 4 00:08:24 server1 kernel: HighMem per-cpu: Feb 4 00:08:24 server1 kernel: cpu 0 hot: low 10, high 30, batch 5 Feb 4 00:08:24 server1 kernel: cpu 0 cold: low 0, high 10, batch 5 Feb 4 00:08:24 server1 kernel: Feb 4 00:08:24 server1 kernel: Free pages: 68140kB (68140kB HighMem) Feb 4 00:08:24 server1 kernel: Active:29706 inactive:190479 dirty:144 writeback:0 unstable:0 free:17035 slab:11015 mapped:9351 pagetables:253 Feb 4 00:08:24 server1 kernel: DMA free:0kB min:16kB low:32kB high:48kB active:40kB inactive:12172kB present:16384kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: Normal free:0kB min:936kB low:1872kB high:2808kB active:97324kB inactive:743392kB present:901120kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: HighMem free:68140kB min:128kB low:256kB high:384kB active:21460kB inactive:6352kB present:98240kB pages_scanned:0 all_unreclaimable? no Feb 4 00:08:24 server1 kernel: protections[]: 0 0 0 Feb 4 00:08:24 server1 kernel: DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB Feb 4 00:08:24 server1 kernel: Normal: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB Feb 4 00:08:24 server1 kernel: HighMem: 1657*4kB 1851*8kB 1113*16kB 489*32kB 171*64kB 18*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 68140kB Feb 4 00:08:24 server1 kernel: Swap cache: add 40, delete 40, find 0/0, race 0+0 Feb 4 00:08:24 server1 kernel: 0 bounce buffer pages Feb 4 00:08:24 server1 kernel: Free swap: 1052088kB Feb 4 00:08:24 server1 kernel: 253936 pages of RAM Feb 4 00:08:24 server1 kernel: 24560 pages of HIGHMEM Feb 4 00:08:24 server1 kernel: 3209 reserved pages Feb 4 00:08:24 server1 kernel: 200567 pages shared Feb 4 00:08:24 server1 kernel: 0 pages swap cached Feb 4 00:08:24 server1 kernel: journal_get_undo_access: No memory for committed data Feb 4 00:08:24 server1 kernel: ext3_free_blocks_sb: aborting transaction: Out of memory in __ext3_journal_get_undo_access Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_free_blocks_sb: Out of memory Feb 4 00:08:24 server1 kernel: Aborting journal on device dm-0. Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_reserve_inode_write: Journal has aborted Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_truncate: Journal has aborted Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_free_blocks_sb: Journal has aborted Feb 4 00:08:24 server1 kernel: ext3_abort called. Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0): ext3_journal_start_sb: Detected aborted journal Feb 4 00:08:24 server1 kernel: Remounting filesystem read-only Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in start_transaction: Journal has aborted Feb 4 00:08:24 server1 last message repeated 8 times Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_free_blocks_sb: Journal has aborted Feb 4 00:08:24 server1 last message repeated 54 times Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_reserve_inode_write: Journal has aborted Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_truncate: Out of memory Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_reserve_inode_write: Journal has aborted Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_orphan_del: Journal has aborted Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_reserve_inode_write: Journal has aborted Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_delete_inode: Out of memory Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_reserve_inode_write: Journal has aborted Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_orphan_del: Journal has aborted Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_reserve_inode_write: Journal has aborted Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_delete_inode: Journal has aborted Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_reserve_inode_write: Journal has aborted Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_truncate: Journal has aborted Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_reserve_inode_write: Journal has aborted Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_orphan_del: Journal has aborted Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_reserve_inode_write: Journal has aborted Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_delete_inode: Journal has aborted Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_reserve_inode_write: Journal has aborted Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_truncate: Journal has aborted Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_reserve_inode_write: Journal has aborted Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_orphan_del: Journal has aborted Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_reserve_inode_write: Journal has aborted Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_delete_inode: Journal has aborted Feb 4 00:08:24 server1 kernel: __journal_remove_journal_head: freeing b_committed_data Feb 4 00:08:24 server1 last message repeated 65 times Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_free_blocks_sb: Journal has aborted Feb 4 00:08:24 server1 last message repeated 18 times Feb 4 00:08:24 server1 kernel: EXT3-fs error (device dm-0) in ext3_reserve_inode_write: Journal has aborted Feb 4 00:08:24 server1 kernel: Assertion failure in journal_stop() at fs/jbd/transaction.c:1344: "transaction->t_updates > 0" Feb 4 00:08:24 server1 kernel: ------------[ cut here ]------------ Feb 4 00:08:24 server1 kernel: kernel BUG at fs/jbd/transaction.c:1344! Feb 4 00:08:24 server1 kernel: invalid operand: 0000 [#1] Feb 4 00:08:24 server1 kernel: Modules linked in: nfsd exportfs lockd nfs_acl parport_pc lp parport autofs4 i2c_dev i2c_core sunrpc dm_mirror button battery ac md5 ipv6 ohci_hcd ehci_hcd snd_intel8x0 snd_ac97_codec snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd_page_alloc snd_mpu401_uart snd_rawmidi snd_seq_device snd soundcore 3c59x mii e1000 floppy ext3 jbd dm_mod Feb 4 00:08:24 server1 kernel: CPU: 0 Feb 4 00:08:24 server1 kernel: EIP: 0060:[<f88642b1>] Not tainted VLI Feb 4 00:08:24 server1 kernel: EFLAGS: 00010246 (2.6.9-42.0.8.EL) Feb 4 00:08:24 server1 kernel: EIP is at journal_stop+0x30/0x42f [jbd] Feb 4 00:08:24 server1 kernel: eax: 00000062 ebx: ffffffe2 ecx: f886ce4a edx: f6156b30 Feb 4 00:08:24 server1 kernel: esi: f7c464c0 edi: c18a7200 ebp: da8082b0 esp: f6156b2c Feb 4 00:08:24 server1 kernel: ds: 007b es: 007b ss: 0068 Feb 4 00:08:24 server1 kernel: Process nfsd (pid: 2947, threadinfo=f6156000 task=c1a0cc50) Feb 4 00:08:24 server1 kernel: Stack: f886ce4a f886ca96 f886ce0e 00000540 f886cf5c c18a7200 c18a7200 ffffffe2 Feb 4 00:08:24 server1 kernel: c1ab8400 f88bb06c 00000000 f88b3d06 f6d67438 00000000 f6d67538 f88af7be Feb 4 00:08:24 server1 kernel: 0000000b 00000000 f6d67618 00000400 f6d67438 da8082b0 f6d67538 07e98504 Feb 4 00:08:24 server1 kernel: Call Trace: Feb 4 00:08:24 server1 kernel: [<f88b3d06>] __ext3_journal_stop+0x19/0x34 [ext3] Feb 4 00:08:24 server1 kernel: [<f88af7be>] ext3_truncate+0x458/0x460 [ext3] Feb 4 00:08:24 server1 kernel: [<c0317627>] __cond_resched+0x14/0x3b Feb 4 00:08:24 server1 kernel: [<f8862737>] journal_start+0x78/0x9e [jbd] Feb 4 00:08:24 server1 kernel: [<f88ad4b5>] ext3_delete_inode+0x60/0xaa [ext3] Feb 4 00:08:24 server1 kernel: [<f88ad455>] ext3_delete_inode+0x0/0xaa [ext3] Feb 4 00:08:24 server1 kernel: [<c018adbb>] generic_delete_inode+0xef/0x229 Feb 4 00:08:24 server1 kernel: [<c018b1e2>] iput+0x5f/0x61 Feb 4 00:08:24 server1 kernel: [<c018633c>] prune_dcache+0x50d/0x70c Feb 4 00:08:24 server1 kernel: [<c0186df1>] shrink_dcache_memory+0x16/0x2d Feb 4 00:08:24 server1 kernel: [<c015628b>] shrink_slab+0xf7/0x14c Feb 4 00:08:24 server1 kernel: [<c0157648>] try_to_free_pages+0xb9/0x189 Feb 4 00:08:24 server1 kernel: [<c014f78d>] __alloc_pages+0x1b5/0x29d Feb 4 00:08:24 server1 kernel: [<c014ce77>] generic_file_buffered_write+0x172/0x4db Feb 4 00:08:24 server1 kernel: [<c01299d0>] current_fs_time+0x44/0x4c Feb 4 00:08:24 server1 kernel: [<c014d51d>] __generic_file_aio_write_nolock+0x33d/0x36b Feb 4 00:08:24 server1 kernel: [<c014d64e>] __generic_file_write_nolock+0x84/0x99 Feb 4 00:08:24 server1 kernel: [<c011f422>] recalc_task_prio+0x128/0x133 Feb 4 00:08:24 server1 kernel: [<c0121853>] autoremove_wake_function+0x0/0x2d Feb 4 00:08:24 server1 kernel: [<c014d971>] generic_file_writev+0x54/0xb3 Feb 4 00:08:24 server1 kernel: [<c014d91d>] generic_file_writev+0x0/0xb3 Feb 4 00:08:24 server1 kernel: [<c016c605>] do_readv_writev+0x1c5/0x245 Feb 4 00:08:24 server1 kernel: [<c016c0f2>] do_sync_write+0x0/0xc9 Feb 4 00:08:24 server1 kernel: [<c0317627>] __cond_resched+0x14/0x3b Feb 4 00:08:24 server1 kernel: [<c016b335>] __dentry_open+0xca/0x16a Feb 4 00:08:24 server1 kernel: [<c016b266>] dentry_open+0x48/0x4d Feb 4 00:08:24 server1 kernel: [<c016c703>] vfs_writev+0x3e/0x43 Feb 4 00:08:24 server1 kernel: [<f8b39600>] nfsd_write+0xeb/0x28f [nfsd] Feb 4 00:08:24 server1 kernel: [<c0317627>] __cond_resched+0x14/0x3b Feb 4 00:08:24 server1 kernel: [<f8b40ee2>] nfsd3_proc_write+0xbf/0xd5 [nfsd] Feb 4 00:08:24 server1 kernel: [<f8b42f90>] nfs3svc_decode_writeargs+0x0/0x243 [nfsd] Feb 4 00:08:24 server1 kernel: [<f8b35947>] nfsd_dispatch+0xba/0x16f [nfsd] Feb 4 00:08:24 server1 kernel: [<f8ace74c>] svc_process+0x432/0x6e2 [sunrpc] Feb 4 00:08:24 server1 kernel: [<f8b355eb>] nfsd+0x2a7/0x549 [nfsd] Feb 4 00:08:24 server1 kernel: [<f8b35344>] nfsd+0x0/0x549 [nfsd] Feb 4 00:08:24 server1 kernel: [<c01041dd>] kernel_thread_helper+0x5/0xb Feb 4 00:08:24 server1 kernel: Code: 53 52 52 8b 30 8b 3e 83 7e 50 00 7f 29 68 5c cf 86 f8 68 40 05 00 00 68 0e ce 86 f8 68 96 ca 86 f8 68 4a ce 86 f8 e8 50 06 8c c7 <0f> 0b 40 05 0e ce 86 f8 83 c4 14 b8 00 f0 ff ff 21 e0 8b 00 39 Feb 4 00:08:24 server1 kernel: <0>Fatal exception: panic in 5 seconds
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Mon, Feb 05, 2007 at 09:33:50AM -0600, Jon Christensen wrote:
The system stopped responding to ssh but still responded to ping. The logs showed this error: (its long)
I might be wrong, but this sure smells like RAM gone bad.
- -- Rodrigo Barbosa "Quid quid Latine dictum sit, altum viditur" "Be excellent to each other ..." - Bill & Ted (Wyld Stallyns)
Try disabling cpuspeed; it has caused numerous kernel panics on all sorts of hardware.
-Drew
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Rodrigo Barbosa Sent: Monday, February 05, 2007 11:06 AM To: CentOS mailing list Subject: Re: [CentOS] kernel error -- system crash
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Mon, Feb 05, 2007 at 09:33:50AM -0600, Jon Christensen wrote:
The system stopped responding to ssh but still responded to ping. The
logs showed this error: (its long)
I might be wrong, but this sure smells like RAM gone bad.
- -- Rodrigo Barbosa "Quid quid Latine dictum sit, altum viditur" "Be excellent to each other ..." - Bill & Ted (Wyld Stallyns)
_______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
[root@server1 ~]# /etc/init.d/cpuspeed status cpuspeed is stopped
Are you referring to something else?
On 2/5/07, Drew Weaver drew.weaver@thenap.com wrote:
Try disabling cpuspeed; it has caused numerous kernel panics on all sorts of hardware.
-Drew
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Rodrigo Barbosa Sent: Monday, February 05, 2007 11:06 AM To: CentOS mailing list Subject: Re: [CentOS] kernel error -- system crash
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Mon, Feb 05, 2007 at 09:33:50AM -0600, Jon Christensen wrote:
The system stopped responding to ssh but still responded to ping. The
logs showed this error: (its long)
I might be wrong, but this sure smells like RAM gone bad.
Rodrigo Barbosa "Quid quid Latine dictum sit, altum viditur" "Be excellent to each other ..." - Bill & Ted (Wyld Stallyns)
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux)
iD8DBQFFx1YApdyWzQ5b5ckRAgonAJoC70CmfVCioJ/eaWFLRhjqNjrKTACgsihz T2q3s9JcBYVbaldkKnvmJho= =XtOy -----END PGP SIGNATURE----- _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
-
Drew Weaver -
Jon Christensen -
Rodrigo Barbosa