J.J.Garcia wrote: >Hi all, > >Just watching for my first time the 'oom-killer' call in a CentOS 4.2 >workstation. System was running normally but with high load of nice process, >iow, rosetta and seti, others appart, nautilus, etc... i realized that after a >night of standby running normally, the next moorning after few manual operations >from X-session in local console, it started a massive swapping, this time not >recovered as previously other time, finishing in a oom-killer call. > >Finally inspected the syslog and realize the oom-killer was called to leave the >system stable having killed firefox-bin, 3 x rosetta, xmms and one nautilus. > >It's a PIII 800 Mhz, 40GB IDE, 512 Mb RAM, 1024 Swap space, up for 4 days. > >I'm not very xperienced in tunning the memmory system nor the swap system, help >appreciated to avoid this memory 'trashing' situtation (14 virtual desktops >openned at the same time). > >Is it a normal situation? Is it a correct protecction measure? It's due a low >bandwitch IDE HD? how can i tune it to avoid this situations? is it a high load >for this workstation? I think the oom-killer was called correctly to avoid the >system collapsed, but i think there was a problem in the swapping subsystem or a >memmory trashing situation to avoid. > >Help and hints appreciated, > >Attached you'll the oom-killer log, > >TIA > >J.J. > > > > > > > > > > > > > >------------------------------------------------------------------------ > >Dec 14 11:48:00 goliathbox kernel: oom-killer: gfp_mask=0xd0 >Dec 14 11:48:00 goliathbox kernel: Mem-info: >Dec 14 11:48:00 goliathbox kernel: DMA per-cpu: >Dec 14 11:48:00 goliathbox kernel: cpu 0 hot: low 2, high 6, batch 1 >Dec 14 11:48:00 goliathbox kernel: cpu 0 cold: low 0, high 2, batch 1 >Dec 14 11:48:00 goliathbox kernel: Normal per-cpu: >Dec 14 11:48:00 goliathbox kernel: cpu 0 hot: low 32, high 96, batch 16 >Dec 14 11:48:00 goliathbox kernel: cpu 0 cold: low 0, high 32, batch 16 >Dec 14 11:48:00 goliathbox kernel: HighMem per-cpu: empty >Dec 14 11:48:00 goliathbox kernel: >Dec 14 11:48:00 goliathbox kernel: Free pages: 676kB (0kB HighMem) >Dec 14 11:48:00 goliathbox kernel: Active:109642 inactive:10334 dirty:0 writeback:0 unstable:0 free:169 slab:3746 mapped:118866 pagetables:1729 >Dec 14 11:48:00 goliathbox kernel: DMA free:20kB min:20kB low:40kB high:60kB active:6132kB inactive:6316kB present:16384kB pages_scanned:20658 all_unreclaimable? yes >Dec 14 11:48:02 goliathbox kernel: protections[]: 0 0 0 >Dec 14 11:48:23 goliathbox kernel: Normal free:656kB min:700kB low:1400kB high:2100kB active:432436kB inactive:35020kB present:507820kB pages_scanned:1102915 all_unreclaimable? yesDec 14 11:48:23 goliathbox kernel: protections[]: 0 0 0 >Dec 14 11:48:23 goliathbox kernel: HighMem free:0kB min:128kB low:256kB high:384kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no >Dec 14 11:48:23 goliathbox kernel: protections[]: 0 0 0 >Dec 14 11:48:23 goliathbox kernel: DMA: 1*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 20kB >Dec 14 11:48:23 goliathbox kernel: Normal: 0*4kB 6*8kB 2*16kB 4*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 656kB >Dec 14 11:48:23 goliathbox kernel: HighMem: empty >Dec 14 11:48:23 goliathbox kernel: Swap cache: add 974772, delete 974259, find 292228/382858, race 0+15 >Dec 14 11:48:23 goliathbox kernel: 0 bounce buffer pages >Dec 14 11:48:23 goliathbox kernel: Free swap: 0kB >Dec 14 11:48:23 goliathbox kernel: 131051 pages of RAM >Dec 14 11:48:23 goliathbox kernel: 0 pages of HIGHMEM >Dec 14 11:48:42 goliathbox kernel: 2433 reserved pages >Dec 14 11:49:26 goliathbox kernel: 26433 pages shared >Dec 14 11:49:26 goliathbox kernel: 513 pages swap cached >Dec 14 11:49:26 goliathbox kernel: Out of Memory: Killed process 13954 (firefox-bin). >Dec 14 11:49:26 goliathbox kernel: oom-killer: gfp_mask=0x1d2 >Dec 14 11:49:26 goliathbox kernel: Mem-info: >Dec 14 11:49:26 goliathbox kernel: DMA per-cpu: >Dec 14 11:49:26 goliathbox kernel: cpu 0 hot: low 2, high 6, batch 1 >Dec 14 11:49:26 goliathbox kernel: cpu 0 cold: low 0, high 2, batch 1 >Dec 14 11:49:26 goliathbox kernel: Normal per-cpu: >Dec 14 11:49:26 goliathbox kernel: cpu 0 hot: low 32, high 96, batch 16 >Dec 14 11:49:26 goliathbox kernel: cpu 0 cold: low 0, high 32, batch 16 >Dec 14 11:49:26 goliathbox kernel: HighMem per-cpu: empty >Dec 14 11:49:26 goliathbox kernel: >Dec 14 11:49:27 goliathbox kernel: Free pages: 676kB (0kB HighMem) >Dec 14 11:49:27 goliathbox kernel: Active:1868 inactive:118396 dirty:0 writeback:0 unstable:0 free:169 slab:3738 mapped:118869 pagetables:1729 >Dec 14 11:49:27 goliathbox kernel: DMA free:20kB min:20kB low:40kB high:60kB active:4232kB inactive:7992kB present:16384kB pages_scanned:20064 all_unreclaimable? yes >Dec 14 11:49:28 goliathbox kernel: protections[]: 0 0 0 >Dec 14 11:49:28 goliathbox kernel: Normal free:656kB min:700kB low:1400kB high:2100kB active:3240kB inactive:465592kB present:507820kB pages_scanned:780089 all_unreclaimable? yes >Dec 14 11:49:28 goliathbox kernel: protections[]: 0 0 0 >Dec 14 11:49:29 goliathbox kernel: HighMem free:0kB min:128kB low:256kB high:384kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no >Dec 14 11:49:29 goliathbox kernel: protections[]: 0 0 0 >Dec 14 11:49:29 goliathbox kernel: DMA: 1*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 20kB >Dec 14 11:49:29 goliathbox kernel: Normal: 0*4kB 6*8kB 2*16kB 4*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 656kB >Dec 14 11:49:29 goliathbox kernel: HighMem: empty >Dec 14 11:49:29 goliathbox kernel: Swap cache: add 974851, delete 974338, find 292235/382874, race 0+15 >Dec 14 11:49:30 goliathbox kernel: 0 bounce buffer pages >Dec 14 11:49:30 goliathbox kernel: Free swap: 0kB >Dec 14 11:49:30 goliathbox kernel: 131051 pages of RAM >Dec 14 11:49:30 goliathbox kernel: 0 pages of HIGHMEM >Dec 14 11:49:31 goliathbox kernel: 2433 reserved pages >Dec 14 11:49:31 goliathbox kernel: 26140 pages shared >Dec 14 11:49:31 goliathbox kernel: 513 pages swap cached >Dec 14 11:49:31 goliathbox kernel: Out of Memory: Killed process 15771 (firefox-bin). >Dec 14 11:49:31 goliathbox kernel: oom-killer: gfp_mask=0xd0 >Dec 14 11:49:31 goliathbox kernel: Mem-info: >Dec 14 11:49:32 goliathbox kernel: DMA per-cpu: >Dec 14 11:49:32 goliathbox kernel: cpu 0 hot: low 2, high 6, batch 1 >Dec 14 11:49:32 goliathbox kernel: cpu 0 cold: low 0, high 2, batch 1 >Dec 14 11:49:32 goliathbox kernel: Normal per-cpu: >Dec 14 11:49:32 goliathbox kernel: cpu 0 hot: low 32, high 96, batch 16 >Dec 14 11:49:32 goliathbox kernel: cpu 0 cold: low 0, high 32, batch 16 >Dec 14 11:49:32 goliathbox kernel: HighMem per-cpu: empty >Dec 14 11:49:33 goliathbox kernel: >Dec 14 11:49:33 goliathbox kernel: Free pages: 676kB (0kB HighMem) >Dec 14 11:49:33 goliathbox kernel: Active:73675 inactive:46271 dirty:0 writeback:0 unstable:0 free:169 slab:3735 mapped:118871 pagetables:1729 >Dec 14 11:49:33 goliathbox kernel: DMA free:20kB min:20kB low:40kB high:60kB active:4384kB inactive:7968kB present:16384kB pages_scanned:20229 all_unreclaimable? yes >Dec 14 11:49:33 goliathbox kernel: protections[]: 0 0 0 >Dec 14 11:49:33 goliathbox kernel: Normal free:656kB min:700kB low:1400kB high:2100kB active:290316kB inactive:177116kB present:507820kB pages_scanned:556644 all_unreclaimable? yesDec 14 11:49:34 goliathbox kernel: protections[]: 0 0 0 >Dec 14 11:49:34 goliathbox kernel: HighMem free:0kB min:128kB low:256kB high:384kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no >Dec 14 11:49:34 goliathbox kernel: protections[]: 0 0 0 >Dec 14 11:49:34 goliathbox kernel: DMA: 1*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 20kB >Dec 14 11:49:34 goliathbox kernel: Normal: 0*4kB 6*8kB 2*16kB 4*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 656kB >Dec 14 11:49:34 goliathbox kernel: HighMem: empty >Dec 14 11:49:35 goliathbox kernel: Swap cache: add 974851, delete 974338, find 292235/382874, race 0+15 >Dec 14 11:49:35 goliathbox kernel: 0 bounce buffer pages >Dec 14 11:49:35 goliathbox kernel: Free swap: 0kB >Dec 14 11:49:35 goliathbox kernel: 131051 pages of RAM >Dec 14 11:49:35 goliathbox kernel: 0 pages of HIGHMEM >Dec 14 11:49:35 goliathbox kernel: 2433 reserved pages >Dec 14 11:49:36 goliathbox kernel: 26474 pages shared >Dec 14 11:49:36 goliathbox kernel: 513 pages swap cached >Dec 14 11:49:36 goliathbox kernel: Out of Memory: Killed process 13973 (rosetta_4.79_i6). >Dec 14 11:49:36 goliathbox kernel: Out of Memory: Killed process 13972 (rosetta_4.79_i6). >Dec 14 11:49:36 goliathbox kernel: Out of Memory: Killed process 13974 (rosetta_4.79_i6). >Dec 14 11:49:36 goliathbox kernel: oom-killer: gfp_mask=0x1d2 >Dec 14 11:49:36 goliathbox kernel: Mem-info: >Dec 14 11:49:37 goliathbox kernel: DMA per-cpu: >Dec 14 11:49:38 goliathbox kernel: cpu 0 hot: low 2, high 6, batch 1 >Dec 14 11:49:38 goliathbox kernel: cpu 0 cold: low 0, high 2, batch 1 >Dec 14 11:49:39 goliathbox kernel: Normal per-cpu: >Dec 14 11:49:39 goliathbox kernel: cpu 0 hot: low 32, high 96, batch 16 >Dec 14 11:49:40 goliathbox kernel: cpu 0 cold: low 0, high 32, batch 16 >Dec 14 11:49:40 goliathbox kernel: HighMem per-cpu: empty >Dec 14 11:49:40 goliathbox kernel: >Dec 14 11:49:41 goliathbox kernel: Free pages: 676kB (0kB HighMem) >Dec 14 11:49:41 goliathbox kernel: Active:5356 inactive:114646 dirty:0 writeback:0 unstable:0 free:169 slab:3735 mapped:118875 pagetables:1729 >Dec 14 11:49:41 goliathbox kernel: DMA free:20kB min:20kB low:40kB high:60kB active:4412kB inactive:8044kB present:16384kB pages_scanned:20361 all_unreclaimable? yes >Dec 14 11:49:41 goliathbox crond(pam_unix)[5432]: session opened for user root by (uid=0) >Dec 14 11:49:41 goliathbox crond(pam_unix)[5434]: session opened for user root by (uid=0) >Dec 14 11:49:41 goliathbox crond(pam_unix)[5433]: session opened for user sm0ketst by (uid=0) >Dec 14 11:49:41 goliathbox kernel: protections[]: 0 0 0 >Dec 14 11:49:42 goliathbox kernel: Normal free:656kB min:700kB low:1400kB high:2100kB active:17012kB inactive:450540kB present:507820kB pages_scanned:2398502 all_unreclaimable? yesDec 14 11:49:43 goliathbox kernel: protections[]: 0 0 0 >Dec 14 11:49:43 goliathbox kernel: HighMem free:0kB min:128kB low:256kB high:384kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no >Dec 14 11:49:43 goliathbox kernel: protections[]: 0 0 0 >Dec 14 11:49:44 goliathbox kernel: DMA: 1*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 20kB >Dec 14 11:49:44 goliathbox kernel: Normal: 0*4kB 6*8kB 2*16kB 4*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 656kB >Dec 14 11:49:44 goliathbox kernel: HighMem: empty >Dec 14 11:49:44 goliathbox kernel: Swap cache: add 974851, delete 974338, find 292235/382874, race 0+15 >Dec 14 11:49:45 goliathbox kernel: 0 bounce buffer pages >Dec 14 11:49:45 goliathbox kernel: Free swap: 0kB >Dec 14 11:49:46 goliathbox kernel: 131051 pages of RAM >Dec 14 11:49:46 goliathbox kernel: 0 pages of HIGHMEM >Dec 14 11:49:46 goliathbox kernel: 2433 reserved pages >Dec 14 11:49:46 goliathbox kernel: 26419 pages shared >Dec 14 11:49:47 goliathbox kernel: 513 pages swap cached >Dec 14 11:49:47 goliathbox kernel: Out of Memory: Killed process 4936 (xmms). >Dec 14 11:49:47 goliathbox kernel: oom-killer: gfp_mask=0x1d2 >Dec 14 11:49:48 goliathbox kernel: Mem-info: >Dec 14 11:49:48 goliathbox kernel: DMA per-cpu: >Dec 14 11:49:48 goliathbox kernel: cpu 0 hot: low 2, high 6, batch 1 >Dec 14 11:49:49 goliathbox kernel: cpu 0 cold: low 0, high 2, batch 1 >Dec 14 11:49:49 goliathbox kernel: Normal per-cpu: >Dec 14 11:49:49 goliathbox kernel: cpu 0 hot: low 32, high 96, batch 16 >Dec 14 11:49:49 goliathbox kernel: cpu 0 cold: low 0, high 32, batch 16 >Dec 14 11:49:49 goliathbox kernel: HighMem per-cpu: empty >Dec 14 11:49:50 goliathbox crond(pam_unix)[5390]: session closed for user root >Dec 14 11:49:50 goliathbox kernel: >Dec 14 11:49:50 goliathbox crond(pam_unix)[5432]: session closed for user root >Dec 14 11:49:50 goliathbox kernel: Free pages: 676kB (0kB HighMem) >Dec 14 11:49:51 goliathbox kernel: Active:31292 inactive:88792 dirty:0 writeback:0 unstable:0 free:169 slab:3717 mapped:118877 pagetables:1729 >Dec 14 11:49:51 goliathbox kernel: DMA free:20kB min:20kB low:40kB high:60kB active:4412kB inactive:8044kB present:16384kB pages_scanned:20955 all_unreclaimable? yes >Dec 14 11:49:51 goliathbox kernel: protections[]: 0 0 0 >Dec 14 11:49:51 goliathbox kernel: Normal free:656kB min:700kB low:1400kB high:2100kB active:120756kB inactive:347124kB present:507820kB pages_scanned:578655 all_unreclaimable? yesDec 14 11:49:51 goliathbox kernel: protections[]: 0 0 0 >Dec 14 11:49:51 goliathbox crond(pam_unix)[5434]: session closed for user root >Dec 14 11:49:51 goliathbox kernel: HighMem free:0kB min:128kB low:256kB high:384kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no >Dec 14 11:49:52 goliathbox crond(pam_unix)[5433]: session closed for user sm0ketst >Dec 14 11:49:52 goliathbox kernel: protections[]: 0 0 0 >Dec 14 11:49:52 goliathbox kernel: DMA: 1*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 20kB >Dec 14 11:49:52 goliathbox kernel: Normal: 0*4kB 6*8kB 2*16kB 4*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 656kB >Dec 14 11:49:52 goliathbox kernel: HighMem: empty >Dec 14 11:49:52 goliathbox kernel: Swap cache: add 974851, delete 974338, find 292235/382874, race 0+15 >Dec 14 11:49:52 goliathbox kernel: 0 bounce buffer pages >Dec 14 11:49:53 goliathbox kernel: Free swap: 0kB >Dec 14 11:49:53 goliathbox kernel: 131051 pages of RAM >Dec 14 11:49:53 goliathbox kernel: 0 pages of HIGHMEM >Dec 14 11:49:53 goliathbox kernel: 2433 reserved pages >Dec 14 11:49:53 goliathbox kernel: 26379 pages shared >Dec 14 11:49:53 goliathbox kernel: 513 pages swap cached >Dec 14 11:49:53 goliathbox kernel: Out of Memory: Killed process 5129 (nautilus). > > >------------------------------------------------------------------------ > >_______________________________________________ >CentOS mailing list >CentOS at centos.org >http://lists.centos.org/mailman/listinfo/centos > > (light) rant alert: WOOP WOOP WOOP !!!!! Since the mid '90's, when I first used Linux (Slackware, from a fistful of floppies, on a 486 w/ 32 MB RAM & some tiny HDD), Linux has quietly become a memory PIG. That 1st install did have X running (FVWM IIRC) & it did run, including porting our code from SGI to Linux, which was the purpose of the work at that time (feasibility study, for NASA Langley). As I write this on my 933 MHz PIII, 1 GB RAM, 1 GB swap, SuSE 8.2, all stock, I observe from a 'free -m' in a shell: [wam at DARKSTAR, ~/FTP, 8:30:18am] 938 % free -m total used free shared buffers cached Mem: 1008 995 13 0 83 748 -/+ buffers/cache: 163 845 Swap: 1023 27 996 or 163 (!!!!!) MB being used, & for what ? 4 KDE destops, a total of 6 RXVT (supposedly light on RAM use) shells open, Mozilla as I write this, NFS & Samba daemons, & *THAT'S IT*. I don't know what to do about it (besides bitch to high heaven), but it is an appalling state of affairs, IMHO. There, I feel better now :-). As to this post, PC100 RAM is not too expensive these days, nothing speeds up swap like more RAM :-). Could you try another 512 MB of RAM ? -- William A. Mahaffey III --------------------------------------------------------------------- Remember, ignorance is bliss, but willful ignorance is LIBERALISM !!!! -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.centos.org/pipermail/centos/attachments/20051214/f0d49f34/attachment-0005.html>