[CentOS] oom-killer.
William A. Mahaffey III
wam at HiWAAY.net
Wed Dec 14 14:43:25 UTC 2005
J.J.Garcia wrote:
>Hi all,
>
>Just watching for my first time the 'oom-killer' call in a CentOS 4.2
>workstation. System was running normally but with high load of nice process,
>iow, rosetta and seti, others appart, nautilus, etc... i realized that after a
>night of standby running normally, the next moorning after few manual operations
>from X-session in local console, it started a massive swapping, this time not
>recovered as previously other time, finishing in a oom-killer call.
>
>Finally inspected the syslog and realize the oom-killer was called to leave the
>system stable having killed firefox-bin, 3 x rosetta, xmms and one nautilus.
>
>It's a PIII 800 Mhz, 40GB IDE, 512 Mb RAM, 1024 Swap space, up for 4 days.
>
>I'm not very xperienced in tunning the memmory system nor the swap system, help
>appreciated to avoid this memory 'trashing' situtation (14 virtual desktops
>openned at the same time).
>
>Is it a normal situation? Is it a correct protecction measure? It's due a low
>bandwitch IDE HD? how can i tune it to avoid this situations? is it a high load
>for this workstation? I think the oom-killer was called correctly to avoid the
>system collapsed, but i think there was a problem in the swapping subsystem or a
>memmory trashing situation to avoid.
>
>Help and hints appreciated,
>
>Attached you'll the oom-killer log,
>
>TIA
>
>J.J.
>
>
>
>
>
>
>
>
>
>
>
>
>
>------------------------------------------------------------------------
>
>Dec 14 11:48:00 goliathbox kernel: oom-killer: gfp_mask=0xd0
>Dec 14 11:48:00 goliathbox kernel: Mem-info:
>Dec 14 11:48:00 goliathbox kernel: DMA per-cpu:
>Dec 14 11:48:00 goliathbox kernel: cpu 0 hot: low 2, high 6, batch 1
>Dec 14 11:48:00 goliathbox kernel: cpu 0 cold: low 0, high 2, batch 1
>Dec 14 11:48:00 goliathbox kernel: Normal per-cpu:
>Dec 14 11:48:00 goliathbox kernel: cpu 0 hot: low 32, high 96, batch 16
>Dec 14 11:48:00 goliathbox kernel: cpu 0 cold: low 0, high 32, batch 16
>Dec 14 11:48:00 goliathbox kernel: HighMem per-cpu: empty
>Dec 14 11:48:00 goliathbox kernel:
>Dec 14 11:48:00 goliathbox kernel: Free pages: 676kB (0kB HighMem)
>Dec 14 11:48:00 goliathbox kernel: Active:109642 inactive:10334 dirty:0 writeback:0 unstable:0 free:169 slab:3746 mapped:118866 pagetables:1729
>Dec 14 11:48:00 goliathbox kernel: DMA free:20kB min:20kB low:40kB high:60kB active:6132kB inactive:6316kB present:16384kB pages_scanned:20658 all_unreclaimable? yes
>Dec 14 11:48:02 goliathbox kernel: protections[]: 0 0 0
>Dec 14 11:48:23 goliathbox kernel: Normal free:656kB min:700kB low:1400kB high:2100kB active:432436kB inactive:35020kB present:507820kB pages_scanned:1102915 all_unreclaimable? yesDec 14 11:48:23 goliathbox kernel: protections[]: 0 0 0
>Dec 14 11:48:23 goliathbox kernel: HighMem free:0kB min:128kB low:256kB high:384kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
>Dec 14 11:48:23 goliathbox kernel: protections[]: 0 0 0
>Dec 14 11:48:23 goliathbox kernel: DMA: 1*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 20kB
>Dec 14 11:48:23 goliathbox kernel: Normal: 0*4kB 6*8kB 2*16kB 4*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 656kB
>Dec 14 11:48:23 goliathbox kernel: HighMem: empty
>Dec 14 11:48:23 goliathbox kernel: Swap cache: add 974772, delete 974259, find 292228/382858, race 0+15
>Dec 14 11:48:23 goliathbox kernel: 0 bounce buffer pages
>Dec 14 11:48:23 goliathbox kernel: Free swap: 0kB
>Dec 14 11:48:23 goliathbox kernel: 131051 pages of RAM
>Dec 14 11:48:23 goliathbox kernel: 0 pages of HIGHMEM
>Dec 14 11:48:42 goliathbox kernel: 2433 reserved pages
>Dec 14 11:49:26 goliathbox kernel: 26433 pages shared
>Dec 14 11:49:26 goliathbox kernel: 513 pages swap cached
>Dec 14 11:49:26 goliathbox kernel: Out of Memory: Killed process 13954 (firefox-bin).
>Dec 14 11:49:26 goliathbox kernel: oom-killer: gfp_mask=0x1d2
>Dec 14 11:49:26 goliathbox kernel: Mem-info:
>Dec 14 11:49:26 goliathbox kernel: DMA per-cpu:
>Dec 14 11:49:26 goliathbox kernel: cpu 0 hot: low 2, high 6, batch 1
>Dec 14 11:49:26 goliathbox kernel: cpu 0 cold: low 0, high 2, batch 1
>Dec 14 11:49:26 goliathbox kernel: Normal per-cpu:
>Dec 14 11:49:26 goliathbox kernel: cpu 0 hot: low 32, high 96, batch 16
>Dec 14 11:49:26 goliathbox kernel: cpu 0 cold: low 0, high 32, batch 16
>Dec 14 11:49:26 goliathbox kernel: HighMem per-cpu: empty
>Dec 14 11:49:26 goliathbox kernel:
>Dec 14 11:49:27 goliathbox kernel: Free pages: 676kB (0kB HighMem)
>Dec 14 11:49:27 goliathbox kernel: Active:1868 inactive:118396 dirty:0 writeback:0 unstable:0 free:169 slab:3738 mapped:118869 pagetables:1729
>Dec 14 11:49:27 goliathbox kernel: DMA free:20kB min:20kB low:40kB high:60kB active:4232kB inactive:7992kB present:16384kB pages_scanned:20064 all_unreclaimable? yes
>Dec 14 11:49:28 goliathbox kernel: protections[]: 0 0 0
>Dec 14 11:49:28 goliathbox kernel: Normal free:656kB min:700kB low:1400kB high:2100kB active:3240kB inactive:465592kB present:507820kB pages_scanned:780089 all_unreclaimable? yes
>Dec 14 11:49:28 goliathbox kernel: protections[]: 0 0 0
>Dec 14 11:49:29 goliathbox kernel: HighMem free:0kB min:128kB low:256kB high:384kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
>Dec 14 11:49:29 goliathbox kernel: protections[]: 0 0 0
>Dec 14 11:49:29 goliathbox kernel: DMA: 1*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 20kB
>Dec 14 11:49:29 goliathbox kernel: Normal: 0*4kB 6*8kB 2*16kB 4*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 656kB
>Dec 14 11:49:29 goliathbox kernel: HighMem: empty
>Dec 14 11:49:29 goliathbox kernel: Swap cache: add 974851, delete 974338, find 292235/382874, race 0+15
>Dec 14 11:49:30 goliathbox kernel: 0 bounce buffer pages
>Dec 14 11:49:30 goliathbox kernel: Free swap: 0kB
>Dec 14 11:49:30 goliathbox kernel: 131051 pages of RAM
>Dec 14 11:49:30 goliathbox kernel: 0 pages of HIGHMEM
>Dec 14 11:49:31 goliathbox kernel: 2433 reserved pages
>Dec 14 11:49:31 goliathbox kernel: 26140 pages shared
>Dec 14 11:49:31 goliathbox kernel: 513 pages swap cached
>Dec 14 11:49:31 goliathbox kernel: Out of Memory: Killed process 15771 (firefox-bin).
>Dec 14 11:49:31 goliathbox kernel: oom-killer: gfp_mask=0xd0
>Dec 14 11:49:31 goliathbox kernel: Mem-info:
>Dec 14 11:49:32 goliathbox kernel: DMA per-cpu:
>Dec 14 11:49:32 goliathbox kernel: cpu 0 hot: low 2, high 6, batch 1
>Dec 14 11:49:32 goliathbox kernel: cpu 0 cold: low 0, high 2, batch 1
>Dec 14 11:49:32 goliathbox kernel: Normal per-cpu:
>Dec 14 11:49:32 goliathbox kernel: cpu 0 hot: low 32, high 96, batch 16
>Dec 14 11:49:32 goliathbox kernel: cpu 0 cold: low 0, high 32, batch 16
>Dec 14 11:49:32 goliathbox kernel: HighMem per-cpu: empty
>Dec 14 11:49:33 goliathbox kernel:
>Dec 14 11:49:33 goliathbox kernel: Free pages: 676kB (0kB HighMem)
>Dec 14 11:49:33 goliathbox kernel: Active:73675 inactive:46271 dirty:0 writeback:0 unstable:0 free:169 slab:3735 mapped:118871 pagetables:1729
>Dec 14 11:49:33 goliathbox kernel: DMA free:20kB min:20kB low:40kB high:60kB active:4384kB inactive:7968kB present:16384kB pages_scanned:20229 all_unreclaimable? yes
>Dec 14 11:49:33 goliathbox kernel: protections[]: 0 0 0
>Dec 14 11:49:33 goliathbox kernel: Normal free:656kB min:700kB low:1400kB high:2100kB active:290316kB inactive:177116kB present:507820kB pages_scanned:556644 all_unreclaimable? yesDec 14 11:49:34 goliathbox kernel: protections[]: 0 0 0
>Dec 14 11:49:34 goliathbox kernel: HighMem free:0kB min:128kB low:256kB high:384kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
>Dec 14 11:49:34 goliathbox kernel: protections[]: 0 0 0
>Dec 14 11:49:34 goliathbox kernel: DMA: 1*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 20kB
>Dec 14 11:49:34 goliathbox kernel: Normal: 0*4kB 6*8kB 2*16kB 4*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 656kB
>Dec 14 11:49:34 goliathbox kernel: HighMem: empty
>Dec 14 11:49:35 goliathbox kernel: Swap cache: add 974851, delete 974338, find 292235/382874, race 0+15
>Dec 14 11:49:35 goliathbox kernel: 0 bounce buffer pages
>Dec 14 11:49:35 goliathbox kernel: Free swap: 0kB
>Dec 14 11:49:35 goliathbox kernel: 131051 pages of RAM
>Dec 14 11:49:35 goliathbox kernel: 0 pages of HIGHMEM
>Dec 14 11:49:35 goliathbox kernel: 2433 reserved pages
>Dec 14 11:49:36 goliathbox kernel: 26474 pages shared
>Dec 14 11:49:36 goliathbox kernel: 513 pages swap cached
>Dec 14 11:49:36 goliathbox kernel: Out of Memory: Killed process 13973 (rosetta_4.79_i6).
>Dec 14 11:49:36 goliathbox kernel: Out of Memory: Killed process 13972 (rosetta_4.79_i6).
>Dec 14 11:49:36 goliathbox kernel: Out of Memory: Killed process 13974 (rosetta_4.79_i6).
>Dec 14 11:49:36 goliathbox kernel: oom-killer: gfp_mask=0x1d2
>Dec 14 11:49:36 goliathbox kernel: Mem-info:
>Dec 14 11:49:37 goliathbox kernel: DMA per-cpu:
>Dec 14 11:49:38 goliathbox kernel: cpu 0 hot: low 2, high 6, batch 1
>Dec 14 11:49:38 goliathbox kernel: cpu 0 cold: low 0, high 2, batch 1
>Dec 14 11:49:39 goliathbox kernel: Normal per-cpu:
>Dec 14 11:49:39 goliathbox kernel: cpu 0 hot: low 32, high 96, batch 16
>Dec 14 11:49:40 goliathbox kernel: cpu 0 cold: low 0, high 32, batch 16
>Dec 14 11:49:40 goliathbox kernel: HighMem per-cpu: empty
>Dec 14 11:49:40 goliathbox kernel:
>Dec 14 11:49:41 goliathbox kernel: Free pages: 676kB (0kB HighMem)
>Dec 14 11:49:41 goliathbox kernel: Active:5356 inactive:114646 dirty:0 writeback:0 unstable:0 free:169 slab:3735 mapped:118875 pagetables:1729
>Dec 14 11:49:41 goliathbox kernel: DMA free:20kB min:20kB low:40kB high:60kB active:4412kB inactive:8044kB present:16384kB pages_scanned:20361 all_unreclaimable? yes
>Dec 14 11:49:41 goliathbox crond(pam_unix)[5432]: session opened for user root by (uid=0)
>Dec 14 11:49:41 goliathbox crond(pam_unix)[5434]: session opened for user root by (uid=0)
>Dec 14 11:49:41 goliathbox crond(pam_unix)[5433]: session opened for user sm0ketst by (uid=0)
>Dec 14 11:49:41 goliathbox kernel: protections[]: 0 0 0
>Dec 14 11:49:42 goliathbox kernel: Normal free:656kB min:700kB low:1400kB high:2100kB active:17012kB inactive:450540kB present:507820kB pages_scanned:2398502 all_unreclaimable? yesDec 14 11:49:43 goliathbox kernel: protections[]: 0 0 0
>Dec 14 11:49:43 goliathbox kernel: HighMem free:0kB min:128kB low:256kB high:384kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
>Dec 14 11:49:43 goliathbox kernel: protections[]: 0 0 0
>Dec 14 11:49:44 goliathbox kernel: DMA: 1*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 20kB
>Dec 14 11:49:44 goliathbox kernel: Normal: 0*4kB 6*8kB 2*16kB 4*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 656kB
>Dec 14 11:49:44 goliathbox kernel: HighMem: empty
>Dec 14 11:49:44 goliathbox kernel: Swap cache: add 974851, delete 974338, find 292235/382874, race 0+15
>Dec 14 11:49:45 goliathbox kernel: 0 bounce buffer pages
>Dec 14 11:49:45 goliathbox kernel: Free swap: 0kB
>Dec 14 11:49:46 goliathbox kernel: 131051 pages of RAM
>Dec 14 11:49:46 goliathbox kernel: 0 pages of HIGHMEM
>Dec 14 11:49:46 goliathbox kernel: 2433 reserved pages
>Dec 14 11:49:46 goliathbox kernel: 26419 pages shared
>Dec 14 11:49:47 goliathbox kernel: 513 pages swap cached
>Dec 14 11:49:47 goliathbox kernel: Out of Memory: Killed process 4936 (xmms).
>Dec 14 11:49:47 goliathbox kernel: oom-killer: gfp_mask=0x1d2
>Dec 14 11:49:48 goliathbox kernel: Mem-info:
>Dec 14 11:49:48 goliathbox kernel: DMA per-cpu:
>Dec 14 11:49:48 goliathbox kernel: cpu 0 hot: low 2, high 6, batch 1
>Dec 14 11:49:49 goliathbox kernel: cpu 0 cold: low 0, high 2, batch 1
>Dec 14 11:49:49 goliathbox kernel: Normal per-cpu:
>Dec 14 11:49:49 goliathbox kernel: cpu 0 hot: low 32, high 96, batch 16
>Dec 14 11:49:49 goliathbox kernel: cpu 0 cold: low 0, high 32, batch 16
>Dec 14 11:49:49 goliathbox kernel: HighMem per-cpu: empty
>Dec 14 11:49:50 goliathbox crond(pam_unix)[5390]: session closed for user root
>Dec 14 11:49:50 goliathbox kernel:
>Dec 14 11:49:50 goliathbox crond(pam_unix)[5432]: session closed for user root
>Dec 14 11:49:50 goliathbox kernel: Free pages: 676kB (0kB HighMem)
>Dec 14 11:49:51 goliathbox kernel: Active:31292 inactive:88792 dirty:0 writeback:0 unstable:0 free:169 slab:3717 mapped:118877 pagetables:1729
>Dec 14 11:49:51 goliathbox kernel: DMA free:20kB min:20kB low:40kB high:60kB active:4412kB inactive:8044kB present:16384kB pages_scanned:20955 all_unreclaimable? yes
>Dec 14 11:49:51 goliathbox kernel: protections[]: 0 0 0
>Dec 14 11:49:51 goliathbox kernel: Normal free:656kB min:700kB low:1400kB high:2100kB active:120756kB inactive:347124kB present:507820kB pages_scanned:578655 all_unreclaimable? yesDec 14 11:49:51 goliathbox kernel: protections[]: 0 0 0
>Dec 14 11:49:51 goliathbox crond(pam_unix)[5434]: session closed for user root
>Dec 14 11:49:51 goliathbox kernel: HighMem free:0kB min:128kB low:256kB high:384kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
>Dec 14 11:49:52 goliathbox crond(pam_unix)[5433]: session closed for user sm0ketst
>Dec 14 11:49:52 goliathbox kernel: protections[]: 0 0 0
>Dec 14 11:49:52 goliathbox kernel: DMA: 1*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 20kB
>Dec 14 11:49:52 goliathbox kernel: Normal: 0*4kB 6*8kB 2*16kB 4*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 656kB
>Dec 14 11:49:52 goliathbox kernel: HighMem: empty
>Dec 14 11:49:52 goliathbox kernel: Swap cache: add 974851, delete 974338, find 292235/382874, race 0+15
>Dec 14 11:49:52 goliathbox kernel: 0 bounce buffer pages
>Dec 14 11:49:53 goliathbox kernel: Free swap: 0kB
>Dec 14 11:49:53 goliathbox kernel: 131051 pages of RAM
>Dec 14 11:49:53 goliathbox kernel: 0 pages of HIGHMEM
>Dec 14 11:49:53 goliathbox kernel: 2433 reserved pages
>Dec 14 11:49:53 goliathbox kernel: 26379 pages shared
>Dec 14 11:49:53 goliathbox kernel: 513 pages swap cached
>Dec 14 11:49:53 goliathbox kernel: Out of Memory: Killed process 5129 (nautilus).
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>CentOS mailing list
>CentOS at centos.org
>http://lists.centos.org/mailman/listinfo/centos
>
>
(light) rant alert: WOOP WOOP WOOP !!!!!
Since the mid '90's, when I first used Linux (Slackware, from a fistful
of floppies, on a 486 w/ 32 MB RAM & some tiny HDD), Linux has quietly
become a memory PIG. That 1st install did have X running (FVWM IIRC) &
it did run, including porting our code from SGI to Linux, which was the
purpose of the work at that time (feasibility study, for NASA Langley).
As I write this on my 933 MHz PIII, 1 GB RAM, 1 GB swap, SuSE 8.2, all
stock, I observe from a 'free -m' in a shell:
[wam at DARKSTAR, ~/FTP, 8:30:18am] 938 % free -m
total used free shared buffers cached
Mem: 1008 995 13 0 83 748
-/+ buffers/cache: 163 845
Swap: 1023 27 996
or 163 (!!!!!) MB being used, & for what ? 4 KDE destops, a total of 6
RXVT (supposedly light on RAM use) shells open, Mozilla as I write this,
NFS & Samba daemons, & *THAT'S IT*. I don't know what to do about it
(besides bitch to high heaven), but it is an appalling state of affairs,
IMHO.
There, I feel better now :-).
As to this post, PC100 RAM is not too expensive these days, nothing
speeds up swap like more RAM :-). Could you try another 512 MB of RAM ?
--
William A. Mahaffey III
---------------------------------------------------------------------
Remember, ignorance is bliss, but
willful ignorance is LIBERALISM !!!!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.centos.org/pipermail/centos/attachments/20051214/f0d49f34/attachment.html>
More information about the CentOS
mailing list