On 27.2.2012 12:57, Tomas Vondra wrote: > On 27 Únor 2012, 11:26, Peter Kjellström wrote: >> On Sunday 26 February 2012 19.59.07 Tomas Vondra wrote: >> ... >>> i.e. about 200 MB of free memory, but apache fails because of segfaults >>> when forking a child process: >>> >>> [16:49:51 2012] [error] (12)Cannot allocate memory: fork: Unable to >>> fork new process >>> [16:51:17 2012] [notice] child pid 2577 exit signal Segmentation >>> fault (11) >> >> In general things can get quite bad with relatively high memory pressure >> and >> no swap. > > Sure, but there's no such pressure. There was almost 200MB of "free" > memory (used for page cache, not dirty thus easy to drop). > >> That said, one thing that comes to mind is stacksize. When forking the >> linux >> kernel needs whatever the current stacksize is to be available as (free + >> free >> swap). >> >> Also, just because you see Y bytes free doesn't mean you can successfully >> malloc that much (fragmentation, memory zones, etc.). > > Yup, I'm aware of that. But it's rather improbable, especially given the > other symptoms. > > Update: After submitting the original post, I've noticed that these issues > probably started about a week ago after upgrading a kernel and several > related packages. I've had a swap there and the issues were not as severe, > so I haven't noticed that before. I do remember I got an OOM error during > that upgrade and I thought I've dealt with it properly, but maybe not. So > I've reinstalled (remove+install) all those packages, rebooted and the > problems disappeared. I will check that in the evening, but hopefully it's > fixed. Well, I've found the actual issue. It clearly was my stupidity as I was messing with overcommit_memory without fully understanding it. What I did was that I set (as mentioned in the original post) vm.overcommit_memory = 2 which limits the amount of available memory to swap + vm.overcommit_ratio * RAM where vm.overcommit_ratio=50 by default, so you can allocate swap + 1/2 the physical memory. This is just fine if you have a swap - for example if you have swap size equal to RAM, this means 150% of RAM is available for processes. The issues start when you disable swap (as I did) - then it effectively limits the available memory to 50% of physical RAM (and receive OOM if you try to allocate more. This is exactly what happened to me :-( So what I did was that I set vm.overcommit_ratio = 100 which gives me 100% of RAM. I know this will give me an OOM if I use all the physical RAM, but that's expected - I don't want to use swap on a virtual machine with poor I/O (and the services are set accordingly). So the moral is don't mess with something you don't fully understand. kind regards Tomas