On 27.2.2012 12:57, Tomas Vondra wrote:
On 27 Únor 2012, 11:26, Peter Kjellström wrote:
On Sunday 26 February 2012 19.59.07 Tomas Vondra wrote: ...
i.e. about 200 MB of free memory, but apache fails because of segfaults when forking a child process:
[16:49:51 2012] [error] (12)Cannot allocate memory: fork: Unable to fork new process [16:51:17 2012] [notice] child pid 2577 exit signal Segmentation fault (11)
In general things can get quite bad with relatively high memory pressure and no swap.
Sure, but there's no such pressure. There was almost 200MB of "free" memory (used for page cache, not dirty thus easy to drop).
That said, one thing that comes to mind is stacksize. When forking the linux kernel needs whatever the current stacksize is to be available as (free + free swap).
Also, just because you see Y bytes free doesn't mean you can successfully malloc that much (fragmentation, memory zones, etc.).
Yup, I'm aware of that. But it's rather improbable, especially given the other symptoms.
Update: After submitting the original post, I've noticed that these issues probably started about a week ago after upgrading a kernel and several related packages. I've had a swap there and the issues were not as severe, so I haven't noticed that before. I do remember I got an OOM error during that upgrade and I thought I've dealt with it properly, but maybe not. So I've reinstalled (remove+install) all those packages, rebooted and the problems disappeared. I will check that in the evening, but hopefully it's fixed.
Well, I've found the actual issue. It clearly was my stupidity as I was messing with overcommit_memory without fully understanding it.
What I did was that I set (as mentioned in the original post)
vm.overcommit_memory = 2
which limits the amount of available memory to
swap + vm.overcommit_ratio * RAM
where vm.overcommit_ratio=50 by default, so you can allocate swap + 1/2 the physical memory. This is just fine if you have a swap - for example if you have swap size equal to RAM, this means 150% of RAM is available for processes.
The issues start when you disable swap (as I did) - then it effectively limits the available memory to 50% of physical RAM (and receive OOM if you try to allocate more. This is exactly what happened to me :-(
So what I did was that I set
vm.overcommit_ratio = 100
which gives me 100% of RAM. I know this will give me an OOM if I use all the physical RAM, but that's expected - I don't want to use swap on a virtual machine with poor I/O (and the services are set accordingly).
So the moral is don't mess with something you don't fully understand.
kind regards Tomas