[CentOS] rsync triggers oomkiller

Fri Mar 21 16:49:07 UTC 2014
John Doe <jdmls at yahoo.com>

From: SilverTip257 <silvertip257 at gmail.com>

> To: CentOS mailing list <centos at centos.org>
> Cc: 
> Sent: Friday, March 21, 2014 5:40 PM
> Subject: Re: [CentOS] rsync triggers oomkiller
> 
> I added a subject so we can track this message on the list easier. ;)
> 
> On Fri, Mar 21, 2014 at 12:19 PM, John Doe <jdmls at yahoo.com> wrote:
> 
>>  Hey,
>> 
>>    kernel: rsync invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0,
>>  oom_score_adj=0
>>    ...
>>    kernel: Out of memory: Kill process 27974 (mysqld) score 361 or
>>  sacrifice child
>>    kernel: Killed process 27974, UID 27, (mysqld) total-vm:3804672kB,
>>  anon-rss:2890828kB, file-rss:3324kB
>> 
>>  rsync whines he wants more RAM and... mysql gets killed...
>>  That makes me a bit sad!
>>  And from my nagios graphs, at that time, ram usage was only 75%
>>  (4.5GB/6GB) and swap usage 0.3% (/2GB)!
>>  I cannot believe an rsync suddenly needed 1.5GB+2GB and was not
>>  satisfied...
>>  And even after the out of memory, the graphs show no swap usage change at
>>  all.
>>  Anybody would have an idea about this "fake" out of memory?
>> 
> 
> Wild.
> I've not encountered oomkiller being triggered when the server has free
> memory and hasn't swapped to disk yet.
> 
> After the fact, it will probably be almost impossible to figure out the
> list of files that rsync was storing in memory.
> 
> This makes me wonder if there's an option to have rsync log the list of
> files (to be synced) to a log file of some sort.  I'll have to look into it
> later today when I have a moment.

Thx for the subject!
After more investigation, I found:
- a vm.swappiness=0 in sysctl.conf, which should not prevent the kernel to swap to prevent an oom.
- the rsync was part of 8 *sequential* rsyncs on 8 servers, rsyncing between 500 and 1000 files at most...

Thx,
JD