[CentOS] Kernel 2.6.32-573.22.1.el6.x86_64, higher than usual load

Thu May 12 09:00:37 UTC 2016
Laura Herrmann <laurah at sbbsnet.net>

I did a batch update on a number of Xen vservers, with kernel 2.6.32-573.26.1 being installed.

In addition to load averages being high and these #s not being reflected by the individual CPU numbers within top, I now have a spattering of occasional daemon crashes that the vservers have historically never had. Such as, here is the start of a nameserver blowing up in my logs:

May 12 04:03:41 XXX kernel: named invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 : #

May 12 04:03:41 XXX kernel: named cpuset=/ mems_allowed=0 : #

May 12 04:03:41 XXX kernel: Pid: 2208, comm: named Not tainted 2.6.32-573.26.1.el6.x86_64 #1 : #

May 12 04:03:41 XXX kernel: Call Trace: : #

This nameserver has never had a problem with its memory. And, there is another daemon on a different vserver that has done it twice, with the same sort of OOM thing.

And.. what kind of sense does this make.

top - 17:48:53 up 2 days,  9:10,  2 users,  load average: 0.47, 0.29, 0.24
Tasks:  95 total,   1 running,  93 sleeping,   0 stopped,   1 zombie
Cpu0  :  0.3%us,  0.7%sy,  0.0%ni, 96.7%id,  2.3%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu1  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st

There's some problem here, at least because of the crashing out of nowhere on formerly-fine daemons. I'll be reverting all my kernels back to 2.6.32-573.18.1.