I did a batch update on a number of Xen vservers, with kernel 2.6.32-573.26.1 being installed.
In addition to load averages being high and these #s not being reflected by the individual CPU numbers within top, I now have a spattering of occasional daemon crashes that the vservers have historically never had. Such as, here is the start of a nameserver blowing up in my logs:
May 12 04:03:41 XXX kernel: named invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 : #
May 12 04:03:41 XXX kernel: named cpuset=/ mems_allowed=0 : #
May 12 04:03:41 XXX kernel: Pid: 2208, comm: named Not tainted 2.6.32-573.26.1.el6.x86_64 #1 : #
May 12 04:03:41 XXX kernel: Call Trace: : #
This nameserver has never had a problem with its memory. And, there is another daemon on a different vserver that has done it twice, with the same sort of OOM thing.
And.. what kind of sense does this make.
top - 17:48:53 up 2 days, 9:10, 2 users, load average: 0.47, 0.29, 0.24
Tasks: 95 total, 1 running, 93 sleeping, 0 stopped, 1 zombie
Cpu0 : 0.3%us, 0.7%sy, 0.0%ni, 96.7%id, 2.3%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu1 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
There's some problem here, at least because of the crashing out of nowhere on formerly-fine daemons. I'll be reverting all my kernels back to 2.6.32-573.18.1.