I have a CentOS 6 system that has suddenly starting significantly slowing down. It runs a django app with an Apache server and MySQL server. There is plenty of disk space and no process seems to be hogging the memory or CPU. But operations that used to take 5 minutes are now taking hours and hours.
Coinciding with this slow down I see these messages in /var/log/messages (this is hand typed, as I can't copy/paste or send email from this system, although for a change I do have remote access via 2 VPN hops):
kernel: BUG: soft lockup - CPU#5 stuck for 67s! [khungtaskd] kernel: BUG: soft lockup - CPU#7 stuck for 67s! [khugepaged]
These messages started appearing today around 7 AM, which is when users started reporting the slowdown. They are still occurring periodically and the system is still slow. Are these messages caused by the slowdown or are they the reason for it? Are they useful in tracking down the problem?