Paul (Crunch) wrote:
On 04/04/2012 12:31 PM, Nataraj wrote:
On 04/04/2012 09:16 AM, Jonathan Alstead wrote:
Hello,
Recently our dell sc1425 server has been locking up with kernel freezes and required a hard reboot on each occasion. I've looked on the centos forums with limited success - each problem seems slightly different (some failure on high load, some not). Our kernel is 2.6.18-274.17.1.el5 and /var/log/messages show the following errors:
Apr 3 12:41:25 sp2 kernel: INFO: task mysqld:15345 blocked for more than 120 seconds.
<snip>
Apr 3 12:41:25 sp2 kernel: Call Trace: Apr 3 12:41:25 sp2 kernel: [<c0622f16>] rwsem_down_write_failed+0x126/0x141 Apr 3 12:41:25 sp2 kernel: [<c0439989>] .text.lock.rwsem+0x2b/0x3a Apr 3 12:41:25 sp2 kernel: [<c046aa6a>] sys_mprotect+0xbd/0x1eb
Apr 3 12:41:25 sp2 kernel: [<c0404f4b>] syscall_call+0x7/0xb
Apr 3 12:41:25 sp2 kernel: ======================= Apr 3 12:41:25 sp2 kernel: INFO: task clamd:15721 blocked for more than 120 seconds.
<snip>
Apr 3 12:41:26 sp2 kernel: Call Trace: Apr 3 12:41:26 sp2 kernel: [<c041f863>] default_wake_function+0x0/0xc Apr 3 12:41:26 sp2 kernel: [<c048e994>] destroy_inode+0x38/0x47 Apr 3 12:41:26 sp2 kernel: [<c0622f16>] rwsem_down_write_failed+0x126/0x141 Apr 3 12:41:26 sp2 kernel: [<c0439989>] .text.lock.rwsem+0x2b/0x3a Apr 3 12:41:26 sp2 kernel: [<c046a32b>] sys_munmap+0x24/0x41
Apr 3 12:41:26 sp2 kernel: [<c0404f4b>] syscall_call+0x7/0xb
<snip> Looking at the stack traces, and that two completely separate processes are being blocked at the same time, I have to suggest another possibility: the drive that /var is on may be having problems... and if it can't be written to, then it can't log errors, either.
mark