On a busy NFS server I've started receiving the following error messages and the system hangs with high load but no work being done. The system is a Dell R510 with 12 x 3TB drives in a RAID-50 configuration. The RAID-50 device is a full disk LVM (no partitions) and one large (36TB) data volume and the system is running CentOS 6.4 fully patched for OS and firmware.
Anyone have any hints as to what might be causing this. It looks to me like the system is starting to swap and then failing and then XFS starts to throw a hissy fit because it can't start allocating pages for it's buffers. Eventually then the system just deadlocks. I'm just looking for someone to confirm my findings and let me know if this is a bug or not.
I've posted the dmesg output onto pastebin http://pastebin.com/YQbhsN6a
Any help is very much appreciated!
From: James A. Peltier jpeltier@sfu.ca
On a busy NFS server I've started receiving the following error messages and the system hangs with high load but no work being done. The system is a Dell R510 with 12 x 3TB drives in a RAID-50 configuration. The RAID-50 device is a full disk LVM (no partitions) and one large (36TB) data volume and the system is running CentOS 6.4 fully patched for OS and firmware.
Anyone have any hints as to what might be causing this. It looks to me like the system is starting to swap and then failing and then XFS starts to throw a hissy fit because it can't start allocating pages for it's buffers. Eventually then the system just deadlocks. I'm just looking for someone to confirm my findings and let me know if this is a bug or not.
I've posted the dmesg output onto pastebin http://pastebin.com/YQbhsN6a
Any help is very much appreciated!
Google seems to suggest to maybe perhaps look at: sysctl vm.min_free_kbytes
JD
-
James A. Peltier -
John Doe