To the list:
1) Someone off the list (thanks!) offered to send me an RT kernel based on 2.6.23.X. So I thought why not try to build a standard / server type based on the standard 2.6.23 version.
2) Red Hat 6 Beta 2 is based on a similar starting point
3) I was not going to fix something Red Hat broke. They responded to me that they only support the configuration of the generic binary they ship. This is the answer I received from them for bug # 558367
"Jarod Wilson 2010-07-14 13:40:06 EDT
Officially, Red Hat does not support rebuilds of its kernel with configurations other than the one Red Hat ships. You're welcome to submit a patch to fix the build, which will be considered for inclusion, and/or wait for someone here to have the time to work on this, but there are only so many hours in the day. Most of Red Hat engineering is busy working on problems related to configurations that Red Hat *does* support. If this is a particularly serious issue for you though, you're welcome to try to escalate it through customer support. "
/Flame proof on
This answer brings to light some very serious limitations. First, if you want to turn on a standard kernel feature and its broken (like what I tried), it is not officially supported by Red Hat nor Centos.
Therefore, xfs, ntfs, and many other features quite a few of us use may be broken at any time when the upstream provider "fixes or updates" a kernel.
/Flame proof off/
I am just stating a warning based on an official answer. NO OPINION OFFERED OR IMPLIED
Now for the fix....
No, it was not me that fixed this. I just thought maybe we should try a kernel that has been thoroughly tested by the "kernel people".
I used the kernel.org, 2.6.23.17 kernel I used the old .config from the latest Centos 5.5 src.rpm Ran make oldconfig I answered most new questions with the default I enabled AMD Opteron, Disabled optimize for size and changed the default CPU frequency governor to performance. I turned off isdn, infiniband, ATM and a few others (we don't use them) I turned on NTFS and with writes (it was broken in 5.2), xfs and jfs
then make; make install_modules mkinitrd ; edit grub; reboot
This seems to work fine on a standard Centos 5.5 x86_64 install. I am going to run this on a test server for a few weeks and see how well it works. If I see any issues I will report them but so far it works just fine