Jake Grimmett schrieb:
If I could ask question about 10Gbit ethernet....
We have a 70 node cluster built on Centos 5.1, using NFS to mount user home areas. At the moment the network is a bottleneck, and it's going to get worse as we add another 112 CPUs in the form of two blade servers.
To help things breath better, we are considering building three new NFS servers, and connecting these and the blade servers directly via 10Gbit to a new core switch, the older nodes will stay with 1Gbit ethernet, but be given new switches uplinked via 10Gbit to the core switch.
Before I spend a great deal of money, can I ask if anyone here has experience of 10GBit? Dell, HP, Supermicro, IBM (etc..) seem to be pushing the NetXen PCIe cards, so I guess these drivers work...? As to media, although CX4 seems cheaper than optical, I hear the cabling is nasty. And is the magical fairy going to fix 10Gbit over cat6A anytime soon?
any thoughts appreciated.
Jake
Jake,
please note that the latest RedHat/CentOS-5.1 kernels have NFS performance issues - if these are (partly) responsible for the bottleneck then you may want to install a kernel which fixes the issue.
Check out RedHat bugzilla 321111.
HTH,
Kay