[CentOS] 10Gbit ethernet

[Jason]

Where abouts are you maxing out?  I assume it's not to the nodes but
rather to the NFS head. If that's the case, you might want to look at a
different, somewhat more time tested approach of standing up additional
NFS heads (as you already mentioned).   Having said that, we are
currently using 10g links between a pair of Cisco 6509E's and another
pair to a set of Foundry FESX48s (SAN traffic). No real issues to speak
of, but this is backbone and not to the server. Our busiest machine,
network wise, is our backup server and we've just bound a pair of 1 gig
links to it with great success.

Jake Grimmett wrote:
> If I could ask question about 10Gbit ethernet....
> 
> We have a 70 node cluster built on Centos 5.1, using NFS to mount user home 
> areas. At the moment the network is a bottleneck, and it's going to get worse 
> as we add another 112 CPUs in the form of two blade servers. 
> 
> To help things breath better, we are considering building three new NFS 
> servers, and connecting these and the blade servers directly via 10Gbit to a 
> new core switch, the older nodes will stay with 1Gbit ethernet, but be given 
> new switches uplinked via 10Gbit to the core switch.
> 
> Before I spend a great deal of money, can I ask if anyone here has experience 
> of 10GBit? Dell, HP, Supermicro, IBM (etc..) seem to be pushing the NetXen 
> PCIe cards, so I guess these drivers work...? As to media, although CX4 seems 
> cheaper than optical, I hear the cabling is nasty. And is the magical fairy 
> going to fix 10Gbit over cat6A anytime soon?
> 
> any thoughts appreciated.
> 
> Jake