Hi all,
I'm just curios and would like some input from the community on this one. We're busy budgeting for a couple of new servers and I thought it would be good to try out the Core i7 CPU's, but see the majority of them don't offer VT-d, but just VT-x. Looking at the LGA1366 range, only the "Intel lga1366 i7 980XE" (from the list of what our suppliers stock) have VT-d, and it costs 4x more than "Intel lga1366 i7 930" or 2x more than "Intel lga1366 i7 960". From a budget perspecitve I could purchase 4 more CPU's, which could translate to 40x - 80x more VM's being hosted for the same capital outlay. Experience has shown that we under-utilize CPU's by a great margin and memory / HDD IO is our biggest bottleneck on any server.
So, if VT-d really necessary? We mainly host XEN virtual machine for the hosting industry, i.e. we don't need / use graphics rendering inside VM's, or need DAS on the VM's, etc.
So, if VT-d really necessary? We mainly host XEN virtual machine for the hosting industry, i.e. we don't need / use graphics rendering inside VM's, or need DAS on the VM's, etc.
Kind Regards Rudi Ahlers SoftDux
VT-d is not a necessity in general. It all depends on the kind of virtualization you run. If you run only paravirtualized guests on Xen, then there is not any need for VT-d (see [1]). If you fully virtualize for instance Windows guests, then specific systems may profit i.e. from a reserved network card by better network I/O.
Whether VT-d is useful to have may too depend on whether pricing for the customer can be adjusted when providing such an extra feature.
From own experience I concur that often with most modern x86 architecture
systems the I/O (network and storage) is the bottleneck in the area of virtualization.
Just curious, do you really run virtualization for hosting on systems with uni-processor design? I mean not choosing professional dual quad- or hexa-core processor systems with Nehalem / Westmere Xeon CPUs or their AMD Opteron counterpart?
Regards
Alexander
On Wed, Sep 15, 2010 at 1:28 PM, Alexander Dalloz ad+lists@uni-x.org wrote:
VT-d is not a necessity in general. It all depends on the kind of virtualization you run. If you run only paravirtualized guests on Xen, then there is not any need for VT-d (see [1]). If you fully virtualize for instance Windows guests, then specific systems may profit i.e. from a reserved network card by better network I/O.
ok, that's kinda what I thought as well. We don't use Windows VM's, but rather dedicated Windows servers if needed.
Whether VT-d is useful to have may too depend on whether pricing for the customer can be adjusted when providing such an extra feature.
From own experience I concur that often with most modern x86 architecture
systems the I/O (network and storage) is the bottleneck in the area of virtualization.
That's what I found as well so I want to concentrate on better I/O throughput. In general we don't saturate the 1GB NIC's. As soon as it gets to a certain threshold, we move some of the VM's to a new server instead. I guess I could look into dual / quad NIC bonding, but that would mean downtime on the server while the NIC's are installed.
Just curious, do you really run virtualization for hosting on systems with uni-processor design? I mean not choosing professional dual quad- or hexa-core processor systems with Nehalem / Westmere Xeon CPUs or their AMD Opteron counterpart?
Generally we use systems with 8cores+, i.e. Quad Core with HT, or dual Quad Core with HT. But I want to see if I could use the money spend on those expensive CPU setups more wisely with RAM & I/O.
Regards
Alexander
[1] http://wiki.xensource.com/xenwiki/VTdHowTo
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Hi,
I'm just curios and would like some input from the community on this one. We're busy budgeting for a couple of new servers and I thought it would be good to try out the Core i7 CPU's, but see the majority of them don't offer VT-d, but just VT-x. Looking at the LGA1366 range, only the "Intel lga1366 i7 980XE" (from the list of what our suppliers stock) have VT-d, and it costs 4x more than "Intel lga1366 i7 930" or 2x more than "Intel lga1366 i7 960".
How about the Xeon processors? Are they cheaper than the i7-980XE there? Also keep in mind your board has to support VT-D as well.
From a budget perspecitve I could purchase 4 more CPU's, which could translate to 40x - 80x more VM's being hosted for the same capital outlay. Experience has shown that we under-utilize CPU's by a great margin and memory / HDD IO is our biggest bottleneck on any server.
From this it seems that you would be running 10 to 20 VM per machine.
According to Intel, it's not recommended if the physical NIC is shared. So in your case, the VT-d issue might be moot.
On 09/15/10 2:19 AM, Rudi Ahlers wrote:
Hi all,
I'm just curios and would like some input from the community on this one. We're busy budgeting for a couple of new servers and I thought it would be good to try out the Core i7 CPU's, but see the majority of them don't offer VT-d, but just VT-x. Looking at the LGA1366 range, only the "Intel lga1366 i7 980XE" (from the list of what our suppliers stock) have VT-d, and it costs 4x more than "Intel lga1366 i7 930" or 2x more than "Intel lga1366 i7 960". From a budget perspecitve I could purchase 4 more CPU's, which could translate to 40x - 80x more VM's being hosted for the same capital outlay. Experience has shown that we under-utilize CPU's by a great margin and memory / HDD IO is our biggest bottleneck on any server.
So, if VT-d really necessary? We mainly host XEN virtual machine for the hosting industry, i.e. we don't need / use graphics rendering inside VM's, or need DAS on the VM's, etc.
Core I7 is the branding for the desktop CPU family. The Server processors are branded Xeon 5500 and 5600 (for dual socket servers) and Xeon 7000 for 4+ socket servers. Typically, desktop processors go with desktop motherboards which don't support ECC memory, probably don't have remote management features, likely don't readily support redundant power, and often have only a single NIC onboard. A server board will likely have significantly more IO bandwidth, oriented towards network and disk IO rather than graphics.
IMHO, the dual socket 5600 family is the sweet spot of price/performance for a VM host, with 2 x 6 cores, and typically 12 memory slots (2x3 per CPU). populate the memory with 6 matching DIMMs for best performance.
On Thu, Sep 16, 2010 at 8:24 AM, John R Pierce pierce@hogranch.com wrote:
Core I7 is the branding for the desktop CPU family. The Server processors are branded Xeon 5500 and 5600 (for dual socket servers) and Xeon 7000 for 4+ socket servers. Typically, desktop processors go with desktop motherboards which don't support ECC memory, probably don't have remote management features, likely don't readily support redundant power, and often have only a single NIC onboard. A server board will likely have significantly more IO bandwidth, oriented towards network and disk IO rather than graphics.
IMHO, the dual socket 5600 family is the sweet spot of price/performance for a VM host, with 2 x 6 cores, and typically 12 memory slots (2x3 per CPU). populate the memory with 6 matching DIMMs for best performance.
Yet the server vendors ship servers, with server chassis, hardware RAID, redundant power supplies, etc & offer Core i7 options. How does that work?
On 09/16/10 12:16 AM, Rudi Ahlers wrote:
Yet the server vendors ship servers, with server chassis, hardware RAID, redundant power supplies, etc& offer Core i7 options. How does that work?
low end servers, i guess. I'd have to see a specific model to comment specifically.
most of the stuff I look at is like HP DL380 G-whatever and so forth, these tend to be dual xeon 5xxx
On Thursday, September 16, 2010 03:37:23 am John R Pierce wrote:
On 09/16/10 12:16 AM, Rudi Ahlers wrote:
Yet the server vendors ship servers, with server chassis, hardware RAID, redundant power supplies, etc& offer Core i7 options. How does that work?
low end servers, i guess. I'd have to see a specific model to comment specifically.
Dell's PowerEdge R210 and R310 are available with Core i3.
On Thu, Sep 16, 2010 at 5:32 PM, Lamar Owen lowen@pari.edu wrote:
On Thursday, September 16, 2010 03:37:23 am John R Pierce wrote:
On 09/16/10 12:16 AM, Rudi Ahlers wrote:
Yet the server vendors ship servers, with server chassis, hardware RAID, redundant power supplies, etc& offer Core i7 options. How does that work?
low end servers, i guess. I'd have to see a specific model to comment specifically.
Dell's PowerEdge R210 and R310 are available with Core i3.
But those aren't servers since the Core iX CPU's are desktop class CPU's ;)
On 09/16/10 8:56 AM, Rudi Ahlers wrote:
Dell's PowerEdge R210 and R310 are available with Core i3.
But those aren't servers since the Core iX CPU's are desktop class CPU's ;)
those are low end servers in Dell's line.
I'd be looking at the R410 or R710 for virtualization hosts, along with external storage. you can put far more memory in these. the 2U version lets you add more network and HBA (storage) interfaces than the 1U has room for.
On Thu, Sep 16, 2010 at 7:09 PM, John R Pierce pierce@hogranch.com wrote:
On 09/16/10 8:56 AM, Rudi Ahlers wrote:
Dell's PowerEdge R210 and R310 are available with Core i3.
But those aren't servers since the Core iX CPU's are desktop class CPU's ;)
those are low end servers in Dell's line.
I'd be looking at the R410 or R710 for virtualization hosts, along with external storage. you can put far more memory in these. the 2U version lets you add more network and HBA (storage) interfaces than the 1U has room for.
John, my statement was a bit sarcastic :)
But, the truth is these servers exist, with Core i3, i5 & i7 CPU's from various vendors. I can get a nice big setup on the SuperMicro side with 32GB RAM, Core i7, 4x RAID 10 HDD's, etc.
But I'm convinced now that XEON's would be better even though they're far more expensive. And at the same time our older Pentium IV, Core2Duo & Core2Quad machines work as well as our XEON machines, but at much cheaper prices - which translates to more money being available to other hardware / services / marketing / etc.
On September 16, 2010 10:23:15 am Rudi Ahlers wrote:
But I'm convinced now that XEON's would be better even though they're far more expensive. And at the same time our older Pentium IV, Core2Duo & Core2Quad machines work as well as our XEON machines, but at much cheaper prices - which translates to more money being available to other hardware / services / marketing / etc.
They're not that much more expensive, at least for the dual-socket boards. And you get a lot better I/O and memory bandwidth from real server boards.
The 6 core Xeons are still kinda pricey though.
Supermicro also makes fine server boards and a great range of rack-mount chassis, including blades, for much cheaper than, say, HP.
On Thu, Sep 16, 2010 at 7:37 PM, Alan Hodgson ahodgson@simkin.ca wrote:
On September 16, 2010 10:23:15 am Rudi Ahlers wrote:
But I'm convinced now that XEON's would be better even though they're far more expensive. And at the same time our older Pentium IV, Core2Duo & Core2Quad machines work as well as our XEON machines, but at much cheaper prices - which translates to more money being available to other hardware / services / marketing / etc.
They're not that much more expensive, at least for the dual-socket boards. And you get a lot better I/O and memory bandwidth from real server boards.
The 6 core Xeons are still kinda pricey though.
Supermicro also makes fine server boards and a great range of rack-mount chassis, including blades, for much cheaper than, say, HP. _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Alan, I am using server boards.....
On Wed, Sep 15, 2010 at 11:19:38AM +0200, Rudi Ahlers wrote:
Hi all,
I'm just curios and would like some input from the community on this one. We're busy budgeting for a couple of new servers and I thought it would be good to try out the Core i7 CPU's, but see the majority of them don't offer VT-d, but just VT-x. Looking at the LGA1366 range, only the "Intel lga1366 i7 980XE" (from the list of what our suppliers stock) have VT-d, and it costs 4x more than "Intel lga1366 i7 930" or 2x more than "Intel lga1366 i7 960". From a budget perspecitve I could purchase 4 more CPU's, which could translate to 40x - 80x more VM's being hosted for the same capital outlay. Experience has shown that we under-utilize CPU's by a great margin and memory / HDD IO is our biggest bottleneck on any server.
So, if VT-d really necessary? We mainly host XEN virtual machine for the hosting industry, i.e. we don't need / use graphics rendering inside VM's, or need DAS on the VM's, etc.
VT-d is marketing term for Intel's IOMMU (IO MMU) implementation, and it's used *only* for PCI passthru, aka giving guest VM direct PCI access to some physical PCI device (nic, hba, etc) on the host hardware.
Xen can actually do PCI passthru *without* VT-d for PV guests, but for Xen HVM guests you *need* VT-d (if you want to use PCI passthru).
VT-d is NOT required for running HVM/Windows guests.
VT-x is the CPU feature that makes it possible to run unmodified guests. VT-d is the chipset IOMMU feature for PCI passthru.
See: http://wiki.xensource.com/xenwiki/XenPCIpassthrough
-- Pasi
-
Alan Hodgson -
Alexander Dalloz -
Emmanuel Noobadmin -
John R Pierce -
Lamar Owen -
Pasi Kärkkäinen -
Rudi Ahlers