On Mon, Mar 01, 2010 at 09:34:45PM -0600, Christopher G. Stach II wrote: > ----- "Dennis J." <dennisml at conversis.de> wrote: > > > What I'm aiming for as a starting point is a 3-4 host cluster with > > about 10 VMs on each host and a 2 system DRBD based cluster as a > > redundant storage backend. > > That's a good idea. > > > The question that bugs me is how I can get enough bandwidth between the > > hosts and the storage to provide the VMs with reasonable I/O > > performance. > > You may also want to investigate whether or not a criss-cross replication setup (1A->2a, 2B->1b) is worth the complexity to you. That will spread the load across two drbd hosts and give you approximately the same fault tolerance at a slightly higher risk. (This is assuming that risk-performance tradeoff is important enough to your project.) > > > If all the 40 VMs start copying files at the same time that would mean > > that the bandwidth share for each VM would be tiny. > > Would they? It's a possibility, and fun to think about, but what are the chances? You will usually run into this with backups, cron, and other scheduled [non-business load] tasks. These are far cheaper to fix with manually adjusting schedules than any other way, unless you are rolling in dough. > > > Would I maybe get away with 4 bonded gbit ethernet ports? Would I > > require fiber channel or 10gbit infrastructure? > > Fuck FC, unless you want to get some out of date, used, gently broken, or no-name stuff, or at least until FCoE comes out. (You're probably better off getting unmanaged IB switches and using iSER.) > > Can't say if 10GbE would even be enough, but it's probably overkill. 10 Gbit Ethernet makes sense if you need over 110MB/sec throughput with sequential reads/writes with large block sizes.. that's what 1 Gbit ethernet can give you. If we're talking about random IO, then 1 Gbit ethernet is good/enough for many environments. Disks are the bottleneck with random IO. -- Pasi