On Thu, 15 Aug 2013, Robert Dinse wrote:
I am getting to where we want to offer virtual serversfor lease but to do so we need some method of measuring and/or limiting traffic to individual guests.
I am wondering what others are using for this purpose?I know that you can look at traffic stats on the bridge on the host machine but that information is lost when the machine is rebooted. I'm wondering if there is any software that databases that information on an ongoing basis and does not lost information across reboots?
At PMman [1], we sample all domU and each dom0, both Xen and KVM based, via libvirt methods every five minutes. We also keep snmp derived data, which are out of scope to your question. We use the virsh 'stat' commands ... nodecpustats nodememstats domblkstat domifstat dommemstat . Then, we stuff detail into a database. The use of the intermediate store of a database provides a 'looser' coupling' than blocking on other methods, so that our control interfaces do not get 'blocked' when things get SNAFU'd
A sample insert looks like this for drive stats on a domU:
//insert stats into db $_dbQuery = "insert into vm_blk_stats set " . "date = now(), " . "vm_name = '".$_vm_name."', " . "vm_server = '".$_vm_server."', " . "vm_running_id = '".$vm_running_ids[$_vm_name]."', " . "device_index = '".$_blk_index."', " . "device = '".$_blkdev."', " . "rd_req = '".$_blk_stats[$_blk_index]['rd_req'] ."', " . "rd_bytes = '".$_blk_stats[$_blk_index]['rd_bytes'] ."', " . "wr_req = '".$_blk_stats[$_blk_index]['wr_req'] ."', " . "wr_bytes = '".$_blk_stats[$_blk_index]['wr_bytes'] ."'"; mysql_query($_dbQuery);
and like this for the VM interfaces:
$_dbQuery = "insert into vm_if_stats set " . "date = now(), " . "vm_name = '".$_vm_name."', " . "vm_server = '".$_vm_server."', " . "vm_running_id = '".$vm_running_ids[$_vm_name]."', " . "device_index = '".$_eth_index."', " . "device = '".$_ethdev."', " . "rx_bytes = '".$_eth_stats[$_eth_index]['rx_bytes'] ."', " . "rx_packets = '".$_eth_stats[$_eth_index]['rx_packets'] ."', " . "rx_errs = '".$_eth_stats[$_eth_index]['rx_errs'] ."', " . "rx_drop = '".$_eth_stats[$_eth_index]['rx_drop'] ."', " . "tx_bytes = '".$_eth_stats[$_eth_index]['tx_bytes'] ."', " . "tx_packets = '".$_eth_stats[$_eth_index]['tx_packets'] ."', " . "tx_errs = '".$_eth_stats[$_eth_index]['tx_errs'] ."', " . "tx_drop = '".$_eth_stats[$_eth_index]['tx_drop'] ."'"; mysql_query($_dbQuery);
The second is an insert for traffic in and out, per interface (which interfaces can 'move around' as to 'name' as to how one queries it out via virsh, as VM's come and go)
We have had several tens of million rows active in those tables over time, but usually 'age them out' when we get north of 20 million into secondary summary tables to keep later query performance reasonable
'domifstat' is useful, because we see circumstances where a VM is seemingly active, but not moving any network traffic in or out (i.e., it has crashed). We use monitoring of traffic stats to detect problems pre-emptively (i.e., before the customer calls). We had an instance of this earlier this week after an attack on an httpd of a client VM, which we identified. I got an external monitoring report,and looked in. On the virsh console, it was reporting OOM problems
Second question, what are the advantaged anddisadvantages of KVM verses Xen? I played with Xen back when I had CentOS 5, but find KVM easier to work with and not much difference in performance.
We run, offer, and support both, both externally and in our developmental labs, but Xen is not the future for people following Red Hat, nothwithstanding the CentOS efforts. Our new development effort is KVM focussed
-- Russ herrold