I have a remote CentOS-5.3 installation that shares traffic with another enterprise. I want to record and measure the actual network traffic that comes into and goes out of that host's net i/f on a monthly basis for billing verification purposes. Does CentOS have a software package to accomplish this?
I have run across references to a package called vnstat as a possible answer to my requirements, but I cannot seem to locate a CentOS rpm package for it.
Regards,
James B. Byrne wrote:
I have a remote CentOS-5.3 installation that shares traffic with another enterprise. I want to record and measure the actual network traffic that comes into and goes out of that host's net i/f on a monthly basis for billing verification purposes. Does CentOS have a software package to accomplish this?
I have run across references to a package called vnstat as a possible answer to my requirements, but I cannot seem to locate a CentOS rpm package for it.
There are several tools that will collect interface traffic data via SNMP and record it so you can graph, show high/low/average values over a time span, etc. Cacti (in the epel repo) is probably the easiest to set up, OpenNMS (http://www.opennms.org) probably the most complete. These could also get their data from a port on a managed switch or router if that makes it easier to show the connections you need to split out.
Les Mikesell wrote:
There are several tools that will collect interface traffic data via SNMP and record it so you can graph, show high/low/average values over a time span, etc. Cacti (in the epel repo) is probably the easiest to set up, OpenNMS (http://www.opennms.org) probably the most complete. These could also get their data from a port on a managed switch or router if that makes it easier to show the connections you need to split out.
One thing to note for billing, often times bandwidth is billed on a 95th percentile level, and cacti is not good for that if you want accuracy.
We use RTG(in my research last year it seemed RTG was the most frequently mentioned tool that was best for this purpose) to measure our main pipes for billing comparison purposes, matches much closer to what the ISPs say, and cacti is quite a bit off. I wouldn't rely on RTG for normal network monitoring(UI isn't that good etc), but for links where billing information is important at least for 95th percentile, don't rely on cacti alone.
Note RTG is not the same as MRTG, though I think I recall seeing RTG was inspired by MRTG.
Not sure how OpenNMS handles that sort of thing.
Not to knock cacti, I use it extensively, currently have a server collecting more than 20 million points of data a day.
nate
nate wrote:
There are several tools that will collect interface traffic data via SNMP and record it so you can graph, show high/low/average values over a time span, etc. Cacti (in the epel repo) is probably the easiest to set up, OpenNMS (http://www.opennms.org) probably the most complete. These could also get their data from a port on a managed switch or router if that makes it easier to show the connections you need to split out.
One thing to note for billing, often times bandwidth is billed on a 95th percentile level, and cacti is not good for that if you want accuracy.
Yes, sometimes you pay for a fixed pipe and sometimes you have burstable capacity where you pay for what you use. Cacti and Opennms both use a data storage format where the samples are stored at their full resolution for some time interval, then averaged into longer aggregates as they age to keep the file size down while still keeping a long history. If you use the rrd or jrobin tools to compute min/max/average/percent values over a time range, I believe they normalize the samples to the worst that applies to any part of the range. That is, if you want samples as collected, you must restrict the range of the request to the time span before aggregation happens. The default in opennms is 2 weeks - I'm not sure about cacti.
We use RTG(in my research last year it seemed RTG was the most frequently mentioned tool that was best for this purpose) to measure our main pipes for billing comparison purposes, matches much closer to what the ISPs say, and cacti is quite a bit off. I wouldn't rely on RTG for normal network monitoring(UI isn't that good etc), but for links where billing information is important at least for 95th percentile, don't rely on cacti alone.
Note RTG is not the same as MRTG, though I think I recall seeing RTG was inspired by MRTG.
Not sure how OpenNMS handles that sort of thing.
Pretty much the same, but it defaults to using a java re-implementation of the rrd tools called jrobin. I don't think there is a way to show percentiles in the stock opennms graphs but the jrobin class has the low level methods and the web site shows a way to do it in a groovy (that's the language, not my impression) script.
Not to knock cacti, I use it extensively, currently have a server collecting more than 20 million points of data a day.
Isp's would likely use either a 30-day or actual month of 5-minute samples, computing a 95th percentile by discarding the highest 5 percent of the samples and picking the highest remaining value. To match that, you'd either have to adjust the rrd/jrobin storage formats to retain full-resolution samples that long or extract the individual sample values before they are aggregated and process them some other way.
If anyone has an easier way, please let me know - I need to do this myself for a few connections.
On Wed, Jul 15, 2009 at 5:53 PM, natecentos@linuxpowered.net wrote:
Les Mikesell wrote:
There are several tools that will collect interface traffic data via SNMP and record it so you can graph, show high/low/average values over a time span, etc. Cacti (in the epel repo) is probably the easiest to set up, OpenNMS (http://www.opennms.org) probably the most complete. These could also get their data from a port on a managed switch or router if that makes it easier to show the connections you need to split out.
One thing to note for billing, often times bandwidth is billed on a 95th percentile level, and cacti is not good for that if you want accuracy.
We use RTG(in my research last year it seemed RTG was the most frequently mentioned tool that was best for this purpose) to measure our main pipes for billing comparison purposes, matches much closer to what the ISPs say, and cacti is quite a bit off. I wouldn't rely on RTG for normal network monitoring(UI isn't that good etc), but for links where billing information is important at least for 95th percentile, don't rely on cacti alone.
Note RTG is not the same as MRTG, though I think I recall seeing RTG was inspired by MRTG.
Not sure how OpenNMS handles that sort of thing.
Not to knock cacti, I use it extensively, currently have a server collecting more than 20 million points of data a day.
nate
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
+1 for RTG
I discover that mrtg is available as a package for CentOS-5. At first blush this seems the answer but would it in fact suffice for my purposes: measuring the total data volume passing over eth0 on the CentOS based host running mrtg?
Is there a how-to guide to configuring snmp for eth0 on CentOS? I am afraid this this is all new territory for me and I am working under serious time and resource constraints. I have to get this done quickly and cheaply and I need a recipe to monitor total eth0 traffic cumulatively over a period of months. Getting the niceties of snmp mastered and in place must wait til a later date.
Any pertinant advice is welcomed.
I have snmpd and mrtg running and reporting against my Cisco router. What I want to do is to configure snmp so that I can monitor network traffic across the host's own eth0 NIC. Is this even possible for a generic NIC running on a x86_64 or i686 host?
On Fri, Jul 17, 2009 at 12:07 AM, James B. Byrnebyrnejb@harte-lyne.ca wrote:
I have snmpd and mrtg running and reporting against my Cisco router. What I want to do is to configure snmp so that I can monitor network traffic across the host's own eth0 NIC. Is this even possible for a generic NIC running on a x86_64 or i686 host?
Shouldn't be a problem since I was monitoring my server's own NIC traffic and load with MRTG before it stopped working. If I'm not mistaken, it's a matter of configuring snmp to check localhost in addition to your router's IP.
On: Wed, 15 Jul 2009 16:22:51 -0400, Filipe Brandenburger filbranden@gmail.com wrote:
Have a look at "netacct-mysql": http://sourceforge.net/projects/netacct-mysql/
It uses pcap which puts your network card in promiscuous mode and send all packets to the running daemon, which might be a little heavyweight on the resources if your server uses a lot of network, but it appears to be very flexible in terms of being able to split accounting for different traffic going through the server.
If you want something simpler than that, you may:
...
HTH, Filipe
I get the digest version of the list and so my relay is a tad late. Thank you for this. I am rapidly discovering that snmp and mrtg may not really suit my purposes as the NIC seemingly lacks the requisite smarts. Your suggestions are most helpful. Thanks.
I learned a great deal about Linux based automated system and network monitoring these past 24 hours. However, when I went to implement an iptables solution per the suggestion of Filipe Brandenburger, I discovered that Webmin already has a script for this setup and ready to go. So, I have enabled that feature on the eth0 port of the remote host and this seems to give me what I need, for now anyway.
Many thanks for all the suggestions and references. Those that I have not already looked into and read I will keep against a future need.
Regards,
Hi,
On Wed, Jul 15, 2009 at 11:46, James B. Byrnebyrnejb@harte-lyne.ca wrote:
I have a remote CentOS-5.3 installation that shares traffic with another enterprise. I want to record and measure the actual network traffic that comes into and goes out of that host's net i/f on a monthly basis for billing verification purposes. Does CentOS have a software package to accomplish this?
Have a look at "netacct-mysql": http://sourceforge.net/projects/netacct-mysql/
It uses pcap which puts your network card in promiscuous mode and send all packets to the running daemon, which might be a little heavyweight on the resources if your server uses a lot of network, but it appears to be very flexible in terms of being able to split accounting for different traffic going through the server.
If you want something simpler than that, you may: 1) Use "ifconfig" and save the amount of bytes received/transmitted, then do that again some time from now and subtract the difference to know how much you sent in that period. A simple script should be able to do that. Beware that that counter may wrap though. Also, you might lose the information if you reboot the server or restart the network. 2) Use "iptables" to set specific rules for what you want to account. You don't have to use "allow" or "reject" or "drop", just write the filter you want. Then use "iptables -v -L" which will show you the number of packets and bytes that went through that rule. You can then use "iptables -Z" to reset the counters for a new period.
HTH, Filipe
-
Filipe Brandenburger -
James B. Byrne -
Les Mikesell -
Lucian＠lastdot.org -
nate -
Noob Centos Admin