There's a whole herd of ways to count and track the amount of data that goes through a given network port. However, I'm looking for a user-friendly (and pretty, if possible) monitor or counter program with the following features:
1. Grand total amount of data in and out since last counter reset. 2. Resettable counters both manual (reset it when you want to) or timed (reset this thing monthly or weekly, etc.) 3. Breakdown of external and internal traffic volume, by gateway if possible.
Basically, I'm looking for a report with fields or a graph similar to this
Total data transferred since the last reset: Total data sent: Total data received: Total data sent through external gateway 192.168.0.1: Total data received through external gateway 192.168.0.1:
And so on if the computer has sent data through other gateways if there are several on a network.
I've found all kinds of counters but none that seem to break out internal traffic from the external traffic by gateway. Perhaps I'm not looking for the right thing?
On Fri, Feb 24, 2012 at 4:40 PM, Frank Cox theatre@melvilletheatre.com wrote:
There's a whole herd of ways to count and track the amount of data that goes through a given network port. However, I'm looking for a user-friendly (and pretty, if possible) monitor or counter program with the following features:
- Grand total amount of data in and out since last counter reset.
- Resettable counters both manual (reset it when you want to) or timed (reset
this thing monthly or weekly, etc.) 3. Breakdown of external and internal traffic volume, by gateway if possible.
Basically, I'm looking for a report with fields or a graph similar to this
Total data transferred since the last reset: Total data sent: Total data received: Total data sent through external gateway 192.168.0.1: Total data received through external gateway 192.168.0.1:
And so on if the computer has sent data through other gateways if there are several on a network.
I've found all kinds of counters but none that seem to break out internal traffic from the external traffic by gateway. Perhaps I'm not looking for the right thing?
I don't think there is anything that tracks things routed through a gateway unless it goes out different interfaces. Do you have different internal/external interfaces? You should be able to pick up the counters from the router interfaces via snmp if you want the totals there.
On Fri, 24 Feb 2012 17:23:31 -0600 Les Mikesell wrote:
I don't think there is anything that tracks things routed through a gateway unless it goes out different interfaces. Do you have different internal/external interfaces? You should be able to pick up the counters from the router interfaces via snmp if you want the totals there.
I have a small network with two cheap dlink routers on it, one attached to a cable modem and the other to a DSL modem. I route certain data through the cable modem, and everything else goes through the DSL.
However, on the computers only have eth0 and I just use route commands on the individual computers to sort out what I want to go out through each modem. My default gateway is 192.168.0.254 and I custom route certain traffic through 192.168.0.1.
Ultimately, I have three kinds of traffic that I'm interested in counting from each computer on my network: internal traffic, from one computer to another (from 192.168.0.x to 192.168.0.y); external traffic through 192.168.0.254; external traffic through 192.168.0.254. I would like to look at a report on each of my computers and find out how much data each one has sent and received during the period in each of those categories.
On Fri, Feb 24, 2012 at 5:34 PM, Frank Cox theatre@melvilletheatre.com wrote:
However, on the computers only have eth0 and I just use route commands on the individual computers to sort out what I want to go out through each modem. My default gateway is 192.168.0.254 and I custom route certain traffic through 192.168.0.1.
Ultimately, I have three kinds of traffic that I'm interested in counting from each computer on my network: internal traffic, from one computer to another (from 192.168.0.x to 192.168.0.y); external traffic through 192.168.0.254; external traffic through 192.168.0.254. I would like to look at a report on each of my computers and find out how much data each one has sent and received during the period in each of those categories.
There is probably some way to do this with iptables, but it doesn't match any normal tracking. You might try ntop and see if you can find the MAC/MAC traffic for the computer/router pairs you want, but there is a lot of overhead for that.
On 02/25/2012 12:34 AM, Frank Cox wrote:
On Fri, 24 Feb 2012 17:23:31 -0600 I have a small network with two cheap dlink routers on it, one attached to a cable modem and the other to a DSL modem. I route certain data through the cable modem, and everything else goes through the DSL.
However, on the computers only have eth0 and I just use route commands on the individual computers to sort out what I want to go out through each modem. My default gateway is 192.168.0.254 and I custom route certain traffic through 192.168.0.1.
Ultimately, I have three kinds of traffic that I'm interested in counting from each computer on my network: internal traffic, from one computer to another (from 192.168.0.x to 192.168.0.y); external traffic through 192.168.0.254; external traffic through 192.168.0.254. I would like to look at a report on each of my computers and find out how much data each one has sent and received during the period in each of those categories.
Can this help? I think friend of mine wrote it, Nenad Opsenica, some 6-7 years ago, I don't think he will mind:
http://www.plcomputers.net/download/svasta/if-kbps
Use it as "if-kbps eth0" and brake with Ctrl+C.
On Sun, 26 Feb 2012 21:11:15 +0100 Ljubomir Ljubojevic wrote:
Can this help? I think friend of mine wrote it, Nenad Opsenica, some 6-7 years ago, I don't think he will mind:
http://www.plcomputers.net/download/svasta/if-kbps
Use it as "if-kbps eth0" and brake with Ctrl+C.
It looks like it does pretty much the same thing as several other monitoring tools that I've looked at. However, none of them separate local traffic from external traffic.
On Sun, Feb 26, 2012 at 2:21 PM, Frank Cox theatre@melvilletheatre.com wrote:
Can this help? I think friend of mine wrote it, Nenad Opsenica, some 6-7 years ago, I don't think he will mind:
http://www.plcomputers.net/download/svasta/if-kbps
Use it as "if-kbps eth0" and brake with Ctrl+C.
It looks like it does pretty much the same thing as several other monitoring tools that I've looked at. However, none of them separate local traffic from external traffic.
Without separate interfaces there's not much to distinguish your local/external concepts. Did you try ntop? It basically acts as a packet sniffer that can group and summarize by a lot of different categories but there is a lot of overhead in building the database with all that info.
On Sun, 26 Feb 2012 14:31:29 -0600 Les Mikesell wrote:
Without separate interfaces there's not much to distinguish your local/external concepts.
As I see it, there's probably some way (that I haven't yet discovered) to separate the traffic by gateway. Or at least some way to distinguish local traffic from external traffic.
Did you try ntop? It basically acts as a packet sniffer that can group and summarize by a lot of different categories but there is a lot of overhead in building the database with all that info.
I'll look at it.
On Sun, Feb 26, 2012 at 3:18 PM, Frank Cox theatre@melvilletheatre.com wrote:
Without separate interfaces there's not much to distinguish your local/external concepts.
As I see it, there's probably some way (that I haven't yet discovered) to separate the traffic by gateway. Or at least some way to distinguish local traffic from external traffic.
There is, but not in the places that normal metrics are gathered. You need to look at the route toward the target IP address for your breakdown. There are tools to do it in iptables, but it is not a common operation so you are probably on your own to set up the matches and read the counters. Someone doing it on a larger scale would probably arrange subnets to isolate the traffic by interfaces or use routers with netflow metrics to track usage by connection.
On Sun, 26 Feb 2012 15:33:08 -0600 Les Mikesell wrote:
There are tools to do it in iptables, but it is not a common operation so you are probably on your own to set up the matches and read the counters.
It's apparently not common, as you say, and I really don't understand why. Folks who buy their bandwith by the mb (thankfully not me) might want to count exactly how much traffic they send out over their satellite or whatever, while omitting the local traffic between their desktop and their laptop. Which is why it surprises me that no tool apparently exists that can do this.
On Sun, Feb 26, 2012 at 3:38 PM, Frank Cox theatre@melvilletheatre.com wrote:
There are tools to do it in iptables, but it is not a common operation so you are probably on your own to set up the matches and read the counters.
It's apparently not common, as you say, and I really don't understand why. Folks who buy their bandwith by the mb (thankfully not me) might want to count exactly how much traffic they send out over their satellite or whatever, while omitting the local traffic between their desktop and their laptop. Which is why it surprises me that no tool apparently exists that can do this.
Most people would just look at the router's own bandwidth measurement or the one at the ISP's end if that is available. I thought what made your case uncommon was that you had multiple machines and multiple routers and wanted the measurements for each pairing even though the packets go over the same interfaces with no inherent separation. If you added interfaces and subnets for each route you wanted to measure separately the normal tools would work naturally.
On Sun, 26 Feb 2012 15:50:32 -0600 Les Mikesell wrote:
Most people would just look at the router's own bandwidth measurement or the one at the ISP's end if that is available.
Possibly, but that wouldn't break it down by machine. And in that situation I'd think a per-machine breakdown would be useful because then you'd know if you should be yelling at the kid, the wife or the family dog when you get the ten thousand dollar ISP bill. Again, it just seems like the sort of thing that folks would want to be able to track in certain situations. But apparently not.
I thought what made your case uncommon was that you had multiple machines and multiple routers and wanted the measurements for each pairing even though the packets go over the same interfaces with no inherent separation.
The separation is the gateway assignment or the lack thereof (for local traffic). But other than that, yep, that's a correct assessment.
If you added interfaces and subnets for each route you wanted to measure separately the normal tools would work naturally.
Indeed, but that adds a whole new layer of complexity to my network that's not really needed for any other purpose.
On Sun, Feb 26, 2012 at 3:55 PM, Frank Cox theatre@melvilletheatre.com wrote:
Most people would just look at the router's own bandwidth measurement or the one at the ISP's end if that is available.
Possibly, but that wouldn't break it down by machine. And in that situation I'd think a per-machine breakdown would be useful because then you'd know if you should be yelling at the kid, the wife or the family dog when you get the ten thousand dollar ISP bill. Again, it just seems like the sort of thing that folks would want to be able to track in certain situations. But apparently not.
Unless you do a lot of local media streaming or network backups, the per-machine usage should be obvious from the interface traffic. And if you actually want to control it, you would force everything through a proxy with user logins - otherwise it is sort of like measuring water usage by how long everyone is in the shower.
I thought what made your case uncommon was that you had multiple machines and multiple routers and wanted the measurements for each pairing even though the packets go over the same interfaces with no inherent separation.
The separation is the gateway assignment or the lack thereof (for local traffic). But other than that, yep, that's a correct assessment.
If you added interfaces and subnets for each route you wanted to measure separately the normal tools would work naturally.
Indeed, but that adds a whole new layer of complexity to my network that's not really needed for any other purpose.
A couple of NICs and a switch aren't all that complicated - but the iptables counters should work. Or you could push one or the other of your routes though a proxy that keeps its own statistics and factor it back out of the relevant interface traffic.
On 02/26/2012 10:55 PM, Frank Cox wrote:
On Sun, 26 Feb 2012 15:50:32 -0600 Les Mikesell wrote:
Most people would just look at the router's own bandwidth measurement or the one at the ISP's end if that is available.
Possibly, but that wouldn't break it down by machine. And in that situation I'd think a per-machine breakdown would be useful because then you'd know if you should be yelling at the kid, the wife or the family dog when you get the ten thousand dollar ISP bill. Again, it just seems like the sort of thing that folks would want to be able to track in certain situations. But apparently not.
I thought what made your case uncommon was that you had multiple machines and multiple routers and wanted the measurements for each pairing even though the packets go over the same interfaces with no inherent separation.
The separation is the gateway assignment or the lack thereof (for local traffic). But other than that, yep, that's a correct assessment.
If you added interfaces and subnets for each route you wanted to measure separately the normal tools would work naturally.
Indeed, but that adds a whole new layer of complexity to my network that's not really needed for any other purpose.
Then use "tc" for bandwidth control ( per source IP ) with pipes much larger then your bandwidth, so you are not limiting, but get only reports of the usage per source (local) IP.
On 02/26/12 12:21 PM, Frank Cox wrote:
separate local traffic from external traffic.
use some iptables rules to select the specific traffic you want to monitor, and view the packet and byte counts via iptables -L -vn
On Sun, 26 Feb 2012 12:34:34 -0800 John R Pierce wrote:
use some iptables rules to select the specific traffic you want to monitor, and view the packet and byte counts via iptables -L -vn
Interesting. I'll look at that.
On 02/26/12 12:34 PM, John R Pierce wrote:
use some iptables rules to select the specific traffic you want to monitor, and view the packet and byte counts via iptables -L -vn
to expand on this, I'd probably write a perl or python script that ran the iptables -vnL command, and parses the output, and store the values in a file or sql database along with a timestamp. If all your iptables sampling rules are in the INPUT table, you can restrict the output with `iptables -vnL INPUT`... this script could feed the data to cacti or another rrdtool/mrtg style monitoring system.
.
On 02/26/2012 09:21 PM, Frank Cox wrote:
On Sun, 26 Feb 2012 21:11:15 +0100 Ljubomir Ljubojevic wrote:
Can this help? I think friend of mine wrote it, Nenad Opsenica, some 6-7 years ago, I don't think he will mind:
http://www.plcomputers.net/download/svasta/if-kbps
Use it as "if-kbps eth0" and brake with Ctrl+C.
It looks like it does pretty much the same thing as several other monitoring tools that I've looked at. However, none of them separate local traffic from external traffic.
Check out Shorewall, it has some fancy features, including firewall/router with one interface, but do not know about this one.
On Sun, Feb 26, 2012 at 02:21:09PM -0600, Frank Cox wrote:
It looks like it does pretty much the same thing as several other monitoring tools that I've looked at. However, none of them separate local traffic from external traffic.
check out http://bandwidthd.sourceforge.net/ - It only supports IPv4, but it's pretty convenient, as you can define what 'local' and 'external' means by IP address.
On Mon, 27 Feb 2012 00:26:59 -0500 Luke S. Crawford wrote:
check out http://bandwidthd.sourceforge.net/ - It only supports IPv4, but it's pretty convenient, as you can define what 'local' and 'external' means by IP address.
Cool! That looks like it could the the real McCoy.
Now I'm off to play with this toy....
On Sun, Feb 26, 2012 at 11:30:14PM -0600, Frank Cox wrote:
On Mon, 27 Feb 2012 00:26:59 -0500 Luke S. Crawford wrote:
check out http://bandwidthd.sourceforge.net/ - It only supports IPv4, but it's pretty convenient, as you can define what 'local' and 'external' means by IP address.
Cool! That looks like it could the the real McCoy.
Now I'm off to play with this toy....
It's pretty cool; I used it for billing for a while. The big problem is that it doesn't support IPv6, and that's pretty much essential these days, I mean, to your more technically savvy customers.
-
Frank Cox -
John R Pierce -
Les Mikesell -
Ljubomir Ljubojevic -
Luke S. Crawford