Hi,
I'm looking for a firewall (preferably on Linux / UNIX) that could automatically block bandwidth abusers as soon as a connection goes over a certain speed, or limit - i.e. either more than say 3Mb/s or 10GB in a giving period (like weekly / monthly).
But, I need it to block the IP to, or where the traffic comes from, or goes to. i.e. a user logs into a web server and upload a LOT of data, then the firewall should block him, but not other people.
Or, someone uploads a small bit of data but downloads a lot of data and then get's blocked. But I need to set thresholds And I should be able to exclude certain IP's / domains from the limits.
Does this make sense?
Can this be done with iptables? If so, how?
If not, what else could I use for this?
A normal DDOS prevention firewall doesn't really work since it only blocks traffic coming in. But I need to limit traffic going out as well.
The servers behind the firewall will serve mail, http, ftp, sql and SSH
On Wed, 2011-08-17 at 21:50 +0200, Rudi Ahlers wrote:
Hi,
I'm looking for a firewall (preferably on Linux / UNIX) that could automatically block bandwidth abusers as soon as a connection goes over a certain speed, or limit - i.e. either more than say 3Mb/s or 10GB in a giving period (like weekly / monthly).
But, I need it to block the IP to, or where the traffic comes from, or goes to. i.e. a user logs into a web server and upload a LOT of data, then the firewall should block him, but not other people.
Or, someone uploads a small bit of data but downloads a lot of data and then get's blocked. But I need to set thresholds And I should be able to exclude certain IP's / domains from the limits.
Does this make sense?
Can this be done with iptables? If so, how?
If not, what else could I use for this?
A normal DDOS prevention firewall doesn't really work since it only blocks traffic coming in. But I need to limit traffic going out as well.
The servers behind the firewall will serve mail, http, ftp, sql and SSH
---- http://tinyurl.com/3n5yn8u
Craig
On 18/08/2011 4:13, Craig White wrote:
On Wed, 2011-08-17 at 21:50 +0200, Rudi Ahlers wrote:
Hi,
I'm looking for a firewall (preferably on Linux / UNIX) that could automatically block bandwidth abusers as soon as a connection goes over a certain speed, or limit - i.e. either more than say 3Mb/s or 10GB in a giving period (like weekly / monthly).
But, I need it to block the IP to, or where the traffic comes from, or goes to. i.e. a user logs into a web server and upload a LOT of data, then the firewall should block him, but not other people.
Or, someone uploads a small bit of data but downloads a lot of data and then get's blocked. But I need to set thresholds And I should be able to exclude certain IP's / domains from the limits.
Does this make sense?
Can this be done with iptables? If so, how?
If not, what else could I use for this?
A normal DDOS prevention firewall doesn't really work since it only blocks traffic coming in. But I need to limit traffic going out as well.
The servers behind the firewall will serve mail, http, ftp, sql and SSH
Would you mind providing the url without using such url shorteners?
Thanks,
Regards
On Thu, 2011-08-18 at 11:36 +0200, Marc Deop i Argemí wrote:
It gives me:-
http://lmgtfy.com/?q=traffic+accounting
which displays a Google search box and an advertisement with nothing about Traffic or about Accounting.
On 08/18/2011 12:06 PM, Always Learning wrote:
On Thu, 2011-08-18 at 11:36 +0200, Marc Deop i Argemí wrote:
It gives me:-
http://lmgtfy.com/?q=traffic+accounting
which displays a Google search box and an advertisement with nothing about Traffic or about Accounting.
Lmgtfy means "let me google that for you". Posting such an url is a pretty standard response to people who ask for help without first making an effort to find some answers (by googling, etc.). The hint is: do your homework first and don't expect spoonfeeding.
Regards, Patrick
On Thu, 2011-08-18 at 19:20 +0200, Patrick Lists wrote:
Lmgtfy means "let me google that for you". Posting such an url is a pretty standard response to people who ask for help without first making an effort to find some answers (by googling, etc.). The hint is: do your homework first and don't expect spoonfeeding.
Thanks Patrick. I do do my own research first, usually via Google or my own technical web pages. I usually get good answers most of the time.
On Thu, Aug 18, 2011 at 7:20 PM, Patrick Lists centos-list@puzzled.xs4all.nl wrote:
On 08/18/2011 12:06 PM, Always Learning wrote:
On Thu, 2011-08-18 at 11:36 +0200, Marc Deop i Argemí wrote:
It gives me:-
http://lmgtfy.com/?q=traffic+accounting
which displays a Google search box and an advertisement with nothing about Traffic or about Accounting.
Lmgtfy means "let me google that for you". Posting such an url is a pretty standard response to people who ask for help without first making an effort to find some answers (by googling, etc.). The hint is: do your homework first and don't expect spoonfeeding.
Regards, Patrick _______________________________________________
And you obviously think I didn't do my homework?
Did you see my specific requirement? Or did you just see "how" and "firewall" and assumed "google" ?
On Thu, 2011-08-18 at 20:45 +0200, Rudi Ahlers wrote:
On Thu, Aug 18, 2011 at 7:20 PM, Patrick Lists centos-list@puzzled.xs4all.nl wrote:
On 08/18/2011 12:06 PM, Always Learning wrote:
On Thu, 2011-08-18 at 11:36 +0200, Marc Deop i Argemí wrote:
It gives me:-
http://lmgtfy.com/?q=traffic+accounting
which displays a Google search box and an advertisement with nothing about Traffic or about Accounting.
Lmgtfy means "let me google that for you". Posting such an url is a pretty standard response to people who ask for help without first making an effort to find some answers (by googling, etc.). The hint is: do your homework first and don't expect spoonfeeding.
Regards, Patrick _______________________________________________
And you obviously think I didn't do my homework?
Did you see my specific requirement? Or did you just see "how" and "firewall" and assumed "google" ?
Perhaps Patrick was doing his own huiswerk and was too busy being spoon fed by his mooi buurvrouwje ;-)
Let's try again:
I need to automatically block any user who abuses bandwidth, either incoming or outgoing. I should be able to set the limits, in either rate/s or usage/s: 1Mb/s or 10GB/h, for example.
Then, any users, connecting from anywhere, on any IP should be blocked - either if he uploads or downloads (i.e ingres & outgres) for a specific amount of time.
My research:
The firewalls which we've tried (both normal Linux iptables and hardware based firewalls) can do this, as long as I can specify the IP's to block - this is standard for an office-type firewall. BUT, I don't have a range of IP's to specify since these particular servers are on the internet, thus any possible IP on the net could connect to the server.
I also need to exclude certain IP's from this rule (i.e. for backup servers which actually need to transfer a lot of traffic).
To some degree this would mean "traffic accounting", but that just keeps a log of traffic usage. And we already measure traffic use with cacti & SNMP. Cacti can send us an email if a certain amount of bandwidth is used up, but it doesn't tell the firewall to block the offending IP address.
DDOS protection type firewalls doesn't help much either since they only block incoming "attacks", but not really normal uploads. They also don't block outgoing traffic once the condition is met.
On Thu, 2011-08-18 at 21:01 +0200, Rudi Ahlers wrote:
I need to automatically block any user who abuses bandwidth, either incoming or outgoing. I should be able to set the limits, in either rate/s or usage/s: 1Mb/s or 10GB/h, for example.
First question is:
(a) how can you get the IP address ?
(b) how can you introduce a, or use an existing, system to record and store the data amounts (bandwidth) and IP addresses ?
(c) how long will this information be retained before being discarded ?
(d) how can you monitor on every change to the data amount ?
(e) will it do both IP4 and IP6 ?
(f) what mechanism can you use to block the IP address ... IP Tables via simple BASH command ?
Its an interesting requirement.
On Thu, Aug 18, 2011 at 9:09 PM, Always Learning centos@u61.u22.net wrote:
On Thu, 2011-08-18 at 21:01 +0200, Rudi Ahlers wrote:
I need to automatically block any user who abuses bandwidth, either incoming or outgoing. I should be able to set the limits, in either rate/s or usage/s: 1Mb/s or 10GB/h, for example.
First question is:
(a) how can you get the IP address ?
I don't fully understand your question? How do you get any IP address from any machine that connects to a server on the internet? netstat shows the IP's, /var/log/http/access.log shows the IP's and I'm sure it's listed in other places as well.
We currently use ntop to monitor the server's usage, but there's no way to automatically block an abusive IP.
(b) how can you introduce a, or use an existing, system to record and store the data amounts (bandwidth) and IP addresses ?
What do you mean?
(c) how long will this information be retained before being discarded ?
How long will what information be retained? And what for? I don't understand the nature of this question?
(d) how can you monitor on every change to the data amount ?
Again, I don't understand what you mean?
(e) will it do both IP4 and IP6 ?
Does it matter? IPV6 is already being used on a wide scale. iptables support both
(f) what mechanism can you use to block the IP address ... IP Tables via simple BASH command ?
if that will do the trick, yes. Any way to block the IP would be fine. iptables would probably be easiest.
Ideally I would like to get a dedicated firewall, or dedicated Linux / UNIX firewall appliance for this purpose as it needs to monitor and protect a whole bunch of servers
Its an interesting requirement.
-- With best regards,
Paul. England, EU.
If there isn't an existing system, or systems you can use together, your only alternative is to create a system to satisfy your requirement. I was speculating on the essentials.
On 8/18/2011 2:15 PM, Rudi Ahlers wrote:
On Thu, Aug 18, 2011 at 9:09 PM, Always Learningcentos@u61.u22.net wrote:
On Thu, 2011-08-18 at 21:01 +0200, Rudi Ahlers wrote:
I need to automatically block any user who abuses bandwidth, either incoming or outgoing. I should be able to set the limits, in either rate/s or usage/s: 1Mb/s or 10GB/h, for example.
First question is:
(a) how can you get the IP address ?
I don't fully understand your question? How do you get any IP address from any machine that connects to a server on the internet? netstat shows the IP's,
You said 'user' which may or may not map to a consistent, single, IP address.
/var/log/http/access.log shows the IP's and I'm sure it's listed in other places as well.
Are these web browser clients, locally attached PCs, or what?
We currently use ntop to monitor the server's usage, but there's no way to automatically block an abusive IP.
What's 'abusive'? If they are using a web app, let the app monitor the connection of a logged in user and handle them appropriately.
Ideally I would like to get a dedicated firewall, or dedicated Linux / UNIX firewall appliance for this purpose as it needs to monitor and protect a whole bunch of servers
A separate box won't know what is going on. Suppose you have a remote mail server relaying in or out for a large number of users. The intermediate box will see a lot of smtp traffic to/from one IP, but it will correspond to a lot of users. Likewise for web users behind a company proxy.
On Thu, Aug 18, 2011 at 9:29 PM, Les Mikesell lesmikesell@gmail.com wrote:
On 8/18/2011 2:15 PM, Rudi Ahlers wrote:
On Thu, Aug 18, 2011 at 9:09 PM, Always Learningcentos@u61.u22.net wrote:
On Thu, 2011-08-18 at 21:01 +0200, Rudi Ahlers wrote:
I need to automatically block any user who abuses bandwidth, either incoming or outgoing. I should be able to set the limits, in either rate/s or usage/s: 1Mb/s or 10GB/h, for example.
First question is:
(a) how can you get the IP address ?
I don't fully understand your question? How do you get any IP address from any machine that connects to a server on the internet? netstat shows the IP's,
You said 'user' which may or may not map to a consistent, single, IP address.
well, a 'user' is anyone accessing the server from the internet, so the IP's will change the whole time.
/var/log/http/access.log shows the IP's and I'm sure it's listed in other places as well.
Are these web browser clients, locally attached PCs, or what?
web / SQL / SMTP / POP3 clients, connecting from the internet.
We currently use ntop to monitor the server's usage, but there's no way to automatically block an abusive IP.
What's 'abusive'? If they are using a web app, let the app monitor the connection of a logged in user and handle them appropriately.
yes, but no monitor can block their IP, that I'm aware of.
Ideally I would like to get a dedicated firewall, or dedicated Linux / UNIX firewall appliance for this purpose as it needs to monitor and protect a whole bunch of servers
A separate box won't know what is going on. Suppose you have a remote mail server relaying in or out for a large number of users. The intermediate box will see a lot of smtp traffic to/from one IP, but it will correspond to a lot of users. Likewise for web users behind a company proxy.
For this very reason I need to exclude certain IP's from the limits.
-- Les Mikesell lesmikesell@gmail.com _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On 8/18/2011 2:01 PM, Rudi Ahlers wrote:
Let's try again:
I need to automatically block any user who abuses bandwidth, either incoming or outgoing. I should be able to set the limits, in either rate/s or usage/s: 1Mb/s or 10GB/h, for example.
Then, any users, connecting from anywhere, on any IP should be blocked
- either if he uploads or downloads (i.e ingres& outgres) for a
specific amount of time.
Those requirements don't mesh very well with the real world. That is, people use use a network that they've been provided or paid for aren't necessarily 'abusing' anything, and blocking access at times when the network isn't fully loaded doesn't help anyone. What's the big picture here? Don't you really need QOS to throttle certain things at peak times only?
On Thu, Aug 18, 2011 at 9:21 PM, Les Mikesell lesmikesell@gmail.com wrote:
On 8/18/2011 2:01 PM, Rudi Ahlers wrote:
Let's try again:
I need to automatically block any user who abuses bandwidth, either incoming or outgoing. I should be able to set the limits, in either rate/s or usage/s: 1Mb/s or 10GB/h, for example.
Then, any users, connecting from anywhere, on any IP should be blocked
- either if he uploads or downloads (i.e ingres& outgres) for a
specific amount of time.
Those requirements don't mesh very well with the real world. That is, people use use a network that they've been provided or paid for aren't necessarily 'abusing' anything, and blocking access at times when the network isn't fully loaded doesn't help anyone. What's the big picture here? Don't you really need QOS to throttle certain things at peak times only?
-- Les Mikesell lesmikesell@gmail.com
Les, it's not really about blocking people who paid.
the servers in question provide a free service and no money is generated from it, but the client still pays for bandwidth so we'd like to cap heavy users a bit to avoid expensive bills.
I know the requirements are strange, but I'm really hoping I could find something that could do this for us. Right now they have someone who monitors ntop and block IP's that way around, but it's inefficient and a salary which could have been spent elsewhere.
Bandwidth in our country is exuberantly expensive, probably about 20x the price of bandwidth in the USA
On 8/18/2011 2:27 PM, Rudi Ahlers wrote:
I need to automatically block any user who abuses bandwidth, either incoming or outgoing. I should be able to set the limits, in either rate/s or usage/s: 1Mb/s or 10GB/h, for example.
Then, any users, connecting from anywhere, on any IP should be blocked
- either if he uploads or downloads (i.e ingres& outgres) for a
specific amount of time.
Those requirements don't mesh very well with the real world. That is, people use use a network that they've been provided or paid for aren't necessarily 'abusing' anything, and blocking access at times when the network isn't fully loaded doesn't help anyone. What's the big picture here? Don't you really need QOS to throttle certain things at peak times only?
Les, it's not really about blocking people who paid.
the servers in question provide a free service and no money is generated from it, but the client still pays for bandwidth so we'd like to cap heavy users a bit to avoid expensive bills.
Are you paying for bandwidth by total bits transferred or by peak or 95th percentile rate?
I know the requirements are strange, but I'm really hoping I could find something that could do this for us. Right now they have someone who monitors ntop and block IP's that way around, but it's inefficient and a salary which could have been spent elsewhere.
You should be able to automate what you are doing with ntop. Or use a netflow collector to centralize the traffic counting and translate your rules into iptables settings.
On Thu, Aug 18, 2011 at 9:38 PM, Les Mikesell lesmikesell@gmail.com wrote:
Are you paying for bandwidth by total bits transferred or by peak or 95th percentile rate?
We pay per MB and the servers are connected to a 100MB/s port.
You should be able to automate what you are doing with ntop. Or use a netflow collector to centralize the traffic counting and translate your rules into iptables settings.
Really? That would be great.
But, I'm not a programmer, so I don't know where to start. And, I need to protect a whole bunch of servers, so ideally this should be done either on a central gateway which connects on the other side of the switch, or a firewall appliance.
Any suggestions?
-- Les Mikesell lesmikesell@gmail.com
On 08/18/11 12:43 PM, Rudi Ahlers wrote:
But, I'm not a programmer, so I don't know where to start.
hire one. your needs and requirements are vague and unique, no off the shelf solution will do exactly what it is you want. you also need to start thinking of your requirements in more precise terms, what the thresholds of traffic that will trigger and reset these blocks or throttles. you probably want to tie this into QoS so that when your algorithm determines that a specific host is over its threshold, you throttle it rather than block it entirely. messy messy messy.
On Thu, 2011-08-18 at 21:27 +0200, Rudi Ahlers wrote:
Bandwidth in our country is exuberantly expensive, probably about 20x the price of bandwidth in the USA
Een oplossing voor Zuid Afrika ?
If your country has good internal Internet connections, host the site in Europe or the USA where bandwidth is a lot cheaper ?
From: Rudi Ahlers Rudi@SoftDux.com
the servers in question provide a free service and no money is generated from it, but the client still pays for bandwidth so we'd like to cap heavy users a bit to avoid expensive bills.
Hum, if it is www traffic, maybe put squid as a reverse proxy and use delay pools?
JD
On Thu, 18 Aug 2011, Rudi Ahlers wrote:
Let's try again:
I need to automatically block any user who abuses bandwidth, either incoming or outgoing. I should be able to set the limits, in either rate/s or usage/s: 1Mb/s or 10GB/h, for example.
Then, any users, connecting from anywhere, on any IP should be blocked
- either if he uploads or downloads (i.e ingres & outgres) for a
specific amount of time.
As one might imagine there is at least one commercial product that seems to fit the bill.
http://www.aspirantinfotech.com/downloads/Cyberoam/pdf/Managing-bandwidth-th...
I mention this as I thought it was well written and thorough. After reading the pdf seems to me there ought to be something open source based upon perhaps this: http://lartc.org/lartc.html
Anyway maybe some food for thought.
On Thu, Aug 18, 2011 at 9:25 PM, Mike mike@microdel.org wrote:
On Thu, 18 Aug 2011, Rudi Ahlers wrote:
Let's try again:
I need to automatically block any user who abuses bandwidth, either incoming or outgoing. I should be able to set the limits, in either rate/s or usage/s: 1Mb/s or 10GB/h, for example.
Then, any users, connecting from anywhere, on any IP should be blocked
- either if he uploads or downloads (i.e ingres & outgres) for a
specific amount of time.
As one might imagine there is at least one commercial product that seems to fit the bill.
http://www.aspirantinfotech.com/downloads/Cyberoam/pdf/Managing-bandwidth-th...
I mention this as I thought it was well written and thorough. After reading the pdf seems to me there ought to be something open source based upon perhaps this: http://lartc.org/lartc.html
Anyway maybe some food for thought. _______________________________________________
Thanx. We already tried the cyberoams, but they didn't work as expected since they manage bandwidth on a per-user basis, and our "users" come from the world-wide-web.
I have read through that document link on http://lartc.org/lartc.html#AEN1393 and the closest I could get is rate limiting, but that doesn't actually block the IP if it goes over a certain threshold, it just slows everything down.
I have read through that document link on http://lartc.org/lartc.html#AEN1393 and the closest I could get is rate limiting, but that doesn't actually block the IP if it goes over a certain threshold, it just slows everything down.
So I'm not sure I fully understand your requirements. Why isn't slowing the user to zero or at least near zero sufficient?
On Thu, Aug 18, 2011 at 9:38 PM, Mike mike@microdel.org wrote:
I have read through that document link on http://lartc.org/lartc.html#AEN1393 and the closest I could get is rate limiting, but that doesn't actually block the IP if it goes over a certain threshold, it just slows everything down.
So I'm not sure I fully understand your requirements. Why isn't slowing the user to zero or at least near zero sufficient?
How do I slow one user down, without affecting the others? The way I understand rate limiting is that you rate limit a certain protocol / port, or IP / IP range.
So, how would I automatically slow down someone (on any IP address, and accessing any protocol) once he hits a certain threshold / limit?
On Thu, 18 Aug 2011, Rudi Ahlers wrote:
On Thu, Aug 18, 2011 at 9:38 PM, Mike mike@microdel.org wrote:
I have read through that document link on http://lartc.org/lartc.html#AEN1393 and the closest I could get is rate limiting, but that doesn't actually block the IP if it goes over a certain threshold, it just slows everything down.
So I'm not sure I fully understand your requirements. Why isn't slowing the user to zero or at least near zero sufficient?
How do I slow one user down, without affecting the others? The way I understand rate limiting is that you rate limit a certain protocol / port, or IP / IP range.
So, how would I automatically slow down someone (on any IP address, and accessing any protocol) once he hits a certain threshold / limit?
I think I understand now and the short answer is that you can't! In other words you're saying that say "Steve" is using a ton of bandwidth so you want to block him. But "Fred" and 10 other users that may be at the same IP address are fine and you don't want to block them. I mean you could conceptually at least block the IP/Source port that "Steve" is "coming from" right now. But the source port (and perhaps IP) will eventually change and your block is now useless.
On Thu, Aug 18, 2011 at 9:52 PM, Mike mike@microdel.org wrote:
On Thu, 18 Aug 2011, Rudi Ahlers wrote:
On Thu, Aug 18, 2011 at 9:38 PM, Mike mike@microdel.org wrote:
I have read through that document link on http://lartc.org/lartc.html#AEN1393 and the closest I could get is rate limiting, but that doesn't actually block the IP if it goes over a certain threshold, it just slows everything down.
So I'm not sure I fully understand your requirements. Why isn't slowing the user to zero or at least near zero sufficient?
How do I slow one user down, without affecting the others? The way I understand rate limiting is that you rate limit a certain protocol / port, or IP / IP range.
So, how would I automatically slow down someone (on any IP address, and accessing any protocol) once he hits a certain threshold / limit?
I think I understand now and the short answer is that you can't! In other words you're saying that say "Steve" is using a ton of bandwidth so you want to block him. But "Fred" and 10 other users that may be at the same IP address are fine and you don't want to block them. I mean you could conceptually at least block the IP/Source port that "Steve" is "coming from" right now. But the source port (and perhaps IP) will eventually change and your block is now useless.
No, not quite.
Steve will have a different IP from Fred. I don't care so much about the users as such, but rather the IP where the connection is from, and to. i.e. I don't need to know what the user's name is, nor match him to a DB like LDAP or something. I purely need to block an abusive IP.
BUT, if Steve changes his IP to circumvent the block, then his new IP should be blocked as well.
On 08/18/11 12:56 PM, Rudi Ahlers wrote:
BUT, if Steve changes his IP to circumvent the block, then his new IP should be blocked as well.
how would you know this?
On 8/18/2011 4:38 PM, John R Pierce wrote:
On 08/18/11 12:56 PM, Rudi Ahlers wrote:
BUT, if Steve changes his IP to circumvent the block, then his new IP should be blocked as well.
how would you know this?
If he is using pop, imap, authenticated smtp, web services with a logged in session, ssh, etc., the applications know the user and may be logging it. But there is nothing central or standard to collate this information, and there are various circumstances that will cause many users to have the same IP source or one user to have several.
On Thu, 2011-08-18 at 21:56 +0200, Rudi Ahlers wrote:
BUT, if Steve changes his IP to circumvent the block, then his new IP should be blocked as well.
How will you know Steve has successfully circumvented your block until until the same Steve, with IP2, eventually exceeds the 'quota' ?
And if Steve gets away with that, he can probably try again with IP3 and IP4 etc. - making a mockery of your bandwidth restriction.
On Fri, Aug 19, 2011 at 12:57 AM, Always Learning centos@u61.u22.net wrote:
On Thu, 2011-08-18 at 21:56 +0200, Rudi Ahlers wrote:
BUT, if Steve changes his IP to circumvent the block, then his new IP should be blocked as well.
How will you know Steve has successfully circumvented your block until until the same Steve, with IP2, eventually exceeds the 'quota' ?
And if Steve gets away with that, he can probably try again with IP3 and IP4 etc. - making a mockery of your bandwidth restriction.
--
The point it, it doesn't matter who the user is. As soon as an IP, any IP exceeds the limit, it should get blocked.
On 08/18/11 4:05 PM, Rudi Ahlers wrote:
The point it, it doesn't matter who the user is. As soon as an IP, any IP exceeds the limit, it should get blocked.
you might take a look at the various fail2ban scripts that are commonly used to block an IP for some period of time after a threshold number of SSH or appache login attempts are made, and you can probably figure out how to implement that same sort of concept to run off whatever per-source-IP traffic statistics you're keeping... of course, if your web and mail and whatever servers are accessed by 100s or 1000s of unique hosts a day, those traffic statistics are going to be quite a lot of overhead to track.
On 08/18/2011 09:31 PM, Rudi Ahlers wrote: [snip]
I have read through that document link on http://lartc.org/lartc.html#AEN1393 and the closest I could get is rate limiting, but that doesn't actually block the IP if it goes over a certain threshold, it just slows everything down.
How about the netfilter quota, fuzzy and iplimit extensions?
http://www.netfilter.org/documentation/HOWTO/netfilter-extensions-HOWTO.html...
http://www.netfilter.org/documentation/HOWTO/netfilter-extensions-HOWTO.html...
http://www.netfilter.org/documentation/HOWTO/netfilter-extensions-HOWTO-3.ht...
Regards, Patrick
Apologies for top posting.
I fear you will either have to work with cacti bandwidth alerts, figuring out how to grab the client IP and push it into iptables; find another way to get the client IP out of cacti and into iptables; or look into the QoS capabilities within Linux.
On 08/18/2011 03:01 PM, Rudi Ahlers wrote:
Let's try again:
I need to automatically block any user who abuses bandwidth, either incoming or outgoing. I should be able to set the limits, in either rate/s or usage/s: 1Mb/s or 10GB/h, for example.
Then, any users, connecting from anywhere, on any IP should be blocked
- either if he uploads or downloads (i.e ingres & outgres) for a
specific amount of time.
My research:
The firewalls which we've tried (both normal Linux iptables and hardware based firewalls) can do this, as long as I can specify the IP's to block - this is standard for an office-type firewall. BUT, I don't have a range of IP's to specify since these particular servers are on the internet, thus any possible IP on the net could connect to the server.
I also need to exclude certain IP's from this rule (i.e. for backup servers which actually need to transfer a lot of traffic).
To some degree this would mean "traffic accounting", but that just keeps a log of traffic usage. And we already measure traffic use with cacti & SNMP. Cacti can send us an email if a certain amount of bandwidth is used up, but it doesn't tell the firewall to block the offending IP address.
DDOS protection type firewalls doesn't help much either since they only block incoming "attacks", but not really normal uploads. They also don't block outgoing traffic once the condition is met.
On 08/18/2011 08:45 PM, Rudi Ahlers wrote:
And you obviously think I didn't do my homework?
Did you see my specific requirement? Or did you just see "how" and "firewall" and assumed "google" ?
I was not referring to you Rudi. Merely pointing out the lmgtfy concept which imho seemed lost on Paul.
And yes I did look at your requirements but don't have the answer for you. Maybe a combination of iptables and tc perhaps with connection tracking thrown in?
Regards, Patrick
On Thu, 2011-08-18 at 21:33 +0200, Patrick Lists wrote:
And yes I did look at your requirements but don't have the answer for you. Maybe a combination of iptables and tc perhaps with connection tracking thrown in?
IP tables would be a good place to link-in; perhaps route requests to a specific port or internal IP address and then examine the traffic before routing it to the correct destination.
On Thu, Aug 18, 2011 at 4:13 AM, Craig White craigwhite@azapple.com wrote:
On Wed, 2011-08-17 at 21:50 +0200, Rudi Ahlers wrote:
Hi,
I'm looking for a firewall (preferably on Linux / UNIX) that could automatically block bandwidth abusers as soon as a connection goes over a certain speed, or limit - i.e. either more than say 3Mb/s or 10GB in a giving period (like weekly / monthly).
But, I need it to block the IP to, or where the traffic comes from, or goes to. i.e. a user logs into a web server and upload a LOT of data, then the firewall should block him, but not other people.
Or, someone uploads a small bit of data but downloads a lot of data and then get's blocked. But I need to set thresholds And I should be able to exclude certain IP's / domains from the limits.
Does this make sense?
Can this be done with iptables? If so, how?
If not, what else could I use for this?
A normal DDOS prevention firewall doesn't really work since it only blocks traffic coming in. But I need to limit traffic going out as well.
The servers behind the firewall will serve mail, http, ftp, sql and SSH
Craig
We already monitor traffic usage on the switches with cacti via SNMP.
But, I need to block traffic abusers automatically. from any IP address, to any IP address.
The firewalls we have, and have tested all need a set of IP addresses to throttle, which won't work in this case. A user can login from any IP address on the internet, and either upload or download exsesively and we need to block that IP address as soon as it's reaches a certain (pre-set by us) threshold
On Aug 17, 2011, at 3:50 PM, Rudi Ahlers Rudi@SoftDux.com wrote:
Hi,
I'm looking for a firewall (preferably on Linux / UNIX) that could automatically block bandwidth abusers as soon as a connection goes over a certain speed, or limit - i.e. either more than say 3Mb/s or 10GB in a giving period (like weekly / monthly).
But, I need it to block the IP to, or where the traffic comes from, or goes to. i.e. a user logs into a web server and upload a LOT of data, then the firewall should block him, but not other people.
Or, someone uploads a small bit of data but downloads a lot of data and then get's blocked. But I need to set thresholds And I should be able to exclude certain IP's / domains from the limits.
Does this make sense?
Can this be done with iptables? If so, how?
If not, what else could I use for this?
A normal DDOS prevention firewall doesn't really work since it only blocks traffic coming in. But I need to limit traffic going out as well.
The servers behind the firewall will serve mail, http, ftp, sql and SSH
Best approach, throttle, you can cause the throttle to increase as the overage increases until it reaches dial-up speed. With some cleverness you can back the throttle out after a period of idle-ness.
-Ross
On 08/17/11 12:50 PM, Rudi Ahlers wrote:
A normal DDOS prevention firewall doesn't really work since it only blocks traffic coming in. But I need to limit traffic going out as well.
The servers behind the firewall will serve mail, http, ftp, sql and SSH
without requests coming in, no web etc traffic can go out.
you want to block your own mail server from sending too much mail to a single host? and block an internet mail server from sending "too much" mail to you? thats not going to end well.
SQL? what are you doing letting a SQL server be publically accessible? SQL servers should only be accessed by application servers over secure connections.
I think as it stands, this is a very poorly thought out idea with much room for gotchas and problems.
-
Always Learning -
Craig White -
John Doe -
John Jasen -
John R Pierce -
Les Mikesell -
Marc Deop i Argemí -
Mike -
Patrick Lists -
Ross Walker -
Rudi Ahlers