Hello,
My home system on a DSL line is getting worn out by bad behavior robots.
Awhile back, I created a .htaccess file that block countries by IP blocks. Its 2MB in size.
I have been running Linux since Slackware 1.0 and moved to Redhat around 2.0. I started after running a BBS using a doorway for newsgroups. Been hooked ever since.
So, today, I tried following the directions for apache.org website, https://httpd.apache.org/docs/current/howto/htaccess.html to move the .htaccess https://httpd.apache.org/docs/current/howto/htaccess.html%20to%20move%20the %20.htaccess file to a file located in a directory /var/www/htdocs.
I'm just not following or understanding. The .htaccess file works but on a slow DSL, I don't want the hits.
I added the following to my httpd.conf:
<Directory "/var/www/htdocs">
AddType text/htdocs ".txt"
</Directory>
And copied my .htaccess to /var/www/htdocs as htaccess.txt
In the example from the apache website, I don't get the: AddType text/example ".exm" Where did they come up .exm?
TIA
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of TE Dukes Sent: Sunday, August 28, 2016 10:36 AM
My home system on a DSL line is getting worn out by bad behavior robots.
Awhile back, I created a .htaccess file that block countries by IP blocks. Its 2MB in size.
...
So, today, I tried following the directions for apache.org website, https://httpd.apache.org/docs/current/howto/htaccess.html to move the .htaccess
What version of CentOS are you using?
For 7.x, and I think 6.x, there is a much simpler way of doing this, using mod_geoip from the Epel repository.
It rejects all unwanted HTTP connections using 403 responses. Here's an example geoip.conf file, which is what I'm using:
---- <IfModule mod_geoip.c> GeoIPEnable On GeoIPDBFile /usr/share/GeoIP/GeoIP.dat MemoryCache GeoIPOutput Env # Proxies SetEnvIf GEOIP_COUNTRY_CODE A1 BlockCountry #Country blocks SetEnvIf GEOIP_COUNTRY_CODE TR BlockCountry SetEnvIf GEOIP_COUNTRY_CODE IR BlockCountry #... more countries using the two char country code </IfModule> ---- On C7 this file goes here /etc/httpd/conf.d/geoip.conf
Make sure that /etc/httpd/conf.modules.d/10-geoip.conf loads the library file, and is not remarked out with a #.
There is more info on mod_geoip here (but use the installation from Epel) though:
http://dev.maxmind.com/geoip/legacy/mod_geoip2/
I run a server for personal family purposes, and use this to block many of the places my family doesn't live...
Al McCann Certe, Toto, sentio nos in Kansate non iam adesse.
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Albert McCann Sent: Sunday, August 28, 2016 2:01 PM To: 'CentOS mailing list' Subject: Re: [CentOS] .htaccess file
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org]
On
Behalf Of TE Dukes Sent: Sunday, August 28, 2016 10:36 AM
My home system on a DSL line is getting worn out by bad behavior robots.
Awhile back, I created a .htaccess file that block countries by IP
blocks.
Its 2MB in size.
...
So, today, I tried following the directions for apache.org website, https://httpd.apache.org/docs/current/howto/htaccess.html to move the .htaccess
What version of CentOS are you using?
For 7.x, and I think 6.x, there is a much simpler way of doing this, using mod_geoip from the Epel repository.
It rejects all unwanted HTTP connections using 403 responses. Here's an example geoip.conf file, which is what I'm using:
<IfModule mod_geoip.c> GeoIPEnable On GeoIPDBFile /usr/share/GeoIP/GeoIP.dat MemoryCache GeoIPOutput Env # Proxies SetEnvIf GEOIP_COUNTRY_CODE A1 BlockCountry #Country blocks SetEnvIf GEOIP_COUNTRY_CODE TR BlockCountry SetEnvIf GEOIP_COUNTRY_CODE IR BlockCountry #... more countries using the two char country code </IfModule> ---- On C7 this file goes here /etc/httpd/conf.d/geoip.conf
Make sure that /etc/httpd/conf.modules.d/10-geoip.conf loads the library file, and is not remarked out with a #.
There is more info on mod_geoip here (but use the installation from Epel) though:
http://dev.maxmind.com/geoip/legacy/mod_geoip2/
I run a server for personal family purposes, and use this to block many of
the
places my family doesn't live...
Al McCann Certe, Toto, sentio nos in Kansate non iam adesse.
[Thomas E Dukes] Thanks,
I'll take a look at that as well. I am getting hit on several services but httpd is getting the majority.
Thanks!!
On 2016-08-28, TE Dukes tdukes@palmettoshopper.com wrote:
I'm just not following or understanding. The .htaccess file works but on a slow DSL, I don't want the hits.
What exactly is slow when you receive requests from remote clients that you don't want? Are you actually seeing problems when clients make requests and Apache has to read in your 2MB .htaccess on every request? And if so, you might also consider moving your blocking even higher, to iptables rules, so that Apache never even has to deal with them.
I added the following to my httpd.conf:
<Directory "/var/www/htdocs"> AddType text/htdocs ".txt"
</Directory>
And copied my .htaccess to /var/www/htdocs as htaccess.txt
Where did you get the idea that this is how to do global Apache configuration? This won't actually do anything useful.
In the example from the apache website, I don't get the: AddType text/example ".exm" Where did they come up .exm?
They made it up as an example, to demonstrate how directives work in .htaccess files versus global Apache config files. It's not meant to demonstrate how to add blocking rules to the global config.
Here's the main point of that page:
"Any directive that you can include in a .htaccess file is better set in a Directory block, as it will have the same effect with better performance."
So, to achieve what I think you're hoping, take all the IPs you're denying in your .htaccess file, put them into a relevant Directory block in a config file under /etc/httpd, reload Apache, and move your .htaccess file out of the way. Then httpd will no longer have to read in .htaccess for every HTTP request.
Or, alternatively, block those IPs using iptables instead. However, clients will still be able to make those requests, and that will still use bandwidth on your DSL. The only way to eliminate that altogether is to block those requests on the other side of your link. That's something you'd have to work out with your ISP, but I don't think it's common for ISPs to put up blocking rules solely for this purpose, or to allow home users to configure such blocks themselves.
--keith
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Keith Keller Sent: Sunday, August 28, 2016 4:23 PM To: centos@centos.org Subject: Re: [CentOS] .htaccess file
On 2016-08-28, TE Dukes tdukes@palmettoshopper.com wrote:
I'm just not following or understanding. The .htaccess file works but on a slow DSL, I don't want the hits.
What exactly is slow when you receive requests from remote clients that
you
don't want? Are you actually seeing problems when clients make requests and Apache has to read in your 2MB .htaccess on every request? And if so, you might also consider moving your blocking even higher, to iptables rules, so that Apache never even has to deal with them.
I added the following to my httpd.conf:
<Directory "/var/www/htdocs"> AddType text/htdocs ".txt"
</Directory>
And copied my .htaccess to /var/www/htdocs as htaccess.txt
Where did you get the idea that this is how to do global Apache configuration? This won't actually do anything useful.
In the example from the apache website, I don't get the: AddType text/example ".exm" Where did they come up .exm?
They made it up as an example, to demonstrate how directives work in .htaccess files versus global Apache config files. It's not meant to demonstrate how to add blocking rules to the global config.
Here's the main point of that page:
"Any directive that you can include in a .htaccess file is better set in a Directory block, as it will have the same effect with better performance."
So, to achieve what I think you're hoping, take all the IPs you're denying
in
your .htaccess file, put them into a relevant Directory block in a config
file
under /etc/httpd, reload Apache, and move your .htaccess file out of the way. Then httpd will no longer have to read in .htaccess for every HTTP request.
Or, alternatively, block those IPs using iptables instead. However,
clients will
still be able to make those requests, and that will still use bandwidth on
your
DSL. The only way to eliminate that altogether is to block those requests
on
the other side of your link. That's something you'd have to work out with your ISP, but I don't think it's common for ISPs to put up blocking rules
solely
for this purpose, or to allow home users to configure such blocks
themselves.
--keith
[Thomas E Dukes] I setup an ipset but quickly ran out of room in the set. I guess I'll have to setup multiple sets. Right now, I'm just trying to take some load off my home server from badbots but I am getting hit on other services as well.
There's nothing on the webserver except a test site I use. Just trying to keep out the ones that ignore robots.txt
Thanks!!
There's nothing on the webserver except a test site I use. Just trying to keep out the ones that ignore robots.txt
If its just a test server, then I'd be tempted to use HTTP AUTH at the top level. Most robots will be blocked by that, and you can use iptables to block the ones that try to guess your password, perhaps with fail2ban.
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Kahlil Hodgson Sent: Sunday, August 28, 2016 6:42 PM To: CentOS mailing list Subject: Re: [CentOS] .htaccess file
There's nothing on the webserver except a test site I use. Just trying to keep out the ones that ignore robots.txt
If its just a test server, then I'd be tempted to use HTTP AUTH at the top level. Most robots will be blocked by that, and you can use iptables to
block
the ones that try to guess your password, perhaps with fail2ban.
--
[Thomas E Dukes] I have thought about that as well.
I do have fail2ban installed as wells as denyhosts.
Thanks!!
On 2016-08-28, TE Dukes tdukes@palmettoshopper.com wrote:
I setup an ipset but quickly ran out of room in the set. I guess I'll have to setup multiple sets.
I'm not familiar with ipsets, but from a quick Google search it seems like you can increase the size of an ipset (or make a new larger one and migrate your IPs to the new one). Multiple sets looks like it'd work as well.
Right now, I'm just trying to take some load off my home server from badbots but I am getting hit on other services as well.
Another possibility for you to look at is sshguard. It can protect against brute force ssh attacks (using iptables rules, which is how I use it) but IIRC it can also protect against http attacks (I've never used it that way, so I don't know how difficult this is).
Can you be more specific about the "load" you're trying to mitigate? Is it really the load on your home system, or is it that attackers are using your bandwidth, or a combination?
--keith
On Sun, Aug 28, 2016 at 5:23 PM, Keith Keller kkeller@wombat.san-francisco.ca.us wrote:
On 2016-08-28, TE Dukes tdukes@palmettoshopper.com wrote:
Right now, I'm just trying to take some load off my home server from badbots but I am getting hit on other services as well.
Another possibility for you to look at is sshguard. It can protect against brute force ssh attacks (using iptables rules, which is how I use it) but IIRC it can also protect against http attacks (I've never used it that way, so I don't know how difficult this is).
I use fail2ban, provides similar functionality like sshguard + Apache mod_evasive (for http DoS attacks).
-- Arun Khan
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Arun Khan Sent: Monday, August 29, 2016 12:48 AM To: CentOS mailing list Subject: Re: [CentOS] .htaccess file
On Sun, Aug 28, 2016 at 5:23 PM, Keith Keller <kkeller@wombat.san- francisco.ca.us> wrote:
On 2016-08-28, TE Dukes tdukes@palmettoshopper.com wrote:
Right now, I'm just trying to take some load off my home server from badbots but I am getting hit on other services as well.
Another possibility for you to look at is sshguard. It can protect against brute force ssh attacks (using iptables rules, which is how I use it) but IIRC it can also protect against http attacks (I've never used it that way, so I don't know how difficult this is).
I use fail2ban, provides similar functionality like sshguard + Apache mod_evasive (for http DoS attacks).
[Thomas E Dukes] I use it as well with denyhosts.
Its on like donkey kong with these badbots!! LOL!!
Thanks!!
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Keith Keller Sent: Sunday, August 28, 2016 8:23 PM To: centos@centos.org Subject: Re: [CentOS] .htaccess file
On 2016-08-28, TE Dukes tdukes@palmettoshopper.com wrote:
I setup an ipset but quickly ran out of room in the set. I guess I'll have to setup multiple sets.
I'm not familiar with ipsets, but from a quick Google search it seems like
you
can increase the size of an ipset (or make a new larger one and migrate
your
IPs to the new one). Multiple sets looks like it'd work as well.
Right now, I'm just trying to take some load off my home server from badbots but I am getting hit on other services as well.
Another possibility for you to look at is sshguard. It can protect
against brute
force ssh attacks (using iptables rules, which is how I use it) but IIRC
it can
also protect against http attacks (I've never used it that way, so I don't
know
how difficult this is).
Can you be more specific about the "load" you're trying to mitigate? Is
it
really the load on your home system, or is it that attackers are using
your
bandwidth, or a combination?
--keith
[Thomas E Dukes] I saw that as well but it was a little vague on how to do that.
Thanks!!
On 2016-08-29, TE Dukes tdukes@palmettoshopper.com wrote:
Can you be more specific about the "load" you're trying to mitigate? Is it really the load on your home system, or is it that attackers are using your bandwidth, or a combination?
[Thomas E Dukes] I saw that as well but it was a little vague on how to do that.
There are two easy (though not quantitative) tests you can do.
First, look at the load on the server. If httpd is using a lot of CPU and putting your load over 1, your main issue is probably the load being generated by .htaccess reads.
If you have another system on your home network, try a speed test. If it performs crappy you probably have a problem with attackers eating your bandwidth.
You and another poster mentioned fail2ban; if you can get that configured to watch and protect both sshd and httpd that will help both problems quite a bit.
--keith
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Keith Keller Sent: Monday, August 29, 2016 7:39 PM To: centos@centos.org Subject: Re: [CentOS] .htaccess file
On 2016-08-29, TE Dukes tdukes@palmettoshopper.com wrote:
Can you be more specific about the "load" you're trying to mitigate? Is it really the load on your home system, or is it that attackers are using your bandwidth, or a combination?
[Thomas E Dukes] I saw that as well but it was a little vague on how to do that.
There are two easy (though not quantitative) tests you can do.
First, look at the load on the server. If httpd is using a lot of CPU and
putting
your load over 1, your main issue is probably the load being generated by .htaccess reads.
[Thomas E Dukes] Its not necessarily the load on my server, but the bandwidth on my dsl.
If you have another system on your home network, try a speed test. If it performs crappy you probably have a problem with attackers eating your bandwidth.
[Thomas E Dukes] I have a fire stick on my network that I stream movies. Getting beatup by badbots isn't helping.
You and another poster mentioned fail2ban; if you can get that configured
to
watch and protect both sshd and httpd that will help both problems quite a bit.
[Thomas E Dukes] I have all the jails setup for the services I'm running. Not sure its working. Not getting any emails.
Thanks!!
On 2016-08-30, TE Dukes tdukes@palmettoshopper.com wrote:
You and another poster mentioned fail2ban; if you can get that configured
to
watch and protect both sshd and httpd that will help both problems quite a bit.
I have all the jails setup for the services I'm running. Not sure its working. Not getting any emails.
Check your logs. fail2ban probably keeps a log of what it's doing, and you can also check the appropriate fail2ban targets (either iptables, /etc/hosts.deny, the Apache config file) to see if they are being populated. You certainly should see something; if you don't it's a likely misconfiguration.
--keith
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Keith Keller Sent: Monday, August 29, 2016 9:33 PM To: centos@centos.org Subject: Re: [CentOS] .htaccess file
On 2016-08-30, TE Dukes tdukes@palmettoshopper.com wrote:
You and another poster mentioned fail2ban; if you can get that configured
to
watch and protect both sshd and httpd that will help both problems quite a bit.
I have all the jails setup for the services I'm running. Not sure its working. Not getting any emails.
Check your logs. fail2ban probably keeps a log of what it's doing, and
you can
also check the appropriate fail2ban targets (either iptables,
/etc/hosts.deny,
the Apache config file) to see if they are being populated. You certainly should see something; if you don't it's a likely misconfiguration.
--keith
[Thomas E Dukes]
I did change the MTA from sendmail to mail since centos uses postfix.
I may need to change that back.
Thanks!!
Hi,
My home system on a DSL line is getting worn out by bad behavior robots.
Awhile back, I created a .htaccess file that block countries by IP blocks. Its 2MB in size.
Do you control your home server ? If so, then .htaccess is the wrong solution, because you need to incorporate blockages in your IP Tables firewall and then use your Apache configuration file to restrict any remaining unwanted visitors.
.htaccess (its possible in Apache to rename it) is inefficient and suitable as a second-rate solution when you are using a hosted service and lack full control of the server. VPSs are cheap and a better alternative to hosted mail and web.
On my servers (C5 and C6) in IP Tables, I have three sets of blockages:
* permanent for all ports * only for web (port 80) * only for emails (port 25)
In web and emails there is a permanent table plus a monthly one (one for every month). Perpetual pests go in the permanent tables and irritants in the monthly table - otherwise the banned IPs entries would get too large.
A compromised computer trying to send me junk mail or trying to wrongly access a web page or attempting to break-in to SQL (instantly identified and IP instantly blocked because I impose string size limits for the ?key=....) has its IP added to the monthly list and remains there until one month after the last access from that IP address.
I am unwilling to be a passive victim of junk mail and web hackers.
All home-made solutions but effective and robust. Centos made all this possible (sincere thanks to the C-Team; they are all 'A*' rated).
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Always Learning Sent: Monday, August 29, 2016 1:50 PM To: CentOS mailing list Subject: Re: [CentOS] .htaccess file
Hi,
My home system on a DSL line is getting worn out by bad behavior robots.
Awhile back, I created a .htaccess file that block countries by IP
blocks.
Its 2MB in size.
Do you control your home server ? If so, then .htaccess is the wrong solution, because you need to incorporate blockages in your IP Tables firewall and then use your Apache configuration file to restrict any
remaining
unwanted visitors.
[Thomas E Dukes]
Yes. I knew .htaccess wasn't the best method. I didn't know about ipsets. It make this so much easier.
.htaccess (its possible in Apache to rename it) is inefficient and
suitable as a
second-rate solution when you are using a hosted service and lack full
control
of the server. VPSs are cheap and a better alternative to hosted mail and web.
On my servers (C5 and C6) in IP Tables, I have three sets of blockages:
- permanent for all ports
- only for web (port 80)
- only for emails (port 25)
In web and emails there is a permanent table plus a monthly one (one for every month). Perpetual pests go in the permanent tables and irritants in
the
monthly table - otherwise the banned IPs entries would get too large.
A compromised computer trying to send me junk mail or trying to wrongly access a web page or attempting to break-in to SQL (instantly identified
and
IP instantly blocked because I impose string size limits for the
?key=....) has
its IP added to the monthly list and remains there until one month after
the
last access from that IP address.
I am unwilling to be a passive victim of junk mail and web hackers.
[Thomas E Dukes] Same here!!
All home-made solutions but effective and robust. Centos made all this possible (sincere thanks to the C-Team; they are all 'A*' rated).
[Thomas E Dukes] Ditto!!
Thanks!!
-
Albert McCann -
Always Learning -
Arun Khan -
Kahlil Hodgson -
Keith Keller -
TE Dukes